{"id":25202,"url":"https://patchwork.libcamera.org/api/1.1/patches/25202/?format=json","web_url":"https://patchwork.libcamera.org/patch/25202/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251125162851.2301793-24-stefan.klug@ideasonboard.com>","date":"2025-11-25T16:28:35","name":"[v3,23/29] libcamera: rkisp1: Handle requested orientation using dewarper","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"5b20e148316151a07d9ce01e6bd52aa0e8c0d5f9","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/1.1/people/184/?format=json","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/25202/mbox/","series":[{"id":5613,"url":"https://patchwork.libcamera.org/api/1.1/series/5613/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5613","date":"2025-11-25T16:28:12","name":"Full dewarper support on imx8mp","version":3,"mbox":"https://patchwork.libcamera.org/series/5613/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/25202/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/25202/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B950BC333C\n\tfor ;\n\tTue, 25 Nov 2025 16:30:05 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5D84D60B33;\n\tTue, 25 Nov 2025 17:30:05 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 571C560A9E\n\tfor ;\n\tTue, 25 Nov 2025 17:30:03 +0100 (CET)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 638636AF;\n\tTue, 25 Nov 2025 17:27:54 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Uu/gG+yM\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1764088074;\n\tbh=wKpcWJscDWlpaAipJQxokq1/WoIZu0thGMePWX1d/xQ=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=Uu/gG+yMmUuHJl7VUljQpB4CrIPKJKP/c2T/v4hZ4n+6Th2UAqnB5cwdL0ACgQJnw\n\trxmire0szVGhaB2MCeWXtCoHqWUC6HOMwGUmvR7wKTo8PNSGAho3sxJslBjhIaii0G\n\tRXMOyrYvP58NiuxvrVIqlj/stm0j5awA/DewGFmA=","From":"Stefan Klug ","To":"libcamera-devel@lists.libcamera.org","Cc":"Stefan Klug ,\n\tPaul Elder ","Subject":"[PATCH v3 23/29] libcamera: rkisp1: Handle requested orientation\n\tusing dewarper","Date":"Tue, 25 Nov 2025 17:28:35 +0100","Message-ID":"<20251125162851.2301793-24-stefan.klug@ideasonboard.com>","X-Mailer":"git-send-email 2.51.0","In-Reply-To":"<20251125162851.2301793-1-stefan.klug@ideasonboard.com>","References":"<20251125162851.2301793-1-stefan.klug@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"When the dewarper is present it can handle arbitrary orientations\nspecified in the requested camera configuration. In that case handle all\ntransformations inside the dewarper (even if the sensor supports some of\nthem) because that makes it easier to handle coordinates for lens\ndewarping inside the dewarper.\n\nThis complicates the path selection a bit, as for transformations that\ninclude a transpose, the format before the dewarper has swapped\nwidth/height.\n\nSigned-off-by: Stefan Klug \nReviewed-by: Paul Elder \n\n---\n\nChanges in v3:\n- Collected tag\n\nChanges in v2:\n- Fix path validation for cases where a transpose is involved\n---\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 89 ++++++++++++++++++------\n 1 file changed, 67 insertions(+), 22 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex e3dca8b837e8..746eeffab207 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -593,11 +593,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\tstatus = Adjusted;\n \t}\n \n-\tOrientation requestedOrientation = orientation;\n-\tcombinedTransform_ = data_->sensor_->computeTransform(&orientation);\n-\tif (orientation != requestedOrientation)\n-\t\tstatus = Adjusted;\n-\n \t/*\n \t * Simultaneous capture of raw and processed streams isn't possible.\n \t * Let the first stream decide on the type.\n@@ -622,7 +617,23 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t}\n \t}\n \n+\t/*\n+\t * If the dewarper supports orientation adjustments, apply that completely\n+\t * there. Even if the sensor supports flips, it is beneficial to do that\n+\t * in the dewarper so that lens dewarping happens on the unflipped image\n+\t */\n+\tbool transposeAfterIsp = false;\n \tbool useDewarper = (data_->canUseDewarper_ && !isRaw);\n+\tif (useDewarper) {\n+\t\tcombinedTransform_ = orientation / data_->sensor_->mountingOrientation();\n+\t\tif (!!(combinedTransform_ & Transform::Transpose))\n+\t\t\ttransposeAfterIsp = true;\n+\t} else {\n+\t\tOrientation requestedOrientation = orientation;\n+\t\tcombinedTransform_ = data_->sensor_->computeTransform(&orientation);\n+\t\tif (orientation != requestedOrientation)\n+\t\t\tstatus = Adjusted;\n+\t}\n \n \t/*\n \t * If there are more than one stream in the configuration figure out the\n@@ -638,12 +649,18 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t/*\n \t * Validate the configuration against the desired path and, if the\n-\t * platform supports it, the dewarper.\n+\t * platform supports it, the dewarper. While iterating over the\n+\t * configurations collect the smallest common sensor format.\n \t */\n+\tSize accumulatedSensorSize;\n \tauto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path,\n \t\t\t\t Stream *stream, Status expectedStatus) {\n \t\tStreamConfiguration tryCfg = cfg;\n \n+\t\t/* Need to validate the path before the transpose */\n+\t\tif (transposeAfterIsp)\n+\t\t\ttryCfg.size.transpose();\n+\n \t\tStatus ret = path->validate(sensor, sensorConfig, &tryCfg);\n \t\tif (ret == Invalid)\n \t\t\treturn false;\n@@ -652,6 +669,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t (expectedStatus == Valid && ret == Adjusted))\n \t\t\treturn false;\n \n+\t\tSize sensorSize = tryCfg.size;\n+\n \t\tif (useDewarper) {\n \t\t\t/*\n \t\t\t * The dewarper output is independent of the ISP path.\n@@ -674,6 +693,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t\tcfg = tryCfg;\n \t\tcfg.setStream(stream);\n+\n+\t\taccumulatedSensorSize = std::max(accumulatedSensorSize, sensorSize);\n \t\treturn true;\n \t};\n \n@@ -724,13 +745,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\treturn Invalid;\n \t}\n \n-\t/* Select the sensor format. */\n-\tSize maxSize;\n-\n-\tfor (const StreamConfiguration &cfg : config_) {\n-\t\tmaxSize = std::max(maxSize, cfg.size);\n-\t}\n-\n \tstd::vector mbusCodes;\n \n \tif (isRaw) {\n@@ -741,7 +755,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t\t [](const auto &value) { return value.second; });\n \t}\n \n-\tsensorFormat_ = sensor->getFormat(mbusCodes, maxSize,\n+\tsensorFormat_ = sensor->getFormat(mbusCodes, accumulatedSensorSize,\n \t\t\t\t\t mainPath->maxResolution());\n \n \tif (sensorFormat_.size.isNull())\n@@ -776,6 +790,22 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera,\n \tif (roles.empty())\n \t\treturn config;\n \n+\tTransform transform = Transform::Identity;\n+\tSize previewSize = kRkISP1PreviewSize;\n+\tbool transposeAfterIsp = false;\n+\tif (data->canUseDewarper_) {\n+\t\ttransform = Orientation::Rotate0 / data->sensor_->mountingOrientation();\n+\t\tif (!!(transform & Transform::Transpose))\n+\t\t\ttransposeAfterIsp = true;\n+\t}\n+\n+\t/*\n+\t * In case of a transpose transform we need to create a path for the\n+\t * transposed size.\n+\t */\n+\tif (transposeAfterIsp)\n+\t\tpreviewSize.transpose();\n+\n \t/*\n \t * As the ISP can't output different color spaces for the main and self\n \t * path, pick a sensible default color space based on the role of the\n@@ -804,7 +834,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera,\n \t\t\tif (!colorSpace)\n \t\t\t\tcolorSpace = ColorSpace::Sycc;\n \n-\t\t\tsize = kRkISP1PreviewSize;\n+\t\t\tsize = previewSize;\n \t\t\tbreak;\n \n \t\tcase StreamRole::VideoRecording:\n@@ -812,7 +842,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera,\n \t\t\tif (!colorSpace)\n \t\t\t\tcolorSpace = ColorSpace::Rec709;\n \n-\t\t\tsize = kRkISP1PreviewSize;\n+\t\t\tsize = previewSize;\n \t\t\tbreak;\n \n \t\tcase StreamRole::Raw:\n@@ -854,6 +884,9 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera,\n \t\tif (!cfg.pixelFormat.isValid())\n \t\t\treturn nullptr;\n \n+\t\tif (transposeAfterIsp && role != StreamRole::Raw)\n+\t\t\tcfg.size.transpose();\n+\n \t\tcfg.colorSpace = colorSpace;\n \t\tcfg.bufferCount = kRkISP1MinBufferCount;\n \t\tconfig->addConfiguration(cfg);\n@@ -876,6 +909,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n \tif (ret)\n \t\treturn ret;\n \n+\tconst PixelFormat &streamFormat = config->at(0).pixelFormat;\n+\tconst PixelFormatInfo &info = PixelFormatInfo::info(streamFormat);\n+\tisRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW;\n+\tdata->usesDewarper_ = data->canUseDewarper_ && !isRaw_;\n+\n+\tTransform transform = config->combinedTransform();\n+\tbool transposeAfterIsp = false;\n+\tif (data->usesDewarper_) {\n+\t\tif (!!(transform & Transform::Transpose))\n+\t\t\ttransposeAfterIsp = true;\n+\t\ttransform = Transform::Identity;\n+\t}\n+\n \t/*\n \t * Configure the format on the sensor output and propagate it through\n \t * the pipeline.\n@@ -885,10 +931,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n \n \tif (config->sensorConfig)\n \t\tret = sensor->applyConfiguration(*config->sensorConfig,\n-\t\t\t\t\t\t config->combinedTransform(),\n+\t\t\t\t\t\t transform,\n \t\t\t\t\t\t &format);\n \telse\n-\t\tret = sensor->setFormat(&format, config->combinedTransform());\n+\t\tret = sensor->setFormat(&format, transform);\n \n \tif (ret < 0)\n \t\treturn ret;\n@@ -915,10 +961,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n \t\t<< \" crop \" << inputCrop;\n \n \tRectangle outputCrop = inputCrop;\n-\tconst PixelFormat &streamFormat = config->at(0).pixelFormat;\n-\tconst PixelFormatInfo &info = PixelFormatInfo::info(streamFormat);\n-\tisRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW;\n-\tdata->usesDewarper_ = dewarper_ && !isRaw_;\n \n \t/* YUYV8_2X8 is required on the ISP source path pad for YUV output. */\n \tif (!isRaw_)\n@@ -1013,11 +1055,14 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n \t\t\t\tif (ret)\n \t\t\t\t\treturn ret;\n \n+\t\t\t\tdewarper_->setTransform(cfg.stream(), config->combinedTransform());\n \t\t\t\t/*\n \t\t\t\t * Apply a default scaler crop that keeps the\n \t\t\t\t * aspect ratio.\n \t\t\t\t */\n \t\t\t\tSize size = cfg.size;\n+\t\t\t\tif (transposeAfterIsp)\n+\t\t\t\t\tsize.transpose();\n \t\t\t\tsize = sensorCrop.size().boundedToAspectRatio(size);\n \n \t\t\t\tControlList ctrls;\n","prefixes":["v3","23/29"]}