From patchwork Tue Nov 25 16:28:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25202 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B950BC333C for ; Tue, 25 Nov 2025 16:30:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D84D60B33; Tue, 25 Nov 2025 17:30:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Uu/gG+yM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 571C560A9E for ; Tue, 25 Nov 2025 17:30:03 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 638636AF; Tue, 25 Nov 2025 17:27:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088074; bh=wKpcWJscDWlpaAipJQxokq1/WoIZu0thGMePWX1d/xQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Uu/gG+yMmUuHJl7VUljQpB4CrIPKJKP/c2T/v4hZ4n+6Th2UAqnB5cwdL0ACgQJnw rxmire0szVGhaB2MCeWXtCoHqWUC6HOMwGUmvR7wKTo8PNSGAho3sxJslBjhIaii0G RXMOyrYvP58NiuxvrVIqlj/stm0j5awA/DewGFmA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 23/29] libcamera: rkisp1: Handle requested orientation using dewarper Date: Tue, 25 Nov 2025 17:28:35 +0100 Message-ID: <20251125162851.2301793-24-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is present it can handle arbitrary orientations specified in the requested camera configuration. In that case handle all transformations inside the dewarper (even if the sensor supports some of them) because that makes it easier to handle coordinates for lens dewarping inside the dewarper. This complicates the path selection a bit, as for transformations that include a transpose, the format before the dewarper has swapped width/height. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham --- Changes in v3: - Collected tag Changes in v2: - Fix path validation for cases where a transpose is involved --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 89 ++++++++++++++++++------ 1 file changed, 67 insertions(+), 22 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index e3dca8b837e8..746eeffab207 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -593,11 +593,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = Adjusted; } - Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); - if (orientation != requestedOrientation) - status = Adjusted; - /* * Simultaneous capture of raw and processed streams isn't possible. * Let the first stream decide on the type. @@ -622,7 +617,23 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } + /* + * If the dewarper supports orientation adjustments, apply that completely + * there. Even if the sensor supports flips, it is beneficial to do that + * in the dewarper so that lens dewarping happens on the unflipped image + */ + bool transposeAfterIsp = false; bool useDewarper = (data_->canUseDewarper_ && !isRaw); + if (useDewarper) { + combinedTransform_ = orientation / data_->sensor_->mountingOrientation(); + if (!!(combinedTransform_ & Transform::Transpose)) + transposeAfterIsp = true; + } else { + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + } /* * If there are more than one stream in the configuration figure out the @@ -638,12 +649,18 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() /* * Validate the configuration against the desired path and, if the - * platform supports it, the dewarper. + * platform supports it, the dewarper. While iterating over the + * configurations collect the smallest common sensor format. */ + Size accumulatedSensorSize; auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, Stream *stream, Status expectedStatus) { StreamConfiguration tryCfg = cfg; + /* Need to validate the path before the transpose */ + if (transposeAfterIsp) + tryCfg.size.transpose(); + Status ret = path->validate(sensor, sensorConfig, &tryCfg); if (ret == Invalid) return false; @@ -652,6 +669,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() (expectedStatus == Valid && ret == Adjusted)) return false; + Size sensorSize = tryCfg.size; + if (useDewarper) { /* * The dewarper output is independent of the ISP path. @@ -674,6 +693,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() cfg = tryCfg; cfg.setStream(stream); + + accumulatedSensorSize = std::max(accumulatedSensorSize, sensorSize); return true; }; @@ -724,13 +745,6 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return Invalid; } - /* Select the sensor format. */ - Size maxSize; - - for (const StreamConfiguration &cfg : config_) { - maxSize = std::max(maxSize, cfg.size); - } - std::vector mbusCodes; if (isRaw) { @@ -741,7 +755,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() [](const auto &value) { return value.second; }); } - sensorFormat_ = sensor->getFormat(mbusCodes, maxSize, + sensorFormat_ = sensor->getFormat(mbusCodes, accumulatedSensorSize, mainPath->maxResolution()); if (sensorFormat_.size.isNull()) @@ -776,6 +790,22 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (roles.empty()) return config; + Transform transform = Transform::Identity; + Size previewSize = kRkISP1PreviewSize; + bool transposeAfterIsp = false; + if (data->canUseDewarper_) { + transform = Orientation::Rotate0 / data->sensor_->mountingOrientation(); + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + } + + /* + * In case of a transpose transform we need to create a path for the + * transposed size. + */ + if (transposeAfterIsp) + previewSize.transpose(); + /* * As the ISP can't output different color spaces for the main and self * path, pick a sensible default color space based on the role of the @@ -804,7 +834,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Sycc; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::VideoRecording: @@ -812,7 +842,7 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!colorSpace) colorSpace = ColorSpace::Rec709; - size = kRkISP1PreviewSize; + size = previewSize; break; case StreamRole::Raw: @@ -854,6 +884,9 @@ PipelineHandlerRkISP1::generateConfiguration(Camera *camera, if (!cfg.pixelFormat.isValid()) return nullptr; + if (transposeAfterIsp && role != StreamRole::Raw) + cfg.size.transpose(); + cfg.colorSpace = colorSpace; cfg.bufferCount = kRkISP1MinBufferCount; config->addConfiguration(cfg); @@ -876,6 +909,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + const PixelFormat &streamFormat = config->at(0).pixelFormat; + const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); + isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + data->usesDewarper_ = data->canUseDewarper_ && !isRaw_; + + Transform transform = config->combinedTransform(); + bool transposeAfterIsp = false; + if (data->usesDewarper_) { + if (!!(transform & Transform::Transpose)) + transposeAfterIsp = true; + transform = Transform::Identity; + } + /* * Configure the format on the sensor output and propagate it through * the pipeline. @@ -885,10 +931,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (config->sensorConfig) ret = sensor->applyConfiguration(*config->sensorConfig, - config->combinedTransform(), + transform, &format); else - ret = sensor->setFormat(&format, config->combinedTransform()); + ret = sensor->setFormat(&format, transform); if (ret < 0) return ret; @@ -915,10 +961,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << " crop " << inputCrop; Rectangle outputCrop = inputCrop; - const PixelFormat &streamFormat = config->at(0).pixelFormat; - const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); - isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; - data->usesDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -1013,11 +1055,14 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + dewarper_->setTransform(cfg.stream(), config->combinedTransform()); /* * Apply a default scaler crop that keeps the * aspect ratio. */ Size size = cfg.size; + if (transposeAfterIsp) + size.transpose(); size = sensorCrop.size().boundedToAspectRatio(size); ControlList ctrls;