From patchwork Mon Dec 16 15:40:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22347 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BDB29C32DA for ; Mon, 16 Dec 2024 15:41:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B9C967F76; Mon, 16 Dec 2024 16:41:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="htBLNQVp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 424E967F56 for ; Mon, 16 Dec 2024 16:41:49 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 86368675; Mon, 16 Dec 2024 16:41:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363672; bh=qiItFYdakEanEKaQeqo3PKo2CBxmZaBRmguQCKOpQXg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=htBLNQVpNJmErTa6tXQRXcRcTm4r1myHQriNX+9pOB3erc+g6bBCga4zNR3Bk69th 1/PohPkSiKYj6Z84lKcg8ISuwxp4xJeJ/MVElr1FYOIRqQ8ZMs9VV5ygTrSawuOQrD 64Py/ukdEfgBTRg/5yDsb1r0lFDNKqyIfeaS/wmo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v4 01/20] pipeline: rkisp1: Fix scope of dewarper stop() exit action Date: Mon, 16 Dec 2024 16:40:41 +0100 Message-ID: <20241216154124.203650-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move the definition of the dewarper stop() action into the scope were the corresponding start() happens. Fixes: 12b553d691d4 ("libcamera: rkisp1: Plumb the dw100 dewarper as V4L2M2M converter") Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Changed title to pipeline: Changes in v3: - Removed blank line from commit message --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 908724e20975..8b92fdebdd63 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1044,8 +1044,8 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL LOG(RkISP1, Error) << "Failed to start dewarper"; return ret; } + actions += [&]() { dewarper_->stop(); }; } - actions += [&]() { dewarper_->stop(); }; } if (data->mainPath_->isEnabled()) { From patchwork Mon Dec 16 15:40:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22348 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C830DC32DA for ; Mon, 16 Dec 2024 15:41:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 75EF967F76; Mon, 16 Dec 2024 16:41:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ega+78fr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 576E167F6D for ; Mon, 16 Dec 2024 16:41:52 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B2E0B675; Mon, 16 Dec 2024 16:41:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363675; bh=p7dat2hlw29p77nf/2ynDNFPwqBK/iiEVvknL9O2Bh4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ega+78frp6bjLqYhBUV7qmBaNy8A9Iyx1WSdltzNkJbaRhLxQd+Sqdj5COon0I8Fi vTCCjmxIFK40LuJ3H08Kzbjx+E715RuyUzuA50nkz8JQnKaY/yTrtsmhDxf4e6ZUha iL6Q0/NCB2yGNcwEOBmpfESu9HWWcsvs3ygS1vME= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v4 02/20] pipeline: rkisp1: Keep aspect ratio on imx8mp Date: Mon, 16 Dec 2024 16:40:42 +0100 Message-ID: <20241216154124.203650-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the current code, the input stage of the image resizer is used to apply a crop to keep the aspect ratio in cases where the requested output aspect ratio differs from the one of the selected sensor mode. On the imx8mp the resizer hardware is not capable of cropping (for reference see also rkisp1-resizer.c:rkisp1_rsz_set_sink_crop() in the linux kernel v6.10). Therefore apply the necessary cropping on the output of the ISP (on the image stabilization block). The cropping code on the image resizer doesn't need modifications as the requested crop gets ignored by the kernel. While at it, remove a todo comment that is no longer needed. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Changed title to pipeline: Changes in v3: - Dropped info message about the crop Changes in v2: - Fixed alignment - Improved comments - Removed todo comment - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 13 +++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 13 ++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 8b92fdebdd63..698fe9ff57a4 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -812,6 +812,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (!isRaw_) format.code = MEDIA_BUS_FMT_YUYV8_2X8; + /* + * On devices without DUAL_CROP (like the imx8mp) cropping needs to be + * done on the ISP/IS output. + */ + if (media_->hwRevision() == RKISP1_V_IMX8MP) { + /* imx8mp has only a single path. */ + const auto &cfg = config->at(0); + Size ispCrop = format.size.boundedToAspectRatio(cfg.size) + .alignedUpTo(2, 2); + rect = ispCrop.centeredTo(Rectangle(format.size).center()); + format.size = ispCrop; + } + LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format << " crop " << rect; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index 236d05af7a2f..eee5b09e2ff0 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -417,11 +417,14 @@ int RkISP1Path::configure(const StreamConfiguration &config, /* * Crop on the resizer input to maintain FOV before downscaling. * - * \todo The alignment to a multiple of 2 pixels is required but may - * change the aspect ratio very slightly. A more advanced algorithm to - * compute the resizer input crop rectangle is needed, and it should - * also take into account the need to crop away the edge pixels affected - * by the ISP processing blocks. + * Note that this does not apply to devices without DUAL_CROP support + * (like imx8mp) , where the cropping needs to be done on the + * ImageStabilizer block on the ISP source pad and therefore is + * configured before this stage. For simplicity we still set the crop. + * This gets ignored by the kernel driver because the hardware is + * missing the capability. + * + * Alignment to a multiple of 2 pixels is required by the resizer. */ Size ispCrop = inputFormat.size.boundedToAspectRatio(config.size) .alignedUpTo(2, 2); From patchwork Mon Dec 16 15:40:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22349 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7250BC32DA for ; Mon, 16 Dec 2024 15:41:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2EE1367F7A; Mon, 16 Dec 2024 16:41:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="H+SOPuYN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AAB0467F78 for ; Mon, 16 Dec 2024 16:41:55 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 11C3882A; Mon, 16 Dec 2024 16:41:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363679; bh=LnhbRJbhNTkk7xb9hRaSCNoSo/GtePhhHMNJe9ULUfM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H+SOPuYNuLkXlM7daHHwC7kVirh+UkyTM/TmK82ThYSQv9rpw17Ssf6GsDKlMszYv 5HOHTYrKyug4DQQ2jkW+Eh2MiVGiPXqoFhpWoFMAHvmx5UPqyf80bDQi2mh0tkVLbj indJjfxXJVi9KovcYlALpBBA1W8QjVxyRtM7kPgE= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v4 03/20] libcamera: geometry: Add Rectangle::transformedBetween() Date: Mon, 16 Dec 2024 16:40:43 +0100 Message-ID: <20241216154124.203650-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handling cropping and scaling within a complicated pipeline involves transformations of rectangles between different coordinate systems. For example the full input of the dewarper (0,0)/1920x1080 might correspond to the rectangle (0, 243)/2592x1458 in sensor coordinates (of a 2592x1944 sensor). Add a function that allows the transformation of a rectangle defined in one reference frame (dewarper) into the coordinates of a second reference frame (sensor). Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v3: - Improved documentation - Added testcase Changes in v2: - Renamed mappedBetween() to transformedBetween() - Improved documentation - Collected tags --- include/libcamera/geometry.h | 3 +++ src/libcamera/geometry.cpp | 49 ++++++++++++++++++++++++++++++++++++ test/geometry.cpp | 11 ++++++++ 3 files changed, 63 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index 9ca5865a3d0d..e5f0a843d314 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -299,6 +299,9 @@ public: __nodiscard Rectangle scaledBy(const Size &numerator, const Size &denominator) const; __nodiscard Rectangle translatedBy(const Point &point) const; + + Rectangle transformedBetween(const Rectangle &source, + const Rectangle &target) const; }; bool operator==(const Rectangle &lhs, const Rectangle &rhs); diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 90ccf8c19f97..81cc8cd538a0 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -837,6 +837,55 @@ Rectangle Rectangle::translatedBy(const Point &point) const return { x + point.x, y + point.y, width, height }; } +/** + * \brief Transform a Rectangle from one reference rectangle to another + * \param[in] source The \a source reference rectangle + * \param[in] destination The \a destination reference rectangle + * + * The \a source and \a destination parameters describe two rectangles defined + * in different reference systems. The Rectangle is translated from the source + * reference system into the destination reference system. + * + * The typical use case for this function is to translate a selection rectangle + * specified in a reference system, in example the sensor's pixel array, into + * the same rectangle re-scaled and translated into a different reference + * system, in example the output frame on which the selection rectangle is + * applied to. + * + * For example, consider a sensor with a resolution of 4040x2360 pixels and a + * assume a rectangle of (100, 100)/3840x2160 (sensorFrame) in sensor + * coordinates is mapped to a rectangle (0,0)/(1920,1080) (displayFrame) in + * display coordinates. This function can be used to transform an arbitrary + * rectangle from display coordinates to sensor coordinates or vice versa: + * + * \code{.cpp} + * Rectangle sensorReference(100, 100, 3840, 2160); + * Rectangle displayReference(0, 0, 1920, 1080); + * + * // Bottom right quarter in sensor coordinates + * Rectangle sensorRect(2020, 100, 1920, 1080); + * displayRect = sensorRect.transformedBetween(sensorReference, displayReference); + * // displayRect is now (960, 540)/960x540 + * + * // Transformation back to sensor coordinates + * sensorRect = displayRect.transformedBetween(displayReference, sensorReference); + * \endcode + */ +Rectangle Rectangle::transformedBetween(const Rectangle &source, + const Rectangle &destination) const +{ + Rectangle r; + double sx = static_cast(destination.width) / source.width; + double sy = static_cast(destination.height) / source.height; + + r.x = static_cast((x - source.x) * sx) + destination.x; + r.y = static_cast((y - source.y) * sy) + destination.y; + r.width = static_cast(width * sx); + r.height = static_cast(height * sy); + + return r; +} + /** * \brief Compare rectangles for equality * \return True if the two rectangles are equal, false otherwise diff --git a/test/geometry.cpp b/test/geometry.cpp index 5760fa3c885a..11df043b733b 100644 --- a/test/geometry.cpp +++ b/test/geometry.cpp @@ -495,6 +495,17 @@ protected: return TestFail; } + Rectangle f1 = Rectangle(100, 200, 3000, 2000); + Rectangle f2 = Rectangle(200, 300, 1500, 1000); + /* Bottom right quarter of the corresponding frames. */ + Rectangle r1 = Rectangle(100 + 1500, 200 + 1000, 1500, 1000); + Rectangle r2 = Rectangle(200 + 750, 300 + 500, 750, 500); + if (r1.transformedBetween(f1, f2) != r2 || + r2.transformedBetween(f2, f1) != r1) { + cout << "Rectangle::transformedBetween() test failed" << endl; + return TestFail; + } + return TestPass; } }; From patchwork Mon Dec 16 15:40:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22350 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4730AC32F6 for ; Mon, 16 Dec 2024 15:42:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED33367F79; Mon, 16 Dec 2024 16:42:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="j/hUrG04"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 783C167F71 for ; Mon, 16 Dec 2024 16:41:58 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CB51B82A; Mon, 16 Dec 2024 16:41:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363681; bh=xnJdSZqd3w9Pk4CsD0ehBtbeHtKdBC2kmTVsoAB1JF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j/hUrG049/hzF2ue/AiypwhfEbPPL8Ssrxj4OpmuDAFH48OtMQcpBzjt70pwt1tKk XCi2fNsp67H4v5J3xcfWKH/cXmYRRTdjEgvq59OH7qe60JSA7DALA0MaHl2oB0GA0p 7+3RH9QdXmENnjMvXU26x2wWhjBWsg0z9LtSeBzM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v4 04/20] pipeline: rkisp1: Split inputCrop and outputCrop Date: Mon, 16 Dec 2024 16:40:44 +0100 Message-ID: <20241216154124.203650-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" One Rectangle instance is used to calculate the inputCrop and the outputCrop of the ISP in the rkisp1 pipeline. Split that into two distinct variables, because both values will be needed in the upcoming patches. This patch does not contain any functional changes. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v2: - Moved declaration of outputCrop after setSelection(inputCrop) to get the potentially updated inputCrop - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 698fe9ff57a4..6ac14cd15c8f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -795,15 +795,16 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret < 0) return ret; - Rectangle rect(0, 0, format.size); - ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &rect); + Rectangle inputCrop(0, 0, format.size); + ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &inputCrop); if (ret < 0) return ret; LOG(RkISP1, Debug) << "ISP input pad configured with " << format - << " crop " << rect; + << " crop " << inputCrop; + Rectangle outputCrop = inputCrop; const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; @@ -821,15 +822,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const auto &cfg = config->at(0); Size ispCrop = format.size.boundedToAspectRatio(cfg.size) .alignedUpTo(2, 2); - rect = ispCrop.centeredTo(Rectangle(format.size).center()); + outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; } LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format - << " crop " << rect; + << " crop " << outputCrop; - ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &rect); + ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &outputCrop); if (ret < 0) return ret; @@ -840,7 +841,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "ISP output pad configured with " << format - << " crop " << rect; + << " crop " << outputCrop; std::map streamConfig; std::vector> outputCfgs; From patchwork Mon Dec 16 15:40:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22351 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 40E85C32F6 for ; Mon, 16 Dec 2024 15:42:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AE9D267F7B; Mon, 16 Dec 2024 16:42:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SvAXYr63"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E954D67F77 for ; Mon, 16 Dec 2024 16:42:00 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 43674675; Mon, 16 Dec 2024 16:41:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363684; bh=jIPjtC4cpuUaIKD7hKyB2rwwweHCDzqAtCd7Oa7ZKQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SvAXYr63JmRTeI7rlSDdY51MpbaXBTX8W2ruuLp9ZElQJsklRvwh3a06ernAKr93A FPDM9j2EnjQr4lB9CnimFF29YDvpGUs6Hs1iqvBBjY3uJSd9Y+i2d1lvPsn84SYPWR dbzV+mjNgX7oPpc1MtL4GjIv4k6kkb9zWlbJKJNw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v4 05/20] pipeline: rkisp1: Reorder sensorInfo collection code Date: Mon, 16 Dec 2024 16:40:45 +0100 Message-ID: <20241216154124.203650-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The sensorInfo (specifically the crop rectangle of the selected sensor mode) is collected to be passed to the IPA later. In an upcoming patch that data will also be needed for correct ScalerCrop handling. Move the collection of the sensorInfo before the dewarper configuration step and refactor the code a bit. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v2: - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6ac14cd15c8f..89946b782854 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -843,6 +843,11 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << "ISP output pad configured with " << format << " crop " << outputCrop; + IPACameraSensorInfo sensorInfo; + ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) + return ret; + std::map streamConfig; std::vector> outputCfgs; @@ -882,14 +887,9 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* Inform IPA of stream configuration and sensor controls. */ - ipa::rkisp1::IPAConfigInfo ipaConfig{}; - - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); - if (ret) - return ret; - - ipaConfig.sensorControls = data->sensor_->controls(); - ipaConfig.paramFormat = paramFormat.fourcc; + ipa::rkisp1::IPAConfigInfo ipaConfig{ sensorInfo, + data->sensor_->controls(), + paramFormat.fourcc }; ret = data->ipa_->configure(ipaConfig, streamConfig, &data->ipaControls_); if (ret) { From patchwork Mon Dec 16 15:40:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22352 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CBC1C32F6 for ; Mon, 16 Dec 2024 15:42:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 736FB67F81; Mon, 16 Dec 2024 16:42:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iwrSbliX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E83B67F79 for ; Mon, 16 Dec 2024 16:42:03 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 42F7B675; Mon, 16 Dec 2024 16:41:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363687; bh=OHbDWZzgC0bwMfQ1fIsCR9z8ra9VisRmR4TaKweDCr8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iwrSbliXCcA/kodFCI/R7xt2RxtIZpjZ+Hhvlui5v7AsqswfT9Ys61T1awE2tutkB 2ogWb4qfFrXxda2cyQuFPQMMkpOx7DFpg3s78mSbrFZMrzh/jjd1A5UwA1EtrKpsfe 8A2uG1yecKr5Q+2MRgEcWH6dAu/th2HDKnHuAlOg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 06/20] libcamera: converter_v4l2_m2m: Add missing override specifier Date: Mon, 16 Dec 2024 16:40:46 +0100 Message-ID: <20241216154124.203650-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation to adding new functions and overrides to the V4L2M2MDevice class add an override specifier to all overridden functions in the class. This prevents -Winconsistent-missing-override warnings. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v4: - Added this patch --- .../internal/converter/converter_v4l2_m2m.h | 25 ++++++++++--------- 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 0bc0d053e2c4..9b8e43ff0b91 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -38,28 +38,29 @@ class V4L2M2MConverter : public Converter public: V4L2M2MConverter(MediaDevice *media); - int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } - bool isValid() const { return m2m_ != nullptr; } + int loadConfiguration([[maybe_unused]] const std::string &filename) override { return 0; } + bool isValid() const override { return m2m_ != nullptr; } - std::vector formats(PixelFormat input); - SizeRange sizes(const Size &input); + std::vector formats(PixelFormat input) override; + SizeRange sizes(const Size &input) override; std::tuple - strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size); + strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size) override; int configure(const StreamConfiguration &inputCfg, - const std::vector> &outputCfg); + const std::vector> + &outputCfg) override; int exportBuffers(const Stream *stream, unsigned int count, - std::vector> *buffers); + std::vector> *buffers) override; - int start(); - void stop(); + int start() override; + void stop() override; int queueBuffers(FrameBuffer *input, - const std::map &outputs); + const std::map &outputs) override; - int setInputCrop(const Stream *stream, Rectangle *rect); - std::pair inputCropBounds(const Stream *stream); + int setInputCrop(const Stream *stream, Rectangle *rect) override; + std::pair inputCropBounds(const Stream *stream) override; private: class V4L2M2MStream : protected Loggable From patchwork Mon Dec 16 15:40:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22353 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5978AC32F6 for ; Mon, 16 Dec 2024 15:42:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED34567F84; Mon, 16 Dec 2024 16:42:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VIzl1Cog"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9DF3D67F82 for ; Mon, 16 Dec 2024 16:42:06 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 01C5C675; Mon, 16 Dec 2024 16:41:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363690; bh=Qp9v4uVEmgzrgQ1G3JWjCc4L7NADhMw2eBuHSQWMJXI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VIzl1Coga1ngd31S6wtezGFUwVvWj628kAcNacjyPT0riX80NXngDuPx870LH/S1q Ipq7+ZeDeLwBNDgE5hsSt0ruA1d0R1h9prc67D+WrQoklmAJKSHlrtLQ3oluRZ7KZ/ qG3AnScI3ImumQLM44UMJDecYaUobPhyXcTLfqqQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 07/20] libcamera: converter_v4l2_m2m: Refactor get crop bounds code Date: Mon, 16 Dec 2024 16:40:47 +0100 Message-ID: <20241216154124.203650-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In an upcoming patch it is necessary to get the crop bounds on the converter itself. The V4L2M2MConverter contains code that is very similar to the get crop bounds code in the V4L2M2MStream. Merge these code blocks into a static function to be used from both classes. This patch contains no functional changes. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v4: - Split patch from libcamera: converter_v4l2_m2m: Improve crop bounds support --- .../converter/converter_v4l2_m2m.cpp | 106 +++++++++--------- 1 file changed, 52 insertions(+), 54 deletions(-) diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index d63ef2f8028f..8c341fe199f6 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -30,6 +30,52 @@ namespace libcamera { LOG_DECLARE_CATEGORY(Converter) +namespace { + +int getCropBounds(V4L2VideoDevice *device, Rectangle &minCrop, + Rectangle &maxCrop) +{ + Rectangle minC; + Rectangle maxC; + + /* Find crop bounds */ + minC.width = 1; + minC.height = 1; + maxC.width = UINT_MAX; + maxC.height = UINT_MAX; + + int ret = device->setSelection(V4L2_SEL_TGT_CROP, &minC); + if (ret) { + LOG(Converter, Error) + << "Could not query minimum selection crop: " + << strerror(-ret); + return ret; + } + + ret = device->getSelection(V4L2_SEL_TGT_CROP_BOUNDS, &maxC); + if (ret) { + LOG(Converter, Error) + << "Could not query maximum selection crop: " + << strerror(-ret); + return ret; + } + + /* Reset the crop to its maximum */ + ret = device->setSelection(V4L2_SEL_TGT_CROP, &maxC); + if (ret) { + LOG(Converter, Error) + << "Could not reset selection crop: " + << strerror(-ret); + return ret; + } + + minCrop = minC; + maxCrop = maxC; + return 0; +} + +} /* namespace */ + /* ----------------------------------------------------------------------------- * V4L2M2MConverter::V4L2M2MStream */ @@ -98,41 +144,10 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC outputBufferCount_ = outputCfg.bufferCount; if (converter_->features() & Feature::InputCrop) { - Rectangle minCrop; - Rectangle maxCrop; - - /* Find crop bounds */ - minCrop.width = 1; - minCrop.height = 1; - maxCrop.width = UINT_MAX; - maxCrop.height = UINT_MAX; - - ret = setInputSelection(V4L2_SEL_TGT_CROP, &minCrop); - if (ret) { - LOG(Converter, Error) - << "Could not query minimum selection crop: " - << strerror(-ret); - return ret; - } - - ret = getInputSelection(V4L2_SEL_TGT_CROP_BOUNDS, &maxCrop); - if (ret) { - LOG(Converter, Error) - << "Could not query maximum selection crop: " - << strerror(-ret); + ret = getCropBounds(m2m_->output(), inputCropBounds_.first, + inputCropBounds_.second); + if (ret) return ret; - } - - /* Reset the crop to its maximum */ - ret = setInputSelection(V4L2_SEL_TGT_CROP, &maxCrop); - if (ret) { - LOG(Converter, Error) - << "Could not reset selection crop: " - << strerror(-ret); - return ret; - } - - inputCropBounds_ = { minCrop, maxCrop }; } return 0; @@ -258,27 +273,10 @@ V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) return; } - /* Discover Feature::InputCrop */ + Rectangle minCrop; Rectangle maxCrop; - maxCrop.width = UINT_MAX; - maxCrop.height = UINT_MAX; - - ret = m2m_->output()->setSelection(V4L2_SEL_TGT_CROP, &maxCrop); - if (ret) - return; - - /* - * Rectangles for cropping targets are defined even if the device - * does not support cropping. Their sizes and positions will be - * fixed in such cases. - * - * Set and inspect a crop equivalent to half of the maximum crop - * returned earlier. Use this to determine whether the crop on - * input is really supported. - */ - Rectangle halfCrop(maxCrop.size() / 2); - ret = m2m_->output()->setSelection(V4L2_SEL_TGT_CROP, &halfCrop); - if (!ret && halfCrop != maxCrop) { + ret = getCropBounds(m2m_->output(), minCrop, maxCrop); + if (!ret && minCrop != maxCrop) { features_ |= Feature::InputCrop; LOG(Converter, Info) From patchwork Mon Dec 16 15:40:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22354 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7F619C32F6 for ; Mon, 16 Dec 2024 15:42:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E32067F89; Mon, 16 Dec 2024 16:42:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PZR4uHot"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B809367F7C for ; Mon, 16 Dec 2024 16:42:08 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2B8D082A; Mon, 16 Dec 2024 16:41:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363692; bh=A2hoAAODKjXuMBejnr8+ldG7/o0OpNQqa305QPLDEik=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PZR4uHotGOFMjpg6BHN5zbAhKOgb2OeB3XkthZlRc42otnLhTDPFKLEdrYv1cBVYt GI0jvEE/9FPr8Zp1ynO1ZINYaYHACJzGH9aaoDd6FKLbJKVuDFJ0WJ2IKD2lBah749 j3nGHjXGjBOse63k8WD0nGWHe1WDOEY3iq46tWvA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 08/20] libcamera: converter: Add function to query crop bounds Date: Mon, 16 Dec 2024 16:40:48 +0100 Message-ID: <20241216154124.203650-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The inputCropBounds_ member of the V4L2M2MConverter::Stream class is only initialized after a V4L2M2MConverter::configure() call, when the streams are initialized. However, the converter has crop limits that do not depend on the configured Streams, and which are currently not accessible from the class interface. Add a new inputCropBounds() function to the V4L2M2MConverter class that allows to retrieve the converter crop limits before any stream is configured. This is particularly useful for pipelines to initialize controls and properties and to implement validation before the Camera gets configured. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Split off from libcamera: converter_v4l2_m2m: Improve crop bounds support - Added separate inputCropBounds() function to the dewarper class Fuxup inputCrop() --- include/libcamera/internal/converter.h | 1 + .../internal/converter/converter_v4l2_m2m.h | 2 ++ src/libcamera/converter.cpp | 13 +++++++++++++ src/libcamera/converter/converter_v4l2_m2m.cpp | 18 ++++++++++++------ 4 files changed, 28 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index ffbb6f345cd5..04187a2a96e8 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -66,6 +66,7 @@ public: const std::map &outputs) = 0; virtual int setInputCrop(const Stream *stream, Rectangle *rect) = 0; + virtual std::pair inputCropBounds() = 0; virtual std::pair inputCropBounds(const Stream *stream) = 0; Signal inputBufferReady; diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 9b8e43ff0b91..402a803959b9 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -60,6 +60,7 @@ public: const std::map &outputs) override; int setInputCrop(const Stream *stream, Rectangle *rect) override; + std::pair inputCropBounds() override { return inputCropBounds_; } std::pair inputCropBounds(const Stream *stream) override; private: @@ -106,6 +107,7 @@ private: std::map> streams_; std::map queue_; + std::pair inputCropBounds_; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 3a3f84344c5e..73c02fdcf4bb 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -185,6 +185,16 @@ Converter::~Converter() /** * \fn Converter::inputCropBounds() + * \brief Retrieve the crop bounds of the converter + * + * Retrieve the minimum and maximum crop bounds of the converter. This can be + * used to query the crop bounds before configuring a stream. + * + * \return A pair containing the minimum and maximum crop bound in that order + */ + +/** + * \fn Converter::inputCropBounds(const Stream *stream) * \brief Retrieve the crop bounds for \a stream * \param[in] stream The output stream * @@ -195,6 +205,9 @@ Converter::~Converter() * this function should be called after the \a stream has been configured using * configure(). * + * When called with an unconfigured \a stream, this function returns a pair of + * null rectangles. + * * \return A pair containing the minimum and maximum crop bound in that order */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 8c341fe199f6..342aa32dab52 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -273,10 +273,9 @@ V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) return; } - Rectangle minCrop; - Rectangle maxCrop; - ret = getCropBounds(m2m_->output(), minCrop, maxCrop); - if (!ret && minCrop != maxCrop) { + ret = getCropBounds(m2m_->output(), inputCropBounds_.first, + inputCropBounds_.second); + if (!ret && inputCropBounds_.first != inputCropBounds_.second) { features_ |= Feature::InputCrop; LOG(Converter, Info) @@ -469,14 +468,21 @@ int V4L2M2MConverter::setInputCrop(const Stream *stream, Rectangle *rect) } /** - * \copydoc libcamera::Converter::inputCropBounds + * \fn libcamera::V4L2M2MConverter::inputCropBounds() + * \copydoc libcamera::Converter::inputCropBounds() + */ + +/** + * \copydoc libcamera::Converter::inputCropBounds(const Stream *stream) */ std::pair V4L2M2MConverter::inputCropBounds(const Stream *stream) { auto iter = streams_.find(stream); - if (iter == streams_.end()) + if (iter == streams_.end()) { + LOG(Converter, Error) << "Invalid output stream"; return {}; + } return iter->second->inputCropBounds(); } From patchwork Mon Dec 16 15:40:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22355 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 494C5C32FB for ; Mon, 16 Dec 2024 15:42:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C75BC67F8A; Mon, 16 Dec 2024 16:42:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CN2/oJLJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A04DF67F81 for ; Mon, 16 Dec 2024 16:42:11 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F3D6E82A; Mon, 16 Dec 2024 16:41:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363695; bh=WLz7itERtgpZabbFOsaQnEE+iI6KY8pZaV0GPVV/gv4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CN2/oJLJZxRgcVTlGHpSSH8sUcqu45TJn1PMH1diJeTuV9fYaQdH9tjtGvSiyuKY3 URPqocdYl5hIJmM4wKlr30rIpNOyE4xdgS68ldLrqyZpRG9sftu7YGiqYjb9TxtWIb vVMX+oyrfBPvDjAA9drdafItZHWwY7otl8mcI5mg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 09/20] libcamera: converter: Add function to check if a stream was configured Date: Mon, 16 Dec 2024 16:40:49 +0100 Message-ID: <20241216154124.203650-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a isConfigured() function to be able to check if a given stream was configured in the converter. This is useful in pipelines to either query device or stream specific crop bounds depending on whether the stream is configured or not. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v4: - Added this patch --- include/libcamera/internal/converter.h | 1 + include/libcamera/internal/converter/converter_v4l2_m2m.h | 1 + src/libcamera/converter.cpp | 7 +++++++ src/libcamera/converter/converter_v4l2_m2m.cpp | 8 ++++++++ 4 files changed, 17 insertions(+) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index 04187a2a96e8..afea4624041b 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -56,6 +56,7 @@ public: virtual int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs) = 0; + virtual bool isConfigured(const Stream *stream) const = 0; virtual int exportBuffers(const Stream *stream, unsigned int count, std::vector> *buffers) = 0; diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 402a803959b9..1ccbfc7c2d4e 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -50,6 +50,7 @@ public: int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfg) override; + bool isConfigured(const Stream *stream) const override; int exportBuffers(const Stream *stream, unsigned int count, std::vector> *buffers) override; diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 73c02fdcf4bb..09ebf4f57d12 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -126,6 +126,13 @@ Converter::~Converter() * \return 0 on success or a negative error code otherwise */ +/** + * \fn Converter::isConfigured() + * \brief Check if a given stream is configured + * \param[in] stream The output stream + * \return True if the \a stream is configured or false otherwise + */ + /** * \fn Converter::exportBuffers() * \brief Export buffers from the converter device diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 342aa32dab52..83daca15b37e 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -437,6 +437,14 @@ int V4L2M2MConverter::configure(const StreamConfiguration &inputCfg, return 0; } +/** + * \copydoc libcamera::Converter::isConfigured + */ +bool V4L2M2MConverter::isConfigured(const Stream *stream) const +{ + return streams_.find(stream) != streams_.end(); +} + /** * \copydoc libcamera::Converter::exportBuffers */ From patchwork Mon Dec 16 15:40:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22356 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5516FC32F6 for ; Mon, 16 Dec 2024 15:42:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 091C467F8D; Mon, 16 Dec 2024 16:42:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KaQ5biGI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D0B3967F8B for ; Mon, 16 Dec 2024 16:42:13 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 445EE675; Mon, 16 Dec 2024 16:41:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363697; bh=ndmsJHnTDuyPLgB4S2I/qzI52TKDwhBOhXbSsAeTVd0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KaQ5biGIJe5b6aBHVHSoX6InDR8/Mm71nKbENTXpAWIhIHvxE8+u+mMm7YVAE4Qjk QHeBGI6eiT0v5bdD4O6SLFujkg5aYM2+EkaR0FAQmDLi99UO5ywKI6MUkB9idIrj// TPe1JlJoSB2bIbDG+n/BfT8YxglPYXJ4Di9llqnM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 10/20] pipeline: rkisp1: Query dewarper crop bounds if no stream configured Date: Mon, 16 Dec 2024 16:40:50 +0100 Message-ID: <20241216154124.203650-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Query the crop bounds on the dewarper instead of the stream in case the camera was not yet configured when updateControls() gets called. This provides sane defaults for the controls. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Split patch from libcamera: converter_v4l2_m2m: Improve crop bounds support --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 89946b782854..56192451eb3c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1220,8 +1220,11 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) ControlInfoMap::Map controls; if (dewarper_) { - std::pair cropLimits = - dewarper_->inputCropBounds(&data->mainPathStream_); + std::pair cropLimits; + if (dewarper_->isConfigured(&data->mainPathStream_)) + cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); + else + cropLimits = dewarper_->inputCropBounds(); controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, cropLimits.second, From patchwork Mon Dec 16 15:40:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22357 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4506BC32F6 for ; Mon, 16 Dec 2024 15:42:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C33A267F8D; Mon, 16 Dec 2024 16:42:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Cs8H6p6K"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A7F3767F86 for ; Mon, 16 Dec 2024 16:42:16 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1AA24675; Mon, 16 Dec 2024 16:41:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363700; bh=6Hr8yXWTAtdEg8LArbI/Jijc7dM/magl3pw4ax8C1Y0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cs8H6p6KsHyQOHHTlwci83btWJzjRcwbNycjmJuX+XCRGTm0ex/Llj5IZdA5UT1fY afXCMIFnzIixbDX7ddcM38UnQ3fk+KiQACTumEfkAOiJeKb/sfssnCMAuh3x67e+H4 o21dCDu0bU6FAH8a8ebFUmtLODiCdsej4wOwGvzw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v4 11/20] pipeline: rkisp1: Fix ScalerCrop to be in sensor coordinates Date: Mon, 16 Dec 2024 16:40:51 +0100 Message-ID: <20241216154124.203650-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" ScalerCrop is specified as being in sensor coordinates. The current dewarper implementation on the imx8mp handles ScalerCrop in dewarper coordinates. This leads to unexpected results and an unusable ScalerCrop control in camshark. Fix that by transforming back and forth between sensor coordinates and dewarper coordinates. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v3: - Rename dewarpSensorCrop_ to scalerMaxCrop_ - Remove unnecessary ScalerCrop max calculation Changes in v2: - Initialize dewarperSensorCrop_ to sane defaults - Collect tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 50 +++++++++++++++++++----- 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 56192451eb3c..ef4aa38478f5 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -205,6 +205,7 @@ private: RkISP1SelfPath selfPath_; std::unique_ptr dewarper_; + Rectangle scalerMaxCrop_; bool useDewarper_; std::optional activeCrop_; @@ -861,6 +862,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); useDewarper_ = ret ? false : true; + + /* + * Calculate the crop rectangle of the data + * flowing into the dewarper in sensor + * coordinates. + */ + scalerMaxCrop_ = + outputCrop.transformedBetween(inputCrop, + sensorInfo.analogCrop); } } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); @@ -1226,10 +1236,19 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) else cropLimits = dewarper_->inputCropBounds(); - controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, - cropLimits.second, - cropLimits.second); - activeCrop_ = cropLimits.second; + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform the limits to sensor coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ + Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, + scalerMaxCrop_); + + controls[&controls::ScalerCrop] = ControlInfo(min, + scalerMaxCrop_, + scalerMaxCrop_); + activeCrop_ = scalerMaxCrop_; } /* Add the IPA registered controls to list of camera controls. */ @@ -1257,6 +1276,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); + scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); + const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1476,22 +1497,33 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { - Rectangle appliedRect = crop.value(); + Rectangle rect = crop.value(); + + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform it into dewarper coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ + std::pair cropLimits = + dewarper_->inputCropBounds(&data->mainPathStream_); + rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &appliedRect); - if (!ret && appliedRect != crop.value()) { + &rect); + rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); + if (!ret && rect != crop.value()) { /* * If the rectangle is changed by setInputCrop on the * dewarper, log a debug message and cache the actual * applied rectangle for metadata reporting. */ LOG(RkISP1, Debug) - << "Applied rectangle " << appliedRect.toString() + << "Applied rectangle " << rect.toString() << " differs from requested " << crop.value().toString(); } - activeCrop_ = appliedRect; + activeCrop_ = rect; } /* From patchwork Mon Dec 16 15:40:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22358 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 48A12C32F6 for ; Mon, 16 Dec 2024 15:42:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EFCDE67F91; Mon, 16 Dec 2024 16:42:22 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ge+rPnnC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9117867F89 for ; Mon, 16 Dec 2024 16:42:19 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 016F6675; Mon, 16 Dec 2024 16:41:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363703; bh=KS+swCWwgQ+GxyE65Yd6uWSlQ4m0+yF3TC/c8mw9RJk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ge+rPnnC18gGktUrmmGH+4MGl/6KU3yLh75/NKXWyP70Np+B+90ZHT3b1mD6Tq/GO 1UkU447YGbSIcAVzQmY6ehpe+77CFdmAUiMjrGWNe2+ZufjPMYQLFFb9zMlK0jcgWq ldnwNWCG4VOClvHUny+JZiHedWwR4hcuWuIKz9dA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v4 12/20] pipeline: rkisp1: Add ScalerMaximumCrop property Date: Mon, 16 Dec 2024 16:40:52 +0100 Message-ID: <20241216154124.203650-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ScalerMaximumCrop property holds the biggest allowed ScalerCrop value. Add it to the rkisp1. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Collected tags Changes in v2: - Moved one hunk to the correct patch 6/7 --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index ef4aa38478f5..abe2dce0024d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1248,6 +1249,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) controls[&controls::ScalerCrop] = ControlInfo(min, scalerMaxCrop_, scalerMaxCrop_); + data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); activeCrop_ = scalerMaxCrop_; } From patchwork Mon Dec 16 15:40:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22359 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 657CBC32F6 for ; Mon, 16 Dec 2024 15:42:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DD5F367F93; Mon, 16 Dec 2024 16:42:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pJDS9ko1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 371FC67F8B for ; Mon, 16 Dec 2024 16:42:22 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 994D782A; Mon, 16 Dec 2024 16:41:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363705; bh=FhxnPk/5CqUsH6IMDYtk9umBD0796wjt/SU4zUBTu/g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pJDS9ko1Y82yTBAUFBsYLCRDzkwHgwj7Nswh0aS4e0RzPD9A1nbPQ505cnuG6X3D2 3K+nxEUPCtaBIGspivJftJCnX4GFe+bKLMWAPZrfdMvjA+VWxA13G6nbEYVV35d5B8 YCu4KqHbHQwNMY89f/ML9dwGRqjWe5IrwKdcc1eY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Stefan Klug , Paul Elder Subject: [PATCH v4 13/20] libcamera: converter: Add functions to adjust config Date: Mon, 16 Dec 2024 16:40:53 +0100 Message-ID: <20241216154124.203650-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add to the Converter interface two functions used by pipeline handlers to validate and adjust the converter input and output configurations by specifying the desired alignment for the adjustment. Add the adjustInputSize() and adjustOutputSize() functions that allows to adjust the converter input/output sizes with the desired alignment. Add a validateOutput() function meant to be used by the pipeline handler implementations of validate(). The function adjusts a StreamConfiguration to a valid configuration produced by the Converter. Signed-off-by: Jacopo Mondi Reviewed-by: Stefan Klug Reviewed-by: Paul Elder --- Changes in v4: - Added docs for returned null sizes - Replaced alignment flags with alignment enum - Collected tags Changes in v3: - Added this patch Signed-off-by: Jacopo Mondi --- include/libcamera/internal/converter.h | 15 ++ .../internal/converter/converter_v4l2_m2m.h | 11 ++ src/libcamera/converter.cpp | 40 +++++ .../converter/converter_v4l2_m2m.cpp | 169 ++++++++++++++++++ 4 files changed, 235 insertions(+) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index afea4624041b..644ec429bb25 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -41,6 +41,11 @@ public: using Features = Flags; + enum class Alignment { + Down = 0, + Up, + }; + Converter(MediaDevice *media, Features features = Feature::None); virtual ~Converter(); @@ -51,9 +56,19 @@ public: virtual std::vector formats(PixelFormat input) = 0; virtual SizeRange sizes(const Size &input) = 0; + virtual Size adjustInputSize(const PixelFormat &pixFmt, + const Size &size, + Alignment align = Alignment::Down) = 0; + virtual Size adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, + Alignment align = Alignment::Down) = 0; + virtual std::tuple strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size) = 0; + virtual int validateOutput(StreamConfiguration *cfg, bool *adjusted, + Alignment align = Alignment::Down) = 0; + virtual int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs) = 0; virtual bool isConfigured(const Stream *stream) const = 0; diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 1ccbfc7c2d4e..0ad7bf7fdbe2 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -47,6 +47,11 @@ public: std::tuple strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size) override; + Size adjustInputSize(const PixelFormat &pixFmt, + const Size &size, Alignment align = Alignment::Down) override; + Size adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, Alignment align = Alignment::Down) override; + int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfg) override; @@ -57,6 +62,9 @@ public: int start() override; void stop() override; + int validateOutput(StreamConfiguration *cfg, bool *adjusted, + Alignment align = Alignment::Down) override; + int queueBuffers(FrameBuffer *input, const std::map &outputs) override; @@ -104,6 +112,9 @@ private: std::pair inputCropBounds_; }; + Size adjustSizes(const Size &size, const std::vector &ranges, + Alignment align); + std::unique_ptr m2m_; std::map> streams_; diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 09ebf4f57d12..d551b908d523 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -50,6 +50,16 @@ LOG_DEFINE_CATEGORY(Converter) * \brief A bitwise combination of features supported by the converter */ +/** + * \enum Converter::Alignment + * \brief The alignment mode specified when adjusting the converter input or + * output sizes + * \var Converter::Alignment::Down + * \brief Adjust the Converter sizes to a smaller valid size + * \var Converter::Alignment::Up + * \brief Adjust the Converter sizes to a larger valid size + */ + /** * \brief Construct a Converter instance * \param[in] media The media device implementing the converter @@ -110,6 +120,26 @@ Converter::~Converter() * \return A range of output image sizes */ +/** + * \fn Converter::adjustInputSize() + * \brief Adjust the converter input \a size to a valid value + * \param[in] pixFmt The pixel format of the converter input stream + * \param[in] size The converter input size to adjust to a valid value + * \param[in] align The desired alignment + * \return The adjusted converter input size or a null Size if \a size cannot + * be adjusted + */ + +/** + * \fn Converter::adjustOutputSize() + * \brief Adjust the converter output \a size to a valid value + * \param[in] pixFmt The pixel format of the converter output stream + * \param[in] size The converter output size to adjust to a valid value + * \param[in] align The desired alignment + * \return The adjusted converter output size or a null Size if \a size cannot + * be adjusted + */ + /** * \fn Converter::strideAndFrameSize() * \brief Retrieve the output stride and frame size for an input configutation @@ -118,6 +148,16 @@ Converter::~Converter() * \return A tuple indicating the stride and frame size or an empty tuple on error */ +/** + * \fn Converter::validateOutput() + * \brief Validate and possibily adjust \a cfg to a valid converter output + * \param[inout] cfg The StreamConfiguration to validate and adjust + * \param[out] adjusted Set to true if \a cfg has been adjusted + * \param[in] align The desired alignment + * \return 0 if \a cfg is valid or has been adjusted, a negative error code + * otherwise if \a cfg cannot be adjusted + */ + /** * \fn Converter::configure() * \brief Configure a set of output stream conversion from an input stream diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index 83daca15b37e..566f18ced51c 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -8,6 +8,7 @@ #include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include #include #include @@ -400,6 +401,127 @@ V4L2M2MConverter::strideAndFrameSize(const PixelFormat &pixelFormat, return std::make_tuple(format.planes[0].bpl, format.planes[0].size); } +/** + * \copydoc libcamera::Converter::adjustInputSize + */ +Size V4L2M2MConverter::adjustInputSize(const PixelFormat &pixFmt, + const Size &size, Alignment align) +{ + auto formats = m2m_->output()->formats(); + V4L2PixelFormat v4l2PixFmt = m2m_->output()->toV4L2PixelFormat(pixFmt); + + auto it = formats.find(v4l2PixFmt); + if (it == formats.end()) { + LOG(Converter, Info) + << "Unsupported pixel format " << pixFmt; + return {}; + } + + return adjustSizes(size, it->second, align); +} + +/** + * \copydoc libcamera::Converter::adjustOutputSize + */ +Size V4L2M2MConverter::adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, Alignment align) +{ + auto formats = m2m_->capture()->formats(); + V4L2PixelFormat v4l2PixFmt = m2m_->capture()->toV4L2PixelFormat(pixFmt); + + auto it = formats.find(v4l2PixFmt); + if (it == formats.end()) { + LOG(Converter, Info) + << "Unsupported pixel format " << pixFmt; + return {}; + } + + return adjustSizes(size, it->second, align); +} + +Size V4L2M2MConverter::adjustSizes(const Size &cfgSize, + const std::vector &ranges, + Alignment align) +{ + Size size = cfgSize; + + if (ranges.size() == 1) { + /* + * The device supports either V4L2_FRMSIZE_TYPE_CONTINUOUS or + * V4L2_FRMSIZE_TYPE_STEPWISE. + */ + const SizeRange &range = *ranges.begin(); + + size.width = std::clamp(size.width, range.min.width, + range.max.width); + size.height = std::clamp(size.height, range.min.height, + range.max.height); + + /* + * Check if any alignment is needed. If the sizes are already + * aligned, or the device supports V4L2_FRMSIZE_TYPE_CONTINUOUS + * with hStep and vStep equal to 1, we're done here. + */ + int widthR = size.width % range.hStep; + int heightR = size.height % range.vStep; + + /* Align up or down according to the caller request. */ + + if (widthR != 0) + size.width = size.width - widthR + + ((align == Alignment::Up) ? range.hStep : 0); + + if (heightR != 0) + size.height = size.height - heightR + + ((align == Alignment::Up) ? range.vStep : 0); + } else { + /* + * The device supports V4L2_FRMSIZE_TYPE_DISCRETE, find the + * size closer to the requested output configuration. + * + * The size ranges vector is not ordered, so we sort it first. + * If we align up, start from the larger element. + */ + std::vector sizes(ranges.size()); + std::transform(ranges.begin(), ranges.end(), std::back_inserter(sizes), + [](const SizeRange &range) { return range.max; }); + std::sort(sizes.begin(), sizes.end()); + + if (align == Alignment::Up) + std::reverse(sizes.begin(), sizes.end()); + + /* + * Return true if s2 is valid according to the desired + * alignment: smaller than s1 if we align down, larger than s1 + * if we align up. + */ + auto nextSizeValid = [](const Size &s1, const Size &s2, Alignment a) { + return a == Alignment::Down + ? (s1.width > s2.width && s1.height > s2.height) + : (s1.width < s2.width && s1.height < s2.height); + }; + + Size newSize; + for (const Size &sz : sizes) { + if (!nextSizeValid(size, sz, align)) + break; + + newSize = sz; + } + + if (newSize.isNull()) { + LOG(Converter, Error) + << "Cannot adjust " << cfgSize + << " to a supported converter size"; + return {}; + } + + size = newSize; + } + + return size; +} + /** * \copydoc libcamera::Converter::configure */ @@ -522,6 +644,53 @@ void V4L2M2MConverter::stop() iter.second->stop(); } +/** + * \copydoc libcamera::Converter::validateOutput + */ +int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, + Alignment align) +{ + V4L2VideoDevice *capture = m2m_->capture(); + V4L2VideoDevice::Formats fmts = capture->formats(); + + if (adjusted) + *adjusted = false; + + PixelFormat fmt = cfg->pixelFormat; + V4L2PixelFormat v4l2PixFmt = capture->toV4L2PixelFormat(fmt); + + auto it = fmts.find(v4l2PixFmt); + if (it == fmts.end()) { + it = fmts.begin(); + v4l2PixFmt = it->first; + cfg->pixelFormat = v4l2PixFmt.toPixelFormat(); + + if (adjusted) + *adjusted = true; + + LOG(Converter, Info) + << "Converter output pixel format adjusted to " + << cfg->pixelFormat; + } + + const Size cfgSize = cfg->size; + cfg->size = adjustSizes(cfgSize, it->second, align); + + if (cfg->size.isNull()) + return -EINVAL; + + if (cfg->size.width != cfgSize.width || + cfg->size.height != cfgSize.height) { + LOG(Converter, Info) + << "Converter size adjusted to " + << cfg->size; + if (adjusted) + *adjusted = true; + } + + return 0; +} + /** * \copydoc libcamera::Converter::queueBuffers */ From patchwork Mon Dec 16 15:40:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22360 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5975FC32F6 for ; Mon, 16 Dec 2024 15:42:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D737667F93; Mon, 16 Dec 2024 16:42:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="D562BQ7b"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 933E467F89 for ; Mon, 16 Dec 2024 16:42:24 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 028FC82A; Mon, 16 Dec 2024 16:41:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363708; bh=XQCiSH9/f2fMfG8OC9CGyFb3yQfO39oUFMgZMLC0aYY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D562BQ7besMaqT+Q0XwqVrFg+ktXFAM8DOqRmQuUudyFrdbw3miz0mSn4r7u1kM3Z uwNXH0QncA5y2P8pJChOdR45pQaqDmPajZiNtfTMDGfIhN3LFA7DOK1lnv5ueT4CeA 7KAB/97Ds9GKT2XPRtVhxdCyy4lVi4NFhmyUNMlk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 14/20] pipeline: rkisp1: Refactor path validation Date: Mon, 16 Dec 2024 16:40:54 +0100 Message-ID: <20241216154124.203650-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Refactor validation code to prepare for extensions in the upcoming patches. Code duplication is reduced by moving parts of the validation logic into a lambda function. This patch does not include any functional changes. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 35 +++++++++++++----------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index abe2dce0024d..18038226912a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -558,50 +558,53 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (config_.size() == 2 && fitsAllPaths(config_[0])) std::reverse(order.begin(), order.end()); + auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, + Stream *stream, Status expectedStatus) { + StreamConfiguration tryCfg = cfg; + if (path->validate(sensor, sensorConfig, &tryCfg) != expectedStatus) + return false; + + cfg = tryCfg; + cfg.setStream(stream); + return true; + }; + bool mainPathAvailable = true; bool selfPathAvailable = data_->selfPath_; + RkISP1Path *mainPath = data_->mainPath_; + RkISP1Path *selfPath = data_->selfPath_; + Stream *mainPathStream = const_cast(&data_->mainPathStream_); + Stream *selfPathStream = const_cast(&data_->selfPathStream_); for (unsigned int index : order) { StreamConfiguration &cfg = config_[index]; /* Try to match stream without adjusting configuration. */ if (mainPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) { + if (validateConfig(cfg, mainPath, mainPathStream, Valid)) { mainPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->mainPathStream_)); continue; } } if (selfPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) { + if (validateConfig(cfg, selfPath, selfPathStream, Valid)) { selfPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->selfPathStream_)); continue; } } /* Try to match stream allowing adjusting configuration. */ if (mainPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) { + if (validateConfig(cfg, mainPath, mainPathStream, Adjusted)) { mainPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->mainPathStream_)); status = Adjusted; continue; } } if (selfPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) { + if (validateConfig(cfg, selfPath, selfPathStream, Adjusted)) { selfPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->selfPathStream_)); status = Adjusted; continue; } From patchwork Mon Dec 16 15:40:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22361 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 493E2C32F6 for ; Mon, 16 Dec 2024 15:42:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E436D67F97; Mon, 16 Dec 2024 16:42:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CtV52Dw8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9173967F89 for ; Mon, 16 Dec 2024 16:42:27 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 03479675; Mon, 16 Dec 2024 16:41:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363711; bh=Cb6ONl86m7aa6iQBw4ziExKImYSlAO2tVJrcmpB5Raw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CtV52Dw8pv2JRtBaR3e+klIivuQZYq5h7m3Z2v98KMkpknmt683Y5bQj96NPkP50D 1ZNusbB+8Jj/xOJo800ikS9MjH30ofoor57PsQ6YkXQgoLiBdeTOpOUkQj2GfDX44b V58Rr1dRFQ/2l3x+IKRKyHKP9zpt3AUWfRAoq540= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 15/20] pipeline: rkisp1: Enable the dewarper unconditionally Date: Mon, 16 Dec 2024 16:40:55 +0100 Message-ID: <20241216154124.203650-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In configure() and in the future in generateConfiguration() the calculated stream sizes and crop rectangles depend on the dewarper being used or not. It is therefore not possible to postpone that decision until the dewarper gets configured. Enable the dewarper unconditionally if it is found and the stream type is not RAW. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 18038226912a..14d4bb9a929b 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -865,7 +865,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (dewarper_ && !isRaw_) { outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); - useDewarper_ = ret ? false : true; + if (ret) + return ret; + + useDewarper_ = true; /* * Calculate the crop rectangle of the data From patchwork Mon Dec 16 15:40:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22362 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 64F52C32F6 for ; Mon, 16 Dec 2024 15:42:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DCFC067F93; Mon, 16 Dec 2024 16:42:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="roIgIrYz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B0B1B67F93 for ; Mon, 16 Dec 2024 16:42:30 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1E59D675; Mon, 16 Dec 2024 16:41:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363714; bh=Y9XUc6pwUUcTXOgIfSoHiwBcpxDYPAnAd21x0kQKGzA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=roIgIrYzZ5VxUL43ofKbjWYoJN8OIa1VLmu2cRoP8dkAcj9yRHSVyQeciaFLttkwq RsSkxKSY3w0GjTu3FO58FnU9iZn+RLr/HUPGpFPlOzOkKEjlYlTZitbEnJVKPj95SY 9IPJedzWbDyLiZHSodo9atM9MxsOcNIFxfwei6zY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 16/20] libcamera: camera: Add a const version of the pipe() function Date: Mon, 16 Dec 2024 16:40:56 +0100 Message-ID: <20241216154124.203650-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow access to the pipeline handler on a const instance of Camera::Private. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v4: - Collected tags - Updated title Changes in v3: - Added this patch --- include/libcamera/internal/camera.h | 1 + src/libcamera/camera.cpp | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/libcamera/internal/camera.h b/include/libcamera/internal/camera.h index 0add0428bb5d..2bb00bbcb644 100644 --- a/include/libcamera/internal/camera.h +++ b/include/libcamera/internal/camera.h @@ -32,6 +32,7 @@ public: ~Private(); PipelineHandler *pipe() { return pipe_.get(); } + const PipelineHandler *pipe() const { return pipe_.get(); } std::list queuedRequests_; ControlInfoMap controlInfo_; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 4c865a46af53..69a7ee5353f1 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -603,6 +603,11 @@ Camera::Private::~Private() * \return The pipeline handler that created this camera */ +/** + * \fn Camera::Private::pipe() const + * \copydoc Camera::Private::pipe() + */ + /** * \fn Camera::Private::validator() * \brief Retrieve the control validator related to this camera From patchwork Mon Dec 16 15:40:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22363 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AFAF2C32F6 for ; Mon, 16 Dec 2024 15:42:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E78967F9C; Mon, 16 Dec 2024 16:42:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ag2O+Voo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3BB3167F72 for ; Mon, 16 Dec 2024 16:42:33 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 96362A30; Mon, 16 Dec 2024 16:41:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363716; bh=L81KN7R/BOj6ABWZJ0ZS63M/EB8ev+WtAxJ7qGap3rU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ag2O+Vooi5599fT6P32k00vFM0ARMpijRElYPq+qoq0ojwMDod5ZhPsa3frS8YC2q rcUAQWQ7hEtGD+GnmbkEGLyJWO1CH3uoWqO/DkXTQECh7z94AP6UtvamcvvNaYJ09f xhSIAmWvM5Ohm4335L6arh+A32x0lDrOcZ6LqB7k= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 17/20] pipeline: rkisp1: make RkISP1CameraConfiguration a friend of the pipeline handler Date: Mon, 16 Dec 2024 16:40:57 +0100 Message-ID: <20241216154124.203650-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the validate() implementation, the RkISP1CameraConfiguration needs access to dewarper related members of the PipelineHandlerRkISP1 object. Allow this access by adding const versions of the pipe accessors and making RkISP1CameraConfiguration a friend of PipelineHandlerRkISP1. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v4: - Changed commit title prefix to pipeline: - Collected tags Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 14d4bb9a929b..55b839e76d06 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -98,6 +98,7 @@ public: } PipelineHandlerRkISP1 *pipe(); + const PipelineHandlerRkISP1 *pipe() const; int loadIPA(unsigned int hwRevision); Stream mainPathStream_; @@ -176,6 +177,7 @@ private: } friend RkISP1CameraData; + friend RkISP1CameraConfiguration; friend RkISP1Frames; int initLinks(Camera *camera, const CameraSensor *sensor, @@ -363,6 +365,11 @@ PipelineHandlerRkISP1 *RkISP1CameraData::pipe() return static_cast(Camera::Private::pipe()); } +const PipelineHandlerRkISP1 *RkISP1CameraData::pipe() const +{ + return static_cast(Camera::Private::pipe()); +} + int RkISP1CameraData::loadIPA(unsigned int hwRevision) { ipa_ = IPAManager::createIPA(pipe(), 1, 1); From patchwork Mon Dec 16 15:40:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22364 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D590FC32F6 for ; Mon, 16 Dec 2024 15:42:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7C47667FA2; Mon, 16 Dec 2024 16:42:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dGB6Z1mW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2653767F96 for ; Mon, 16 Dec 2024 16:42:36 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 85B95675; Mon, 16 Dec 2024 16:41:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363719; bh=tkmoaOSYrjXM44kUH5XIBTLAECUP8cyQy5EmmE3ToVA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dGB6Z1mWET1Yoaxuph1RltN/nN4EKD5VPenNi3DZODaAOUEYAz/yjLaJ9dc/GO5xS 9CkV81qeJDoPM9k/vGLuTffJy5uAPGeUnJTDY7VRBa8dBtOjVL+AqKcqU0Fuo9ZyM0 TrahKADXru4yOo9DvXyiZwhY3NS0sSc6cfY+fVYY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 18/20] pipeline: rkisp1: Fix config validation when dewarper is used Date: Mon, 16 Dec 2024 16:40:58 +0100 Message-ID: <20241216154124.203650-19-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is used, config->validate() needs to take the restrictions of the dewarper into account. Add the corresponding checks. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Small fixes in comments - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 46 +++++++++++++++++++++--- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 55b839e76d06..432a01694913 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -497,6 +497,7 @@ bool RkISP1CameraConfiguration::fitsAllPaths(const StreamConfiguration &cfg) CameraConfiguration::Status RkISP1CameraConfiguration::validate() { + const PipelineHandlerRkISP1 *pipe = data_->pipe(); const CameraSensor *sensor = data_->sensor_.get(); unsigned int pathCount = data_->selfPath_ ? 2 : 1; Status status; @@ -553,6 +554,18 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } + bool useDewarper = false; + if (pipe->dewarper_) { + /* + * Platforms with dewarper support, such as i.MX8MP, support + * only a single stream. We can inspect config_[0] only here. + */ + bool isRaw = PixelFormatInfo::info(config_[0].pixelFormat).colourEncoding == + PixelFormatInfo::ColourEncodingRAW; + if (!isRaw) + useDewarper = true; + } + /* * If there are more than one stream in the configuration figure out the * order to evaluate the streams. The first stream has the highest @@ -565,12 +578,31 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (config_.size() == 2 && fitsAllPaths(config_[0])) std::reverse(order.begin(), order.end()); + /* + * Validate the configuration against the desired path and, if the + * platform supports it, the dewarper. + */ auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, Stream *stream, Status expectedStatus) { StreamConfiguration tryCfg = cfg; - if (path->validate(sensor, sensorConfig, &tryCfg) != expectedStatus) + + Status ret = path->validate(sensor, sensorConfig, &tryCfg); + if (ret == Invalid) + return false; + + if (!useDewarper && + (expectedStatus == Valid && ret == Adjusted)) return false; + if (useDewarper) { + bool adjusted; + + pipe->dewarper_->validateOutput(&tryCfg, &adjusted, + Converter::Alignment::Down); + if (expectedStatus == Valid && adjusted) + return false; + } + cfg = tryCfg; cfg.setStream(stream); return true; @@ -820,6 +852,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + useDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -832,8 +865,13 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (media_->hwRevision() == RKISP1_V_IMX8MP) { /* imx8mp has only a single path. */ const auto &cfg = config->at(0); - Size ispCrop = format.size.boundedToAspectRatio(cfg.size) - .alignedUpTo(2, 2); + Size ispCrop = format.size.boundedToAspectRatio(cfg.size); + if (useDewarper_) + ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, + ispCrop); + else + ispCrop.alignUpTo(2, 2); + outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; } @@ -875,8 +913,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; - useDewarper_ = true; - /* * Calculate the crop rectangle of the data * flowing into the dewarper in sensor From patchwork Mon Dec 16 15:40:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22365 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED60CC32F6 for ; Mon, 16 Dec 2024 15:42:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 65ED267FA1; Mon, 16 Dec 2024 16:42:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RVbnb6PC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9BA567F9A for ; Mon, 16 Dec 2024 16:42:38 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 33DF182A; Mon, 16 Dec 2024 16:42:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363722; bh=4D69et1CbHfB2HVgM+731TlbSbrVJ4XG7A6AK7c5V+4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RVbnb6PCwj0Ne9zJCeOHFjkHFAvJTZ9KEhoAxir81MpMbKzEqmOKIJ7e+qcBwCZdj zfBns7VRaz0Ue8/Lb1hlLfgrJQifMj1L6eD7L/rbdMIaAS3IUzXDOhCeICOjMcMA9n al0PTfVil+xsGBTYKEeL86MnhhkP2qWuajY0Ba1E= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 19/20] libcamera: camera_sensor: Add parameter to limit returned sensor size Date: Mon, 16 Dec 2024 16:40:59 +0100 Message-ID: <20241216154124.203650-20-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The getFormat function takes the aspect ratio and the area of the requested size into account when choosing the best sensor size. In case the sensor is connected to an rkisp1 the maximum supported frame size of the ISP is another constraining factor for the selection of the best format. Add a maxSize parameter to support such a constraint. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v4: - Collected tags Changes in v3: - Added this patch --- include/libcamera/internal/camera_sensor.h | 2 +- src/libcamera/sensor/camera_sensor.cpp | 3 +++ src/libcamera/sensor/camera_sensor_legacy.cpp | 9 +++++++-- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index d030e254a552..605ea8136900 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -53,7 +53,7 @@ public: virtual V4L2SubdeviceFormat getFormat(const std::vector &mbusCodes, - const Size &size) const = 0; + const Size &size, const Size maxSize = Size()) const = 0; virtual int setFormat(V4L2SubdeviceFormat *format, Transform transform = Transform::Identity) = 0; virtual int tryFormat(V4L2SubdeviceFormat *format) const = 0; diff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp index 208a1603cb32..a131ac224ec0 100644 --- a/src/libcamera/sensor/camera_sensor.cpp +++ b/src/libcamera/sensor/camera_sensor.cpp @@ -116,6 +116,7 @@ CameraSensor::~CameraSensor() = default; * \brief Retrieve the best sensor format for a desired output * \param[in] mbusCodes The list of acceptable media bus codes * \param[in] size The desired size + * \param[in] maxSize The maximum size * * Media bus codes are selected from \a mbusCodes, which lists all acceptable * codes in decreasing order of preference. Media bus codes supported by the @@ -134,6 +135,8 @@ CameraSensor::~CameraSensor() = default; * bandwidth. * - The desired \a size shall be supported by one of the media bus code listed * in \a mbusCodes. + * - The desired \a size shall fit into the maximum size \a maxSize if it is not + * null. * * When multiple media bus codes can produce the same size, the code at the * lowest position in \a mbusCodes is selected. diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index 17d6fa680e39..32989c19c019 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -74,7 +74,8 @@ public: Size resolution() const override; V4L2SubdeviceFormat getFormat(const std::vector &mbusCodes, - const Size &size) const override; + const Size &size, + const Size maxSize) const override; int setFormat(V4L2SubdeviceFormat *format, Transform transform = Transform::Identity) override; int tryFormat(V4L2SubdeviceFormat *format) const override; @@ -699,7 +700,7 @@ Size CameraSensorLegacy::resolution() const V4L2SubdeviceFormat CameraSensorLegacy::getFormat(const std::vector &mbusCodes, - const Size &size) const + const Size &size, Size maxSize) const { unsigned int desiredArea = size.width * size.height; unsigned int bestArea = UINT_MAX; @@ -716,6 +717,10 @@ CameraSensorLegacy::getFormat(const std::vector &mbusCodes, for (const SizeRange &range : formats->second) { const Size &sz = range.max; + if (!maxSize.isNull() && + (sz.width > maxSize.width || sz.height > maxSize.height)) + continue; + if (sz.width < size.width || sz.height < size.height) continue; From patchwork Mon Dec 16 15:41:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22366 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0EE9EC32F6 for ; Mon, 16 Dec 2024 15:42:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8D11167FA7; Mon, 16 Dec 2024 16:42:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uOX+Dgat"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BBE4867FA4 for ; Mon, 16 Dec 2024 16:42:41 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bfdf:3a3c:e45:66e3]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2F0EF82A; Mon, 16 Dec 2024 16:42:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734363725; bh=jeL0tv/MP6uIXkrmDqdzN8kqG5EECeHMHez1p3CUyLA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uOX+DgattsfmvBeTQVm6JjSUsi8G4iHkYDi4+PXq7SLN8MfhNqGA0M1sxKv2t2+ig eSaSv5ywnwOr1+rG8wq6vIDZ/VWL8ybml/SrIt8NijibC9Lm2f4HHSBKe13qQLR0GH ijV2YRWEC/vQCLEHmDkDnSI0P+6Mr+Zm9qOiHo/U= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi , Paul Elder Subject: [PATCH v4 20/20] pipeline: rkisp1: Limit sensor size to max resolution Date: Mon, 16 Dec 2024 16:41:00 +0100 Message-ID: <20241216154124.203650-21-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241216154124.203650-1-stefan.klug@ideasonboard.com> References: <20241216154124.203650-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In RkISPPath::validate() the sensor sizes are correctly filtered to the ones supported by the ISP. But later in RkISPPipeline::configure() the configured stream size is passed to sensor->getFormat() and the CameraSensor class chooses the best sensor format for the requested stream size. This can result in a sensor format that is too big for the ISP. Fix that by supplying the maximum resolution supported by the ISP to setFormat(). Fixes: 761545407c76 ("pipeline: rkisp1: Filter out sensor sizes not supported by the pipeline") Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v4: - Changed commit tite prefix to pipeline: Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 432a01694913..35c793da9bba 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -677,7 +677,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() [](const auto &value) { return value.second; }); } - sensorFormat_ = sensor->getFormat(mbusCodes, maxSize); + sensorFormat_ = sensor->getFormat(mbusCodes, maxSize, + mainPath->maxResolution()); if (sensorFormat_.size.isNull()) sensorFormat_.size = sensor->resolution(); diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index 45be8476616c..2a1ef0abe6d1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -62,6 +62,7 @@ public: int queueBuffer(FrameBuffer *buffer) { return video_->queueBuffer(buffer); } Signal &bufferReady() { return video_->bufferReady; } + const Size &maxResolution() const { return maxResolution_; } private: void populateFormats();