From patchwork Fri Dec 6 10:13:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22193 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 71E7BBF415 for ; Fri, 6 Dec 2024 10:13:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 215BC66121; Fri, 6 Dec 2024 11:13:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Jv44pKlZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B9BF966121 for ; Fri, 6 Dec 2024 11:13:51 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 28A3374C; Fri, 6 Dec 2024 11:13:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480002; bh=1aGKYMCKt5K+IXdrsLHVElOYWh2VWsVZUQHcG7tgHOc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Jv44pKlZeViSx3euygHTcKGLkc7mcaAroE95XelK/r6rChxcwIJpnV1vPCYwePxXw TR6c280XKaeGmkubpEyOw5dR1vlFIoziZyDc2ea3fgh9lyp5ItVG2FyGsRSeVRqH/C fKU/EsnHjPeW3HUAyR4x4BCjpYerDgYF8iCdVyIs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v3 01/17] libcamera: rkisp1: Fix scope of dewarper stop() exit action Date: Fri, 6 Dec 2024 11:13:23 +0100 Message-ID: <20241206101344.767170-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move the definition of the dewarper stop() action into the scope were the corresponding start() happens. Fixes: 12b553d691d4 ("libcamera: rkisp1: Plumb the dw100 dewarper as V4L2M2M converter") Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v3: - Removed blank line from commit message --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 908724e20975..8b92fdebdd63 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1044,8 +1044,8 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL LOG(RkISP1, Error) << "Failed to start dewarper"; return ret; } + actions += [&]() { dewarper_->stop(); }; } - actions += [&]() { dewarper_->stop(); }; } if (data->mainPath_->isEnabled()) { From patchwork Fri Dec 6 10:13:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22194 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 97F4FBF415 for ; Fri, 6 Dec 2024 10:13:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1CDA166136; Fri, 6 Dec 2024 11:13:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="L2fBPcp7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 89BE666132 for ; Fri, 6 Dec 2024 11:13:54 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 129919FC; Fri, 6 Dec 2024 11:13:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480005; bh=LMV129RDKFT4QPNf1sligPF7WaN2hmiQ+v+hxw9Q9vk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L2fBPcp7BP9qf3ldpQxnmrBqGFaM41AHneFeNhlfCM18rG+7beKQ1kYZnV2mmNzG8 G3+H+/JeBJaeVQ2pTd2lhVHW+iQMFok+7xO/baAXYCI1w6bEejK/1W3olt6HvxTund iNg47ECztfuF/TlA5aY+HrdnvtG0r85440LH/6U8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v3 02/17] libcamera: rkisp1: Keep aspect ratio on imx8mp Date: Fri, 6 Dec 2024 11:13:24 +0100 Message-ID: <20241206101344.767170-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the current code, the input stage of the image resizer is used to apply a crop to keep the aspect ratio in cases where the requested output aspect ratio differs from the one of the selected sensor mode. On the imx8mp the resizer hardware is not capable of cropping (for reference see also rkisp1-resizer.c:rkisp1_rsz_set_sink_crop() in the linux kernel v6.10). Therefore apply the necessary cropping on the output of the ISP (on the image stabilization block). The cropping code on the image resizer doesn't need modifications as the requested crop gets ignored by the kernel. While at it, remove a todo comment that is no longer needed. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v3: - Dropped info message about the crop Changes in v2: - Fixed alignment - Improved comments - Removed todo comment - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 13 +++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 13 ++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 8b92fdebdd63..698fe9ff57a4 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -812,6 +812,19 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (!isRaw_) format.code = MEDIA_BUS_FMT_YUYV8_2X8; + /* + * On devices without DUAL_CROP (like the imx8mp) cropping needs to be + * done on the ISP/IS output. + */ + if (media_->hwRevision() == RKISP1_V_IMX8MP) { + /* imx8mp has only a single path. */ + const auto &cfg = config->at(0); + Size ispCrop = format.size.boundedToAspectRatio(cfg.size) + .alignedUpTo(2, 2); + rect = ispCrop.centeredTo(Rectangle(format.size).center()); + format.size = ispCrop; + } + LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format << " crop " << rect; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index 236d05af7a2f..eee5b09e2ff0 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -417,11 +417,14 @@ int RkISP1Path::configure(const StreamConfiguration &config, /* * Crop on the resizer input to maintain FOV before downscaling. * - * \todo The alignment to a multiple of 2 pixels is required but may - * change the aspect ratio very slightly. A more advanced algorithm to - * compute the resizer input crop rectangle is needed, and it should - * also take into account the need to crop away the edge pixels affected - * by the ISP processing blocks. + * Note that this does not apply to devices without DUAL_CROP support + * (like imx8mp) , where the cropping needs to be done on the + * ImageStabilizer block on the ISP source pad and therefore is + * configured before this stage. For simplicity we still set the crop. + * This gets ignored by the kernel driver because the hardware is + * missing the capability. + * + * Alignment to a multiple of 2 pixels is required by the resizer. */ Size ispCrop = inputFormat.size.boundedToAspectRatio(config.size) .alignedUpTo(2, 2); From patchwork Fri Dec 6 10:13:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22195 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7CE02BF415 for ; Fri, 6 Dec 2024 10:14:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 291DB66139; Fri, 6 Dec 2024 11:14:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YMWTmiWA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B8C166135 for ; Fri, 6 Dec 2024 11:13:57 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AB52174C; Fri, 6 Dec 2024 11:13:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480007; bh=LnhbRJbhNTkk7xb9hRaSCNoSo/GtePhhHMNJe9ULUfM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YMWTmiWApO5HxpzofpDKynWGDCCd9yA2/Xg5x8jVQQeWOrlB49aZugbvqgURpXVaZ vvmN+hBl48bL0k4wE9a7zHwec6zdPv2IV5nh+mBw4037TizHEQCqinaJClV4WQ9e1O gTtjoUTZ2DPlpky7XdRVjjuOTV9AW09Cyqsojml0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v3 03/17] libcamera: geometry: Add Rectangle::transformedBetween() Date: Fri, 6 Dec 2024 11:13:25 +0100 Message-ID: <20241206101344.767170-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handling cropping and scaling within a complicated pipeline involves transformations of rectangles between different coordinate systems. For example the full input of the dewarper (0,0)/1920x1080 might correspond to the rectangle (0, 243)/2592x1458 in sensor coordinates (of a 2592x1944 sensor). Add a function that allows the transformation of a rectangle defined in one reference frame (dewarper) into the coordinates of a second reference frame (sensor). Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v3: - Improved documentation - Added testcase Changes in v2: - Renamed mappedBetween() to transformedBetween() - Improved documentation - Collected tags --- include/libcamera/geometry.h | 3 +++ src/libcamera/geometry.cpp | 49 ++++++++++++++++++++++++++++++++++++ test/geometry.cpp | 11 ++++++++ 3 files changed, 63 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index 9ca5865a3d0d..e5f0a843d314 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -299,6 +299,9 @@ public: __nodiscard Rectangle scaledBy(const Size &numerator, const Size &denominator) const; __nodiscard Rectangle translatedBy(const Point &point) const; + + Rectangle transformedBetween(const Rectangle &source, + const Rectangle &target) const; }; bool operator==(const Rectangle &lhs, const Rectangle &rhs); diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 90ccf8c19f97..81cc8cd538a0 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -837,6 +837,55 @@ Rectangle Rectangle::translatedBy(const Point &point) const return { x + point.x, y + point.y, width, height }; } +/** + * \brief Transform a Rectangle from one reference rectangle to another + * \param[in] source The \a source reference rectangle + * \param[in] destination The \a destination reference rectangle + * + * The \a source and \a destination parameters describe two rectangles defined + * in different reference systems. The Rectangle is translated from the source + * reference system into the destination reference system. + * + * The typical use case for this function is to translate a selection rectangle + * specified in a reference system, in example the sensor's pixel array, into + * the same rectangle re-scaled and translated into a different reference + * system, in example the output frame on which the selection rectangle is + * applied to. + * + * For example, consider a sensor with a resolution of 4040x2360 pixels and a + * assume a rectangle of (100, 100)/3840x2160 (sensorFrame) in sensor + * coordinates is mapped to a rectangle (0,0)/(1920,1080) (displayFrame) in + * display coordinates. This function can be used to transform an arbitrary + * rectangle from display coordinates to sensor coordinates or vice versa: + * + * \code{.cpp} + * Rectangle sensorReference(100, 100, 3840, 2160); + * Rectangle displayReference(0, 0, 1920, 1080); + * + * // Bottom right quarter in sensor coordinates + * Rectangle sensorRect(2020, 100, 1920, 1080); + * displayRect = sensorRect.transformedBetween(sensorReference, displayReference); + * // displayRect is now (960, 540)/960x540 + * + * // Transformation back to sensor coordinates + * sensorRect = displayRect.transformedBetween(displayReference, sensorReference); + * \endcode + */ +Rectangle Rectangle::transformedBetween(const Rectangle &source, + const Rectangle &destination) const +{ + Rectangle r; + double sx = static_cast(destination.width) / source.width; + double sy = static_cast(destination.height) / source.height; + + r.x = static_cast((x - source.x) * sx) + destination.x; + r.y = static_cast((y - source.y) * sy) + destination.y; + r.width = static_cast(width * sx); + r.height = static_cast(height * sy); + + return r; +} + /** * \brief Compare rectangles for equality * \return True if the two rectangles are equal, false otherwise diff --git a/test/geometry.cpp b/test/geometry.cpp index 5760fa3c885a..11df043b733b 100644 --- a/test/geometry.cpp +++ b/test/geometry.cpp @@ -495,6 +495,17 @@ protected: return TestFail; } + Rectangle f1 = Rectangle(100, 200, 3000, 2000); + Rectangle f2 = Rectangle(200, 300, 1500, 1000); + /* Bottom right quarter of the corresponding frames. */ + Rectangle r1 = Rectangle(100 + 1500, 200 + 1000, 1500, 1000); + Rectangle r2 = Rectangle(200 + 750, 300 + 500, 750, 500); + if (r1.transformedBetween(f1, f2) != r2 || + r2.transformedBetween(f2, f1) != r1) { + cout << "Rectangle::transformedBetween() test failed" << endl; + return TestFail; + } + return TestPass; } }; From patchwork Fri Dec 6 10:13:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22196 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C663BBF415 for ; Fri, 6 Dec 2024 10:14:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 554226613D; Fri, 6 Dec 2024 11:14:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mDPpJQIG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 113C966138 for ; Fri, 6 Dec 2024 11:14:00 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B216474C; Fri, 6 Dec 2024 11:13:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480010; bh=xnJdSZqd3w9Pk4CsD0ehBtbeHtKdBC2kmTVsoAB1JF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mDPpJQIGzWaBeMNeNWUjAF6ualcPusrfADLgRUGuFnPf+zi6KmrV5ZTvSgN+9MPKo xtNa4/TL1DNoO3XLGcd8TsZHlzYKs2MYCLenQS2upuK7Zk1Z8SqH0lEgQ/q+tzJ+QM /Up27C+/FhhRCpk/Lau7rAlnQwsuJqUrhMTIH3C0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v3 04/17] pipeline: rkisp1: Split inputCrop and outputCrop Date: Fri, 6 Dec 2024 11:13:26 +0100 Message-ID: <20241206101344.767170-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" One Rectangle instance is used to calculate the inputCrop and the outputCrop of the ISP in the rkisp1 pipeline. Split that into two distinct variables, because both values will be needed in the upcoming patches. This patch does not contain any functional changes. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v2: - Moved declaration of outputCrop after setSelection(inputCrop) to get the potentially updated inputCrop - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 698fe9ff57a4..6ac14cd15c8f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -795,15 +795,16 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret < 0) return ret; - Rectangle rect(0, 0, format.size); - ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &rect); + Rectangle inputCrop(0, 0, format.size); + ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &inputCrop); if (ret < 0) return ret; LOG(RkISP1, Debug) << "ISP input pad configured with " << format - << " crop " << rect; + << " crop " << inputCrop; + Rectangle outputCrop = inputCrop; const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; @@ -821,15 +822,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const auto &cfg = config->at(0); Size ispCrop = format.size.boundedToAspectRatio(cfg.size) .alignedUpTo(2, 2); - rect = ispCrop.centeredTo(Rectangle(format.size).center()); + outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; } LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format - << " crop " << rect; + << " crop " << outputCrop; - ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &rect); + ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &outputCrop); if (ret < 0) return ret; @@ -840,7 +841,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "ISP output pad configured with " << format - << " crop " << rect; + << " crop " << outputCrop; std::map streamConfig; std::vector> outputCfgs; From patchwork Fri Dec 6 10:13:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22197 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1CB1CBF415 for ; Fri, 6 Dec 2024 10:14:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9D48E66144; Fri, 6 Dec 2024 11:14:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eRdIRhu3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B61866136 for ; Fri, 6 Dec 2024 11:14:02 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2C2C074C; Fri, 6 Dec 2024 11:13:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480013; bh=jIPjtC4cpuUaIKD7hKyB2rwwweHCDzqAtCd7Oa7ZKQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eRdIRhu3E3tx0XZw00IhKkppvMTazu1iVZDUV5ifeSIGjcCZppDeLEH8i0iiU9kJP IJEmeZH9vs+5zj+rBdbTMZU5FPhUafk1eWl5/f9RZ6/tksrWUQiLx6tTrkT7YMoEaz O62+A9lU2OLHCxRB/BUsgy2PyZyo3l6vknTKoGmA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Jacopo Mondi Subject: [PATCH v3 05/17] pipeline: rkisp1: Reorder sensorInfo collection code Date: Fri, 6 Dec 2024 11:13:27 +0100 Message-ID: <20241206101344.767170-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The sensorInfo (specifically the crop rectangle of the selected sensor mode) is collected to be passed to the IPA later. In an upcoming patch that data will also be needed for correct ScalerCrop handling. Move the collection of the sensorInfo before the dewarper configuration step and refactor the code a bit. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v2: - Collected tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6ac14cd15c8f..89946b782854 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -843,6 +843,11 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << "ISP output pad configured with " << format << " crop " << outputCrop; + IPACameraSensorInfo sensorInfo; + ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) + return ret; + std::map streamConfig; std::vector> outputCfgs; @@ -882,14 +887,9 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* Inform IPA of stream configuration and sensor controls. */ - ipa::rkisp1::IPAConfigInfo ipaConfig{}; - - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); - if (ret) - return ret; - - ipaConfig.sensorControls = data->sensor_->controls(); - ipaConfig.paramFormat = paramFormat.fourcc; + ipa::rkisp1::IPAConfigInfo ipaConfig{ sensorInfo, + data->sensor_->controls(), + paramFormat.fourcc }; ret = data->ipa_->configure(ipaConfig, streamConfig, &data->ipaControls_); if (ret) { From patchwork Fri Dec 6 10:13:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22198 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 275F3BF415 for ; Fri, 6 Dec 2024 10:14:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C370566148; Fri, 6 Dec 2024 11:14:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qlo48CKQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 944126613D for ; Fri, 6 Dec 2024 11:14:05 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3BD469FC; Fri, 6 Dec 2024 11:13:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480016; bh=V/J4wJ4pkpHSj6jhg+sH3h5VyeMjLU9+u+us9Tt+Ack=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qlo48CKQl1GbIr10HQs27hU6b64vP7/8VGrKltwi7tSrjlJzJEHA+cPmycUMr0Swj rx1Q4LvXQimsnNo/wx/emYlJRHJhDLAdg8jSBPaKqzvONOslRn0UiKDPDlGaQhvQCA +aiF4DOjkat5TuiQO/hA0U3gIF2Gzh9yvgGZrrVM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 06/17] libcamera: converter_v4l2_m2m: Improve crop bounds support Date: Fri, 6 Dec 2024 11:13:28 +0100 Message-ID: <20241206101344.767170-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a converter (dw100 on imx8mp in this case) is used in a pipeline, the ScalerCrop control get's created at createCamera() time. As this is before configure(), no streams were configured on the converter and the stream specific crop bounds are not known. Extend the converter class to report the default crop bounds in case the provided stream is not found. Signed-off-by: Stefan Klug --- .../internal/converter/converter_v4l2_m2m.h | 1 + src/libcamera/converter.cpp | 3 + .../converter/converter_v4l2_m2m.cpp | 113 +++++++++--------- 3 files changed, 59 insertions(+), 58 deletions(-) diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index 0bc0d053e2c4..a5286166f574 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -105,6 +105,7 @@ private: std::map> streams_; std::map queue_; + std::pair inputCropBounds_; }; } /* namespace libcamera */ diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index 3a3f84344c5e..aa16970cd446 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -195,6 +195,9 @@ Converter::~Converter() * this function should be called after the \a stream has been configured using * configure(). * + * When called with an invalid \a stream, the function returns the default crop + * bounds of the converter. + * * \return A pair containing the minimum and maximum crop bound in that order */ diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index d63ef2f8028f..bd7e5cce600d 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -30,6 +30,52 @@ namespace libcamera { LOG_DECLARE_CATEGORY(Converter) +namespace { + +int getCropBounds(V4L2VideoDevice *device, Rectangle &minCrop, + Rectangle &maxCrop) +{ + Rectangle minC; + Rectangle maxC; + + /* Find crop bounds */ + minC.width = 1; + minC.height = 1; + maxC.width = UINT_MAX; + maxC.height = UINT_MAX; + + int ret = device->setSelection(V4L2_SEL_TGT_CROP, &minC); + if (ret) { + LOG(Converter, Error) + << "Could not query minimum selection crop: " + << strerror(-ret); + return ret; + } + + ret = device->getSelection(V4L2_SEL_TGT_CROP_BOUNDS, &maxC); + if (ret) { + LOG(Converter, Error) + << "Could not query maximum selection crop: " + << strerror(-ret); + return ret; + } + + /* Reset the crop to its maximum */ + ret = device->setSelection(V4L2_SEL_TGT_CROP, &maxC); + if (ret) { + LOG(Converter, Error) + << "Could not reset selection crop: " + << strerror(-ret); + return ret; + } + + minCrop = minC; + maxCrop = maxC; + return 0; +} + +} /* namespace */ + /* ----------------------------------------------------------------------------- * V4L2M2MConverter::V4L2M2MStream */ @@ -98,41 +144,10 @@ int V4L2M2MConverter::V4L2M2MStream::configure(const StreamConfiguration &inputC outputBufferCount_ = outputCfg.bufferCount; if (converter_->features() & Feature::InputCrop) { - Rectangle minCrop; - Rectangle maxCrop; - - /* Find crop bounds */ - minCrop.width = 1; - minCrop.height = 1; - maxCrop.width = UINT_MAX; - maxCrop.height = UINT_MAX; - - ret = setInputSelection(V4L2_SEL_TGT_CROP, &minCrop); - if (ret) { - LOG(Converter, Error) - << "Could not query minimum selection crop: " - << strerror(-ret); - return ret; - } - - ret = getInputSelection(V4L2_SEL_TGT_CROP_BOUNDS, &maxCrop); - if (ret) { - LOG(Converter, Error) - << "Could not query maximum selection crop: " - << strerror(-ret); - return ret; - } - - /* Reset the crop to its maximum */ - ret = setInputSelection(V4L2_SEL_TGT_CROP, &maxCrop); - if (ret) { - LOG(Converter, Error) - << "Could not reset selection crop: " - << strerror(-ret); + ret = getCropBounds(m2m_->output(), inputCropBounds_.first, + inputCropBounds_.second); + if (ret) return ret; - } - - inputCropBounds_ = { minCrop, maxCrop }; } return 0; @@ -258,27 +273,9 @@ V4L2M2MConverter::V4L2M2MConverter(MediaDevice *media) return; } - /* Discover Feature::InputCrop */ - Rectangle maxCrop; - maxCrop.width = UINT_MAX; - maxCrop.height = UINT_MAX; - - ret = m2m_->output()->setSelection(V4L2_SEL_TGT_CROP, &maxCrop); - if (ret) - return; - - /* - * Rectangles for cropping targets are defined even if the device - * does not support cropping. Their sizes and positions will be - * fixed in such cases. - * - * Set and inspect a crop equivalent to half of the maximum crop - * returned earlier. Use this to determine whether the crop on - * input is really supported. - */ - Rectangle halfCrop(maxCrop.size() / 2); - ret = m2m_->output()->setSelection(V4L2_SEL_TGT_CROP, &halfCrop); - if (!ret && halfCrop != maxCrop) { + ret = getCropBounds(m2m_->output(), inputCropBounds_.first, + inputCropBounds_.second); + if (!ret && inputCropBounds_.first != inputCropBounds_.second) { features_ |= Feature::InputCrop; LOG(Converter, Info) @@ -477,10 +474,10 @@ std::pair V4L2M2MConverter::inputCropBounds(const Stream *stream) { auto iter = streams_.find(stream); - if (iter == streams_.end()) - return {}; + if (iter != streams_.end()) + return iter->second->inputCropBounds(); - return iter->second->inputCropBounds(); + return inputCropBounds_; } /** From patchwork Fri Dec 6 10:13:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22199 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7E5E3BF415 for ; Fri, 6 Dec 2024 10:14:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E15566148; Fri, 6 Dec 2024 11:14:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qNNW5X0R"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E11966147 for ; Fri, 6 Dec 2024 11:14:08 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0A9EF9FC; Fri, 6 Dec 2024 11:13:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480019; bh=lkLu06CK3ndS/9wQqnPGGRn5ROVBHx/zFZHyO4c7MEI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qNNW5X0Rosj6hjz2b6nTYLzIUKdIx4lhA7QHT6zw9bdUrkA+d0pnWwRo+jzw9dVQB V0toFXuVZjGP/XL+6ItecO7f1CcuOb5othYJAvbsoQvrPw9fdjj2uSMF4pat+45T4M nHP6DQFadc2PhQLAWfpS1P2ysEp7OQTNtUYBwCuM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 07/17] pipeline: rkisp1: Fix ScalerCrop to be in sensor coordinates Date: Fri, 6 Dec 2024 11:13:29 +0100 Message-ID: <20241206101344.767170-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" ScalerCrop is specified as being in sensor coordinates. The current dewarper implementation on the imx8mp handles ScalerCrop in dewarper coordinates. This leads to unexpected results and an unusable ScalerCrop control in camshark. Fix that by transforming back and forth between sensor coordinates and dewarper coordinates. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- Changes in v3: - Rename dewarpSensorCrop_ to scalerMaxCrop_ - Remove unnecessary ScalerCrop max calculation Changes in v2: - Initialize dewarperSensorCrop_ to sane defaults - Collect tags --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 50 +++++++++++++++++++----- 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 89946b782854..ad556ec85a2c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -205,6 +205,7 @@ private: RkISP1SelfPath selfPath_; std::unique_ptr dewarper_; + Rectangle scalerMaxCrop_; bool useDewarper_; std::optional activeCrop_; @@ -861,6 +862,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); useDewarper_ = ret ? false : true; + + /* + * Calculate the crop rectangle of the data + * flowing into the dewarper in sensor + * coordinates. + */ + scalerMaxCrop_ = + outputCrop.transformedBetween(inputCrop, + sensorInfo.analogCrop); } } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); @@ -1223,10 +1233,19 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) std::pair cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, - cropLimits.second, - cropLimits.second); - activeCrop_ = cropLimits.second; + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform the limits to sensor coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ + Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, + scalerMaxCrop_); + + controls[&controls::ScalerCrop] = ControlInfo(min, + scalerMaxCrop_, + scalerMaxCrop_); + activeCrop_ = scalerMaxCrop_; } /* Add the IPA registered controls to list of camera controls. */ @@ -1254,6 +1273,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); + scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); + const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1473,22 +1494,33 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { - Rectangle appliedRect = crop.value(); + Rectangle rect = crop.value(); + + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform it into dewarper coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ + std::pair cropLimits = + dewarper_->inputCropBounds(&data->mainPathStream_); + rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &appliedRect); - if (!ret && appliedRect != crop.value()) { + &rect); + rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); + if (!ret && rect != crop.value()) { /* * If the rectangle is changed by setInputCrop on the * dewarper, log a debug message and cache the actual * applied rectangle for metadata reporting. */ LOG(RkISP1, Debug) - << "Applied rectangle " << appliedRect.toString() + << "Applied rectangle " << rect.toString() << " differs from requested " << crop.value().toString(); } - activeCrop_ = appliedRect; + activeCrop_ = rect; } /* From patchwork Fri Dec 6 10:13:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22200 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3059EBF415 for ; Fri, 6 Dec 2024 10:14:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B7B666614D; Fri, 6 Dec 2024 11:14:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KF4pmQK1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F040466148 for ; Fri, 6 Dec 2024 11:14:10 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 962A49FC; Fri, 6 Dec 2024 11:13:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480021; bh=PeCrOych0B7QXQoGmkEvSCAKuUehc/eOSBzHYFIRX1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KF4pmQK1XhINueHRcUUkh1z9RGdNFpupzQizHOadxv1i8DSYG704LmHRi3SOknlWP caJ1+5P32LezoN3mwnj/HC7Jm107KTiOX+bh9ZO6prRnR0r9DPJX9oMm+KAyN3j6K3 P73FMlfkn9KingSOBwykl37o/8kgsrjdZiEp8f0w= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 08/17] pipeline: rkisp1: Add ScalerMaximumCrop property Date: Fri, 6 Dec 2024 11:13:30 +0100 Message-ID: <20241206101344.767170-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ScalerMaximumCrop property holds the biggest allowed ScalerCrop value. Add it to the rkisp1. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Jacopo Mondi --- Changes in v2: - Moved one hunk to the correct patch 6/7 --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index ad556ec85a2c..098c560ca5c8 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1245,6 +1246,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) controls[&controls::ScalerCrop] = ControlInfo(min, scalerMaxCrop_, scalerMaxCrop_); + data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); activeCrop_ = scalerMaxCrop_; } From patchwork Fri Dec 6 10:13:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22201 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 15114BF415 for ; Fri, 6 Dec 2024 10:14:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 84B3966149; Fri, 6 Dec 2024 11:14:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rolPBWeI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E56466147 for ; Fri, 6 Dec 2024 11:14:14 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DCEB874C; Fri, 6 Dec 2024 11:13:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480025; bh=OVUhbjooxzBOWajQ6a2W1ABbAAkYO8boaBBU0dPBAzk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rolPBWeIMZh+7iM5EjhVyuEHuH+dhcm03DZtZ7oKtvzSzV3jGtG859DqiAVKhgJu4 quPcTCna6kdltih5OY3+9ifIRwXO6Z9Upe/acwji4FPuK+BD+JhDQIph11JtEOtWb9 WYMSOZDDGXJJjEJoy1cF/gSmvjQQn6CCRxfkOOIk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v3 09/17] libcamera: converter: Add functions to adjust config Date: Fri, 6 Dec 2024 11:13:31 +0100 Message-ID: <20241206101344.767170-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add to the Converter interface two functions used by pipeline handlers to validate and adjust the converter input and output configurations by specifying the desired alignment for the adjustment. Add the adjustInputSize() and adjustOutputSize() functions that allows to adjust the converter input/output sizes with the desired alignment. Add a validateOutput() function meant to be used by the pipeline handler implementations of validate(). The function adjusts a StreamConfiguration to a valid configuration produced by the Converter. Signed-off-by: Jacopo Mondi Reviewed-by: Stefan Klug Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- include/libcamera/internal/converter.h | 17 ++ .../internal/converter/converter_v4l2_m2m.h | 11 ++ src/libcamera/converter.cpp | 43 +++++ .../converter/converter_v4l2_m2m.cpp | 169 ++++++++++++++++++ 4 files changed, 240 insertions(+) diff --git a/include/libcamera/internal/converter.h b/include/libcamera/internal/converter.h index ffbb6f345cd5..9213ae5b9c33 100644 --- a/include/libcamera/internal/converter.h +++ b/include/libcamera/internal/converter.h @@ -41,6 +41,13 @@ public: using Features = Flags; + enum class Alignment { + Down = 0, + Up = (1 << 0), + }; + + using Alignments = Flags; + Converter(MediaDevice *media, Features features = Feature::None); virtual ~Converter(); @@ -51,9 +58,19 @@ public: virtual std::vector formats(PixelFormat input) = 0; virtual SizeRange sizes(const Size &input) = 0; + virtual Size adjustInputSize(const PixelFormat &pixFmt, + const Size &size, + Alignments align = Alignment::Down) = 0; + virtual Size adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, + Alignments align = Alignment::Down) = 0; + virtual std::tuple strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size) = 0; + virtual int validateOutput(StreamConfiguration *cfg, bool *adjusted, + Alignments align = Alignment::Down) = 0; + virtual int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfgs) = 0; virtual int exportBuffers(const Stream *stream, unsigned int count, diff --git a/include/libcamera/internal/converter/converter_v4l2_m2m.h b/include/libcamera/internal/converter/converter_v4l2_m2m.h index a5286166f574..89bd2878b190 100644 --- a/include/libcamera/internal/converter/converter_v4l2_m2m.h +++ b/include/libcamera/internal/converter/converter_v4l2_m2m.h @@ -47,6 +47,11 @@ public: std::tuple strideAndFrameSize(const PixelFormat &pixelFormat, const Size &size); + Size adjustInputSize(const PixelFormat &pixFmt, + const Size &size, Alignments align = Alignment::Down) override; + Size adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, Alignments align = Alignment::Down) override; + int configure(const StreamConfiguration &inputCfg, const std::vector> &outputCfg); int exportBuffers(const Stream *stream, unsigned int count, @@ -55,6 +60,9 @@ public: int start(); void stop(); + int validateOutput(StreamConfiguration *cfg, bool *adjusted, + Alignments align = Alignment::Down) override; + int queueBuffers(FrameBuffer *input, const std::map &outputs); @@ -101,6 +109,9 @@ private: std::pair inputCropBounds_; }; + Size adjustSizes(const Size &size, const std::vector &ranges, + Alignments align); + std::unique_ptr m2m_; std::map> streams_; diff --git a/src/libcamera/converter.cpp b/src/libcamera/converter.cpp index aa16970cd446..c3da162b7de7 100644 --- a/src/libcamera/converter.cpp +++ b/src/libcamera/converter.cpp @@ -50,6 +50,21 @@ LOG_DEFINE_CATEGORY(Converter) * \brief A bitwise combination of features supported by the converter */ +/** + * \enum Converter::Alignment + * \brief The alignment mode specified when adjusting the converter input or + * output sizes + * \var Converter::Alignment::Down + * \brief Adjust the Converter sizes to a smaller valid size + * \var Converter::Alignment::Up + * \brief Adjust the Converter sizes to a larger valid size + */ + +/** + * \typedef Converter::Alignments + * \brief A bitwise combination of alignments supported by the converter + */ + /** * \brief Construct a Converter instance * \param[in] media The media device implementing the converter @@ -110,6 +125,24 @@ Converter::~Converter() * \return A range of output image sizes */ +/** + * \fn Converter::adjustInputSize() + * \brief Adjust the converter input \a size to a valid value + * \param[in] pixFmt The pixel format of the converter input stream + * \param[in] size The converter input size to adjust to a valid value + * \param[in] align The desired alignment + * \return The adjusted converter input size + */ + +/** + * \fn Converter::adjustOutputSize() + * \brief Adjust the converter output \a size to a valid value + * \param[in] pixFmt The pixel format of the converter output stream + * \param[in] size The converter output size to adjust to a valid value + * \param[in] align The desired alignment + * \return The adjusted converter output size + */ + /** * \fn Converter::strideAndFrameSize() * \brief Retrieve the output stride and frame size for an input configutation @@ -118,6 +151,16 @@ Converter::~Converter() * \return A tuple indicating the stride and frame size or an empty tuple on error */ +/** + * \fn Converter::validateOutput() + * \brief Validate and possibily adjust \a cfg to a valid converter output + * \param[inout] cfg The StreamConfiguration to validate and adjust + * \param[out] adjusted Set to true if \a cfg has been adjusted + * \param[in] align The desired alignment + * \return 0 if \a cfg is valid or has been adjusted, a negative error code + * otherwise if \a cfg cannot be adjusted + */ + /** * \fn Converter::configure() * \brief Configure a set of output stream conversion from an input stream diff --git a/src/libcamera/converter/converter_v4l2_m2m.cpp b/src/libcamera/converter/converter_v4l2_m2m.cpp index bd7e5cce600d..6857472b29f2 100644 --- a/src/libcamera/converter/converter_v4l2_m2m.cpp +++ b/src/libcamera/converter/converter_v4l2_m2m.cpp @@ -8,6 +8,7 @@ #include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include #include #include @@ -400,6 +401,127 @@ V4L2M2MConverter::strideAndFrameSize(const PixelFormat &pixelFormat, return std::make_tuple(format.planes[0].bpl, format.planes[0].size); } +/** + * \copydoc libcamera::Converter::adjustInputSize + */ +Size V4L2M2MConverter::adjustInputSize(const PixelFormat &pixFmt, + const Size &size, Alignments align) +{ + auto formats = m2m_->output()->formats(); + V4L2PixelFormat v4l2PixFmt = m2m_->output()->toV4L2PixelFormat(pixFmt); + + auto it = formats.find(v4l2PixFmt); + if (it == formats.end()) { + LOG(Converter, Info) + << "Unsupported pixel format " << pixFmt; + return {}; + } + + return adjustSizes(size, it->second, align); +} + +/** + * \copydoc libcamera::Converter::adjustOutputSize + */ +Size V4L2M2MConverter::adjustOutputSize(const PixelFormat &pixFmt, + const Size &size, Alignments align) +{ + auto formats = m2m_->capture()->formats(); + V4L2PixelFormat v4l2PixFmt = m2m_->capture()->toV4L2PixelFormat(pixFmt); + + auto it = formats.find(v4l2PixFmt); + if (it == formats.end()) { + LOG(Converter, Info) + << "Unsupported pixel format " << pixFmt; + return {}; + } + + return adjustSizes(size, it->second, align); +} + +Size V4L2M2MConverter::adjustSizes(const Size &cfgSize, + const std::vector &ranges, + Alignments align) +{ + Size size = cfgSize; + + if (ranges.size() == 1) { + /* + * The device supports either V4L2_FRMSIZE_TYPE_CONTINUOUS or + * V4L2_FRMSIZE_TYPE_STEPWISE. + */ + const SizeRange &range = *ranges.begin(); + + size.width = std::clamp(size.width, range.min.width, + range.max.width); + size.height = std::clamp(size.height, range.min.height, + range.max.height); + + /* + * Check if any alignment is needed. If the sizes are already + * aligned, or the device supports V4L2_FRMSIZE_TYPE_CONTINUOUS + * with hStep and vStep equal to 1, we're done here. + */ + int widthR = size.width % range.hStep; + int heightR = size.height % range.vStep; + + /* Align up or down according to the caller request. */ + + if (widthR != 0) + size.width = size.width - widthR + + ((align == Alignment::Up) ? range.hStep : 0); + + if (heightR != 0) + size.height = size.height - heightR + + ((align == Alignment::Up) ? range.vStep : 0); + } else { + /* + * The device supports V4L2_FRMSIZE_TYPE_DISCRETE, find the + * size closer to the requested output configuration. + * + * The size ranges vector is not ordered, so we sort it first. + * If we align up, start from the larger element. + */ + std::vector sizes(ranges.size()); + std::transform(ranges.begin(), ranges.end(), std::back_inserter(sizes), + [](const SizeRange &range) { return range.max; }); + std::sort(sizes.begin(), sizes.end()); + + if (align == Alignment::Up) + std::reverse(sizes.begin(), sizes.end()); + + /* + * Return true if s2 is valid according to the desired + * alignment: smaller than s1 if we align down, larger than s1 + * if we align up. + */ + auto nextSizeValid = [](const Size &s1, const Size &s2, Alignments a) { + return a == Alignment::Down + ? (s1.width > s2.width && s1.height > s2.height) + : (s1.width < s2.width && s1.height < s2.height); + }; + + Size newSize; + for (const Size &sz : sizes) { + if (!nextSizeValid(size, sz, align)) + break; + + newSize = sz; + } + + if (newSize.isNull()) { + LOG(Converter, Error) + << "Cannot adjust " << cfgSize + << " to a supported converter size"; + return {}; + } + + size = newSize; + } + + return size; +} + /** * \copydoc libcamera::Converter::configure */ @@ -507,6 +629,53 @@ void V4L2M2MConverter::stop() iter.second->stop(); } +/** + * \copydoc libcamera::Converter::validateOutput + */ +int V4L2M2MConverter::validateOutput(StreamConfiguration *cfg, bool *adjusted, + Alignments align) +{ + V4L2VideoDevice *capture = m2m_->capture(); + V4L2VideoDevice::Formats fmts = capture->formats(); + + if (adjusted) + *adjusted = false; + + PixelFormat fmt = cfg->pixelFormat; + V4L2PixelFormat v4l2PixFmt = capture->toV4L2PixelFormat(fmt); + + auto it = fmts.find(v4l2PixFmt); + if (it == fmts.end()) { + it = fmts.begin(); + v4l2PixFmt = it->first; + cfg->pixelFormat = v4l2PixFmt.toPixelFormat(); + + if (adjusted) + *adjusted = true; + + LOG(Converter, Info) + << "Converter output pixel format adjusted to " + << cfg->pixelFormat; + } + + const Size cfgSize = cfg->size; + cfg->size = adjustSizes(cfgSize, it->second, align); + + if (cfg->size.isNull()) + return -EINVAL; + + if (cfg->size.width != cfgSize.width || + cfg->size.height != cfgSize.height) { + LOG(Converter, Info) + << "Converter size adjusted to " + << cfg->size; + if (adjusted) + *adjusted = true; + } + + return 0; +} + /** * \copydoc libcamera::Converter::queueBuffers */ From patchwork Fri Dec 6 10:13:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22202 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6A3F7BF415 for ; Fri, 6 Dec 2024 10:14:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9AE6366152; Fri, 6 Dec 2024 11:14:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lIuSWZqN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4512266150 for ; Fri, 6 Dec 2024 11:14:17 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E37BF74C; Fri, 6 Dec 2024 11:13:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480028; bh=beaQDYwQKgfEBs+qNtBHHojP0XeZnjdVWoLcfmAqrfU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lIuSWZqNLUgTl0vRdfy+yEtW+y9IcLn0f7D4SjQpwoPxuOA0nqoLV/0qfW4yoshW7 Zk2S4VEMKAOb8oMXZPubAdbszgjcCKp3Y7syGZHCDKNHyjzje0VeYtHlO78QckEAWp MzBt54kcX9/lTlrFwd0LiJsPF5PoU/LT7gQCjnQo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v3 10/17] libcamera: stream: Add operator<<(StreamConfiguration) Date: Fri, 6 Dec 2024 11:13:32 +0100 Message-ID: <20241206101344.767170-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The StreamConfiguration class only implmenets toString() but doesn't offer an overload of operator<<() which is convenient to use. Add an overload for operator<<(StreamConfiguration) which is based on the usage of StreamConfiguration::toString(). Signed-off-by: Jacopo Mondi --- include/libcamera/stream.h | 2 ++ src/libcamera/stream.cpp | 13 +++++++++++++ 2 files changed, 15 insertions(+) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index 071b71698acb..ea228aea7d56 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -61,6 +61,8 @@ private: StreamFormats formats_; }; +std::ostream &operator<<(std::ostream &out, StreamConfiguration cfg); + enum class StreamRole { Raw, StillCapture, diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index 1f75dbbc5b64..a093abf48c7c 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -395,6 +395,19 @@ std::string StreamConfiguration::toString() const return size.toString() + "-" + pixelFormat.toString(); } +/** + * \brief Insert a text representation of a StreamConfiguration into an output + * stream + * \param[in] out The output stream + * \param[in] cfg The StreamConfiguration + * \return The output stream \a out + */ +std::ostream &operator<<(std::ostream &out, StreamConfiguration cfg) +{ + out << cfg.toString(); + return out; +} + /** * \enum StreamRole * \brief Identify the role a stream is intended to play From patchwork Fri Dec 6 10:13:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22203 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9B43BF415 for ; Fri, 6 Dec 2024 10:14:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5CC4766154; Fri, 6 Dec 2024 11:14:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VnLWf9K9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 21DE866150 for ; Fri, 6 Dec 2024 11:14:20 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B41DF74C; Fri, 6 Dec 2024 11:13:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480030; bh=yHnzyZRM5IV6dwy8fEaHRE2myWj0YapRWHx+OZjs6Dg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VnLWf9K9nu+2HBaruLZAYzPTlnK5iTmxDE5ZYRXbAjxWc2VtLeY8GVLsn0gyMFWNo U2xyMqP/j34/KGYq7Ie0x7Ubh+C8DfCnYzANVLJZxDz5Jm6DxQX117Xd9bunHFfy5F k4wWTD2kOKF7vIBwvMZD/2G+ql+T6NrhYiPPC3Dw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi Subject: [PATCH v3 11/17] libcamera: rkisp1: Refactor path validation Date: Fri, 6 Dec 2024 11:13:33 +0100 Message-ID: <20241206101344.767170-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Refactor validation code to prepare for extensions in the upcoming patches. Code duplication is reduced by moving parts of the validation logic into a lambda function. This patch does not include any functional changes. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 35 +++++++++++++----------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 098c560ca5c8..7f41092ee2d5 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -558,50 +558,53 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (config_.size() == 2 && fitsAllPaths(config_[0])) std::reverse(order.begin(), order.end()); + auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, + Stream *stream, Status expectedStatus) { + StreamConfiguration tryCfg = cfg; + if (path->validate(sensor, sensorConfig, &tryCfg) != expectedStatus) + return false; + + cfg = tryCfg; + cfg.setStream(stream); + return true; + }; + bool mainPathAvailable = true; bool selfPathAvailable = data_->selfPath_; + RkISP1Path *mainPath = data_->mainPath_; + RkISP1Path *selfPath = data_->selfPath_; + Stream *mainPathStream = const_cast(&data_->mainPathStream_); + Stream *selfPathStream = const_cast(&data_->selfPathStream_); for (unsigned int index : order) { StreamConfiguration &cfg = config_[index]; /* Try to match stream without adjusting configuration. */ if (mainPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) { + if (validateConfig(cfg, mainPath, mainPathStream, Valid)) { mainPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->mainPathStream_)); continue; } } if (selfPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) { + if (validateConfig(cfg, selfPath, selfPathStream, Valid)) { selfPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->selfPathStream_)); continue; } } /* Try to match stream allowing adjusting configuration. */ if (mainPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) { + if (validateConfig(cfg, mainPath, mainPathStream, Adjusted)) { mainPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->mainPathStream_)); status = Adjusted; continue; } } if (selfPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) { + if (validateConfig(cfg, selfPath, selfPathStream, Adjusted)) { selfPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->selfPathStream_)); status = Adjusted; continue; } From patchwork Fri Dec 6 10:13:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22204 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C3FD9BF415 for ; Fri, 6 Dec 2024 10:14:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6894166158; Fri, 6 Dec 2024 11:14:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aBbnTj3z"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9C7536614C for ; Fri, 6 Dec 2024 11:14:22 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 41FA0B2B; Fri, 6 Dec 2024 11:13:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480033; bh=SD1MetfQnRDPnrrch3GNJCqw121mp6YT4Qd8TmNiDpU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aBbnTj3zZ6TuonWKUiH1vp9yo2IV5sMmyUu2YebbUHL1uzp6Bjb8tGedDeUqCpIQX E78hGw1FUfBDiOUhdf0gXcvhlY4alx9K91MGgyBhaVcoifYSjry+b0HycN6rsjTwlG CCPHUiZaRkXw/uLsaTWpUDYDQ+XMcViG1KQrZJeM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi Subject: [PATCH v3 12/17] libcamera: rkisp1: Enable the dewarper unconditionally Date: Fri, 6 Dec 2024 11:13:34 +0100 Message-ID: <20241206101344.767170-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In configure() and in the future in generateConfiguration() the calculated stream sizes and crop rectangles depend on the dewarper being used or not. It is therefore not possible to postpone that decision until the dewarper gets configured. Enable the dewarper unconditionally if it is found and the stream type is not RAW. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 7f41092ee2d5..2c8d670768b1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -865,7 +865,10 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (dewarper_ && !isRaw_) { outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); - useDewarper_ = ret ? false : true; + if (ret) + return ret; + + useDewarper_ = true; /* * Calculate the crop rectangle of the data From patchwork Fri Dec 6 10:13:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22205 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 00B98BF415 for ; Fri, 6 Dec 2024 10:14:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 99FBC6615E; Fri, 6 Dec 2024 11:14:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b0AwQfUs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C0156618B2 for ; Fri, 6 Dec 2024 11:14:25 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 67E2C74C; Fri, 6 Dec 2024 11:13:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480036; bh=6vFUOeOCRCtPZGaIxRMkuSIV7NUWa438becyOBI3Q4o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b0AwQfUst2AEYhfIhHb4xvEx0SrRkTXQSk0mhZ1YX1FDbLMY9JQrurpZMwqzV0ROa 71BXW+aIv5g/XH5a1CDfQD6oI4MFGtzvm/VZ4NbucJZ0fRxmjlZZjeUQ5KbGAiDR0e OlRHv2L+frOVKFtum5/XnkzKGfXKy207fSBTvd+Y= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 13/17] camera: Add a const version of the pipe() function Date: Fri, 6 Dec 2024 11:13:35 +0100 Message-ID: <20241206101344.767170-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow access to the pipeline handler on a const instance of Camera::Private. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- include/libcamera/internal/camera.h | 1 + src/libcamera/camera.cpp | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/libcamera/internal/camera.h b/include/libcamera/internal/camera.h index 0add0428bb5d..2bb00bbcb644 100644 --- a/include/libcamera/internal/camera.h +++ b/include/libcamera/internal/camera.h @@ -32,6 +32,7 @@ public: ~Private(); PipelineHandler *pipe() { return pipe_.get(); } + const PipelineHandler *pipe() const { return pipe_.get(); } std::list queuedRequests_; ControlInfoMap controlInfo_; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 4c865a46af53..69a7ee5353f1 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -603,6 +603,11 @@ Camera::Private::~Private() * \return The pipeline handler that created this camera */ +/** + * \fn Camera::Private::pipe() const + * \copydoc Camera::Private::pipe() + */ + /** * \fn Camera::Private::validator() * \brief Retrieve the control validator related to this camera From patchwork Fri Dec 6 10:13:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22206 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 373FDBF415 for ; Fri, 6 Dec 2024 10:14:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 79AFE66158; Fri, 6 Dec 2024 11:14:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pz2O245V"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 71909618B2 for ; Fri, 6 Dec 2024 11:14:28 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1CF209FC; Fri, 6 Dec 2024 11:13:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480039; bh=bjb3g6OquJEqFRgvtN/2g4Td9yLBkm5COZN0HzuAZSA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pz2O245VW4ZHQyHsgfcSJkv1LXgchGLvGsKVy0crs7HT6o4lYEkTKX7Jc48Zqp7XS O9N4ieo7YjZ58ztRJ6FgkfUAq5cOdFeHkWWNTXDRDMOjHHsXsSiib/PjsEU6I34nHt ZiUQjjg+uEpJB+PlsLYGzF6wHmrT/df+dYgRQGBs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 14/17] libcamera: rkisp1: make RkISP1CameraConfiguration a friend of the pipeline handler Date: Fri, 6 Dec 2024 11:13:36 +0100 Message-ID: <20241206101344.767170-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the validate() implementation, the RkISP1CameraConfiguration needs access to dewarper related members of the PipelineHandlerRkISP1 object. Allow this access by adding const versions of the pipe accessors and making RkISP1CameraConfiguration a friend of PipelineHandlerRkISP1. Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2c8d670768b1..c582e164670c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -98,6 +98,7 @@ public: } PipelineHandlerRkISP1 *pipe(); + const PipelineHandlerRkISP1 *pipe() const; int loadIPA(unsigned int hwRevision); Stream mainPathStream_; @@ -176,6 +177,7 @@ private: } friend RkISP1CameraData; + friend RkISP1CameraConfiguration; friend RkISP1Frames; int initLinks(Camera *camera, const CameraSensor *sensor, @@ -363,6 +365,11 @@ PipelineHandlerRkISP1 *RkISP1CameraData::pipe() return static_cast(Camera::Private::pipe()); } +const PipelineHandlerRkISP1 *RkISP1CameraData::pipe() const +{ + return static_cast(Camera::Private::pipe()); +} + int RkISP1CameraData::loadIPA(unsigned int hwRevision) { ipa_ = IPAManager::createIPA(pipe(), 1, 1); From patchwork Fri Dec 6 10:13:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22207 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 23651BF415 for ; Fri, 6 Dec 2024 10:14:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D271866160; Fri, 6 Dec 2024 11:14:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="egz94B5O"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 13DC266160 for ; Fri, 6 Dec 2024 11:14:31 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B3DC8B2B; Fri, 6 Dec 2024 11:14:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480041; bh=SaitqzZTDlM3ZfuQ4qpicIQfI4erfuEpujcKvfB2nmo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=egz94B5OE/6FMcdnKpMgpDIU1YKak+d8tU02rT6kGY6GNQF2M5/NGd5nZOPqt+zwu SizrfbL4uj51dD0tSh1fKlEGLywen559s1A+wrvcqRG0XFX/ebWdnpaFCwI/FtcUxP u/2jhGZ8UpXjB4hoODT7yVmvG3sKWBtffEGUY9iA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi Subject: [PATCH v3 15/17] pipeline: rkisp1: Fix config validation when dewarper is used Date: Fri, 6 Dec 2024 11:13:37 +0100 Message-ID: <20241206101344.767170-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the dewarper is used, config->validate() needs to take the restrictions of the dewarper into account. Add the corresponding checks. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 46 +++++++++++++++++++++--- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c582e164670c..2baff6da7cb7 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -497,6 +497,7 @@ bool RkISP1CameraConfiguration::fitsAllPaths(const StreamConfiguration &cfg) CameraConfiguration::Status RkISP1CameraConfiguration::validate() { + const PipelineHandlerRkISP1 *pipe = data_->pipe(); const CameraSensor *sensor = data_->sensor_.get(); unsigned int pathCount = data_->selfPath_ ? 2 : 1; Status status; @@ -553,6 +554,18 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } } + bool useDewarper = false; + if (pipe->dewarper_) { + /* + * Platforms with dewarper support, such as i.MX8MP, support a + * single stream. We can inspect config_[0] only here. + */ + bool isRaw = PixelFormatInfo::info(config_[0].pixelFormat).colourEncoding == + PixelFormatInfo::ColourEncodingRAW; + if (!isRaw) + useDewarper = true; + } + /* * If there are more than one stream in the configuration figure out the * order to evaluate the streams. The first stream has the highest @@ -565,12 +578,31 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (config_.size() == 2 && fitsAllPaths(config_[0])) std::reverse(order.begin(), order.end()); + /* + * Validate the configuration against the desired path and, if the + * platform supports it, the dewarper. + */ auto validateConfig = [&](StreamConfiguration &cfg, RkISP1Path *path, Stream *stream, Status expectedStatus) { StreamConfiguration tryCfg = cfg; - if (path->validate(sensor, sensorConfig, &tryCfg) != expectedStatus) + + Status ret = path->validate(sensor, sensorConfig, &tryCfg); + if (ret == Invalid) + return false; + + if (!useDewarper && + (expectedStatus == Valid && ret == Adjusted)) return false; + if (useDewarper) { + bool adjusted; + + pipe->dewarper_->validateOutput(&tryCfg, &adjusted, + Converter::Alignment::Down); + if (expectedStatus == Valid && adjusted) + return false; + } + cfg = tryCfg; cfg.setStream(stream); return true; @@ -820,6 +852,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + useDewarper_ = dewarper_ && !isRaw_; /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ if (!isRaw_) @@ -832,8 +865,13 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (media_->hwRevision() == RKISP1_V_IMX8MP) { /* imx8mp has only a single path. */ const auto &cfg = config->at(0); - Size ispCrop = format.size.boundedToAspectRatio(cfg.size) - .alignedUpTo(2, 2); + Size ispCrop = format.size.boundedToAspectRatio(cfg.size); + if (useDewarper_) + ispCrop = dewarper_->adjustInputSize(cfg.pixelFormat, + ispCrop); + else + ispCrop.alignUpTo(2, 2); + outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); format.size = ispCrop; } @@ -875,8 +913,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; - useDewarper_ = true; - /* * Calculate the crop rectangle of the data * flowing into the dewarper in sensor From patchwork Fri Dec 6 10:13:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22208 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2710CBF415 for ; Fri, 6 Dec 2024 10:14:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7C8666160; Fri, 6 Dec 2024 11:14:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eE6FyvHr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D1E26614E for ; Fri, 6 Dec 2024 11:14:33 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1C8C1FEF; Fri, 6 Dec 2024 11:14:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480044; bh=vGNNqTgfLWPEDfm0GWkfBDh0tk2UWublFjvscvtM8T4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eE6FyvHrjJhR+CpO5QCwyvff3FdnXJ1U70+Tu3QWteCEwXV7GkivMSgHPRtzDyW/C ttGtjNM/VcbbIYT4J0RLWhVItIojsYvepM7i+IyeGG1gN6wAkNAjTaQG+JF2++xlSy 0aQaWznDoHwTOgn3/qPPLZCQcYleHdKZ7YQcqYwo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi Subject: [PATCH v3 16/17] libcamera: camera_sensor: Add parameter to limit returned sensor size Date: Fri, 6 Dec 2024 11:13:38 +0100 Message-ID: <20241206101344.767170-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The getFormat function takes the aspect ratio and the area of the requested size into account when choosing the best sensor size. In case the sensor is connected to an rkisp1 the maximum supported frame size of the ISP is another constraining factor for the selection of the best format. Add a maxSize parameter to support such a constraint. Signed-off-by: Stefan Klug Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- include/libcamera/internal/camera_sensor.h | 2 +- src/libcamera/sensor/camera_sensor.cpp | 3 +++ src/libcamera/sensor/camera_sensor_legacy.cpp | 9 +++++++-- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index d030e254a552..605ea8136900 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -53,7 +53,7 @@ public: virtual V4L2SubdeviceFormat getFormat(const std::vector &mbusCodes, - const Size &size) const = 0; + const Size &size, const Size maxSize = Size()) const = 0; virtual int setFormat(V4L2SubdeviceFormat *format, Transform transform = Transform::Identity) = 0; virtual int tryFormat(V4L2SubdeviceFormat *format) const = 0; diff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp index 208a1603cb32..a131ac224ec0 100644 --- a/src/libcamera/sensor/camera_sensor.cpp +++ b/src/libcamera/sensor/camera_sensor.cpp @@ -116,6 +116,7 @@ CameraSensor::~CameraSensor() = default; * \brief Retrieve the best sensor format for a desired output * \param[in] mbusCodes The list of acceptable media bus codes * \param[in] size The desired size + * \param[in] maxSize The maximum size * * Media bus codes are selected from \a mbusCodes, which lists all acceptable * codes in decreasing order of preference. Media bus codes supported by the @@ -134,6 +135,8 @@ CameraSensor::~CameraSensor() = default; * bandwidth. * - The desired \a size shall be supported by one of the media bus code listed * in \a mbusCodes. + * - The desired \a size shall fit into the maximum size \a maxSize if it is not + * null. * * When multiple media bus codes can produce the same size, the code at the * lowest position in \a mbusCodes is selected. diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index 17d6fa680e39..32989c19c019 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -74,7 +74,8 @@ public: Size resolution() const override; V4L2SubdeviceFormat getFormat(const std::vector &mbusCodes, - const Size &size) const override; + const Size &size, + const Size maxSize) const override; int setFormat(V4L2SubdeviceFormat *format, Transform transform = Transform::Identity) override; int tryFormat(V4L2SubdeviceFormat *format) const override; @@ -699,7 +700,7 @@ Size CameraSensorLegacy::resolution() const V4L2SubdeviceFormat CameraSensorLegacy::getFormat(const std::vector &mbusCodes, - const Size &size) const + const Size &size, Size maxSize) const { unsigned int desiredArea = size.width * size.height; unsigned int bestArea = UINT_MAX; @@ -716,6 +717,10 @@ CameraSensorLegacy::getFormat(const std::vector &mbusCodes, for (const SizeRange &range : formats->second) { const Size &sz = range.max; + if (!maxSize.isNull() && + (sz.width > maxSize.width || sz.height > maxSize.height)) + continue; + if (sz.width < size.width || sz.height < size.height) continue; From patchwork Fri Dec 6 10:13:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22209 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F2EB2BF415 for ; Fri, 6 Dec 2024 10:14:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9BAAB6615E; Fri, 6 Dec 2024 11:14:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="exmpAeJp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B925A6615E for ; Fri, 6 Dec 2024 11:14:36 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:3543:aebe:e043:ef86]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6A38A9FC; Fri, 6 Dec 2024 11:14:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1733480047; bh=lyXsebzoyYik2koXgwhSPV/lOyjoeiT5oyUKcJDoxP4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=exmpAeJp3XULm2nPEMZqzSDLWvhXUXZUcc6NultY+nZfVYvLUfFs+gwBL5PSLO9VJ ta+1CA7CoNL2vR2NbpcIb0VIERSzs6JYNCbb5B6C0nlcA7y4YnL27t34H0Co6FTETY 365mb9I8/pnSTx/ph2L+qtO77aVflHZzUd9Fs29k= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 17/17] libcamera: pipeline: rkisp1: Limit sensor size to max resolution Date: Fri, 6 Dec 2024 11:13:39 +0100 Message-ID: <20241206101344.767170-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241206101344.767170-1-stefan.klug@ideasonboard.com> References: <20241206101344.767170-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In RkISPPath::validate() the sensor sizes are correctly filtered to the ones supported by the ISP. But later in RkISPPipeline::configure() the configured stream size is passed to sensor->getFormat() and the CameraSensor class chooses the best sensor format for the requested stream size. This can result in a sensor format that is too big for the ISP. Fix that by supplying the maximum resolution supported by the ISP to setFormat(). Fixes: 761545407c76 ("pipeline: rkisp1: Filter out sensor sizes not supported by the pipeline") Signed-off-by: Stefan Klug Reviewed-by: Jacopo Mondi Reviewed-by: Paul Elder --- Changes in v3: - Added this patch --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2baff6da7cb7..4dcc5a4fa6a1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -677,7 +677,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() [](const auto &value) { return value.second; }); } - sensorFormat_ = sensor->getFormat(mbusCodes, maxSize); + sensorFormat_ = sensor->getFormat(mbusCodes, maxSize, + mainPath->maxResolution()); if (sensorFormat_.size.isNull()) sensorFormat_.size = sensor->resolution(); diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index 45be8476616c..2a1ef0abe6d1 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -62,6 +62,7 @@ public: int queueBuffer(FrameBuffer *buffer) { return video_->queueBuffer(buffer); } Signal &bufferReady() { return video_->bufferReady; } + const Size &maxResolution() const { return maxResolution_; } private: void populateFormats();