From patchwork Wed Nov 20 08:57:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22023 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F0046C32F8 for ; Wed, 20 Nov 2024 08:58:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5058665F30; Wed, 20 Nov 2024 09:58:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GGnh6uCx"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C00E65F34 for ; Wed, 20 Nov 2024 09:58:01 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1421955A; Wed, 20 Nov 2024 09:57:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093063; bh=rTHEm2I2/JjOjsNF3WwTQuGllzAp9ljJxq9U367L40g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GGnh6uCxBnjCdixlQbMeTuT1Fn1jzJkg0cp9ADA7sj7GHO0FA3su4CCJw24B6ub+m wtxOuxvgCz3Onn9lF/iXsN6p29ybhzvz0/gGAySGmCE4BBOCGuTDrKLDNnWHPh992x 0kH5NGiw5PpcqEOQWcwDpOzP6NoMqLMy/BnsM5nI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 1/7] libcamera: rkisp1: Fix scope of dewarper stop() exit action Date: Wed, 20 Nov 2024 09:57:40 +0100 Message-ID: <20241120085753.1993436-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move the definition of the dewarper stop() action into the scope were the corresponding start() happens. Fixes: 12b553d691d4 ("libcamera: rkisp1: Plumb the dw100 dewarper as V4L2M2M converter") Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6c6d711f91b3..5ffcfbce56be 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1043,8 +1043,8 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL LOG(RkISP1, Error) << "Failed to start dewarper"; return ret; } + actions += [&]() { dewarper_->stop(); }; } - actions += [&]() { dewarper_->stop(); }; } if (data->mainPath_->isEnabled()) { From patchwork Wed Nov 20 08:57:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22024 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A0B48C32F9 for ; Wed, 20 Nov 2024 08:58:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6516165F3F; Wed, 20 Nov 2024 09:58:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="E5eAK/26"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A688665F39 for ; Wed, 20 Nov 2024 09:58:03 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B205275A; Wed, 20 Nov 2024 09:57:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093065; bh=F2YyGqPGMui8INijisWJh3kDevPUX8Q9wzEnmljIx6U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E5eAK/26f+HhPHNFwp5MCs4aKxJ6Y6Ban971zmxrZ+C6Ff/a4nGxowTPZKDHNYFev HIw2aetc11d6PzKQqVzjT2+JADv754vKb09iC8IswaoaOGzd+g3u9Bt8oQJfrEwCRe ijaM0BbIlvIVPLxz0EH2xU4zZsBBNVY9drf49rwQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 2/7] libcamera: rkisp1: Keep aspect ratio on imx8mp Date: Wed, 20 Nov 2024 09:57:41 +0100 Message-ID: <20241120085753.1993436-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the current code, the input stage of the image resizer is used to apply a crop to keep the aspect ratio in cases where the requested output aspect ratio differs from the one of the selected sensor mode. On the imx8mp the resizer hardware is not capable of cropping (for reference see also rkisp1-resizer.c:rkisp1_rsz_set_sink_crop() in the linux kernel v6.10). Therefore apply the necessary cropping on the output of the ISP (on the image stabilization block). The cropping code on the image resizer doesn't need modifications as the requested crop gets ignored by the kernel. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 +++++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 6 ++++++ 2 files changed, 21 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 5ffcfbce56be..9d36554cec6e 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -811,6 +811,21 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (!isRaw_) format.code = MEDIA_BUS_FMT_YUYV8_2X8; + /* + * On devices without DUAL_CROP (like the imx8mp) cropping needs to be + * done on the ISP/IS output. + */ + if (media_->hwRevision() == RKISP1_V_IMX8MP) { + const auto &cfg = config->at(0); + Size ispCrop = format.size.boundedToAspectRatio(cfg.size) + .alignedUpTo(2, 2); + rect = ispCrop.centeredTo(Rectangle(format.size).center()); + if (ispCrop != format.size) + LOG(RkISP1, Info) << "ISP output needs to be cropped to " + << rect; + format.size = ispCrop; + } + LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format << " crop " << rect; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index 236d05af7a2f..0651de464907 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -417,6 +417,12 @@ int RkISP1Path::configure(const StreamConfiguration &config, /* * Crop on the resizer input to maintain FOV before downscaling. * + * Note that this does not apply on the imx8mp, where the cropping needs + * to be done on the ImageStabilizer output and therefore is configured + * before this stage. For simplicity we still set the crop. This gets + * ignored by the kernel driver because the hardware is missing the + * capability. + * * \todo The alignment to a multiple of 2 pixels is required but may * change the aspect ratio very slightly. A more advanced algorithm to * compute the resizer input crop rectangle is needed, and it should From patchwork Wed Nov 20 08:57:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22025 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B1D6AC330B for ; Wed, 20 Nov 2024 08:58:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2212165F49; Wed, 20 Nov 2024 09:58:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ugdSNCjD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD93A65F3E for ; Wed, 20 Nov 2024 09:58:06 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D31F2928; Wed, 20 Nov 2024 09:57:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093068; bh=ZvuOzT6tTsES0CuCVFv8GhwTaFIPAZiE0r28BpY439I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ugdSNCjDg06SVNuSJt8ddyRH4qAahxC7tZSLBpw28/cI2Sp3kxpmbJAKEMAEZcbWR wPWbD8mu14Mi1ljyepSX+C+P+ex17BtTPsVC/UHDUloPqAZq5gYpIqvawfEHxJvWQ1 gpmranSEETDb4EXg6W4FVebdJnlojNovaeMTdQ3U= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 3/7] libcamera: geometry: Add Rectangle::mappedBetween() Date: Wed, 20 Nov 2024 09:57:42 +0100 Message-ID: <20241120085753.1993436-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handling cropping and scaling within a complicated pipeline involves transformations of rectangles between different coordinate systems. For example the full input of the dewarper (0,0)/1920x1080 might correspond to the rectangle (0, 243)/2592x1458 in sensor coordinates (of a 2592x1944 sensor). Add a function that allows the transformation of a rectangle defined in one reference frame (dewarper) into the coordinates of a second reference frame (sensor). Signed-off-by: Stefan Klug --- include/libcamera/geometry.h | 2 ++ src/libcamera/geometry.cpp | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/libcamera/geometry.h b/include/libcamera/geometry.h index 9ca5865a3d0d..8c5f34f4dec1 100644 --- a/include/libcamera/geometry.h +++ b/include/libcamera/geometry.h @@ -299,6 +299,8 @@ public: __nodiscard Rectangle scaledBy(const Size &numerator, const Size &denominator) const; __nodiscard Rectangle translatedBy(const Point &point) const; + + Rectangle mappedBetween(const Rectangle &source, const Rectangle &target) const; }; bool operator==(const Rectangle &lhs, const Rectangle &rhs); diff --git a/src/libcamera/geometry.cpp b/src/libcamera/geometry.cpp index 90ccf8c19f97..7e29fe35801e 100644 --- a/src/libcamera/geometry.cpp +++ b/src/libcamera/geometry.cpp @@ -837,6 +837,39 @@ Rectangle Rectangle::translatedBy(const Point &point) const return { x + point.x, y + point.y, width, height }; } +/** + * \brief Maps a Rectangle from one reference frame to another + * \param[in] source The source reference frame + * \param[in] destination The destination reference frame + * + * The params source and destinations specify two reference frames for the same + * data. The rectangle is translated from the source reference frame into the + * destination reference frame. + * + * For example, consider a sensor with a resolution of 2592x1458 pixels and a + * assume a rectangle of (0, 243)/2592x1458 (sensorFrame) in sensor coordinates + * is mapped to a rectangle (0,0)/(1920,1080) (displayFrame) in display + * coordinates. This function can be used to transform an arbitrary rectangle + * from display coordinates to sensor coordinates or vice versa: + * + * \code{.cpp} + * displayRect = sensorRect.mappedBetween(sensorFrame, displayFrame); + * \endcode + */ +Rectangle Rectangle::mappedBetween(const Rectangle &source, + const Rectangle &destination) const +{ + Rectangle r; + double sx = static_cast(destination.width) / source.width; + double sy = static_cast(destination.height) / source.height; + + r.x = static_cast((x - source.x) * sx) + destination.x; + r.y = static_cast((y - source.y) * sy) + destination.y; + r.width = static_cast(width * sx); + r.height = static_cast(height * sy); + return r; +} + /** * \brief Compare rectangles for equality * \return True if the two rectangles are equal, false otherwise From patchwork Wed Nov 20 08:57:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22026 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2F53C32F5 for ; Wed, 20 Nov 2024 08:58:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 335A865F4B; Wed, 20 Nov 2024 09:58:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Fzmtp63V"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F16A565F43 for ; Wed, 20 Nov 2024 09:58:09 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0E25455A; Wed, 20 Nov 2024 09:57:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093072; bh=b9BVMnXt5XXFsm6kduEaTS6/A863DI2w2MXEoLKD9FU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fzmtp63VsSm54LmYYHsxjsDfVXDjw98rMfq9Jz8wij41o/xfvLmgdBw7BjxFJaard pFunTL+gFV64MtRKCpb82IkG6W57GsCsVhLuPur29H9BS94s2rINKbtDH9XohHTGYf TesrUPbSY1TLZ7LWbIBhbCH9CPFcRe4CxOSLY4cQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 4/7] pipeline: rkisp1: Split inputCrop and outputCrop Date: Wed, 20 Nov 2024 09:57:43 +0100 Message-ID: <20241120085753.1993436-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" One Rectangle instance is used to calculate the inputCrop and the outputCrop of the ISP in the rkisp1 pipeline. Split that into two distinct variables, because both values will be needed in the upcoming patches. This patch does not contain any functional changes. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 9d36554cec6e..6491a48ee83c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -794,14 +794,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret < 0) return ret; - Rectangle rect(0, 0, format.size); - ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &rect); + Rectangle inputCrop(0, 0, format.size); + Rectangle outputCrop = inputCrop; + ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &inputCrop); if (ret < 0) return ret; LOG(RkISP1, Debug) << "ISP input pad configured with " << format - << " crop " << rect; + << " crop " << inputCrop; const PixelFormat &streamFormat = config->at(0).pixelFormat; const PixelFormatInfo &info = PixelFormatInfo::info(streamFormat); @@ -819,18 +820,18 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const auto &cfg = config->at(0); Size ispCrop = format.size.boundedToAspectRatio(cfg.size) .alignedUpTo(2, 2); - rect = ispCrop.centeredTo(Rectangle(format.size).center()); + outputCrop = ispCrop.centeredTo(Rectangle(format.size).center()); if (ispCrop != format.size) LOG(RkISP1, Info) << "ISP output needs to be cropped to " - << rect; + << outputCrop; format.size = ispCrop; } LOG(RkISP1, Debug) << "Configuring ISP output pad with " << format - << " crop " << rect; + << " crop " << outputCrop; - ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &rect); + ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &outputCrop); if (ret < 0) return ret; @@ -841,7 +842,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "ISP output pad configured with " << format - << " crop " << rect; + << " crop " << outputCrop; std::map streamConfig; std::vector> outputCfgs; From patchwork Wed Nov 20 08:57:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22027 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B520CC32F8 for ; Wed, 20 Nov 2024 08:58:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5316B65F40; Wed, 20 Nov 2024 09:58:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RoI1R87W"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4A1FD65F4C for ; Wed, 20 Nov 2024 09:58:12 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5234C75A; Wed, 20 Nov 2024 09:57:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093074; bh=YsRH3k0BMH2X0TuJFgfW6hsk+gw9uUhLBTj+/VBU7wA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RoI1R87WPgrPH6RzsspX47QsbRiJykemi860aX+XLBnxAGxUq2PWBe397axrKlUT9 PyRZanhW2/ISRgZ97kEcMBCPOeKMU6UbRJcaMA7I+QvhT/GLXy+QYbwn3GeMMnl0RX eElKkX9fDOrlZmwS8x41qblizt07dswn0vJXkPIo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 5/7] pipeline: rkisp1: Reorder sensorInfo collection code Date: Wed, 20 Nov 2024 09:57:44 +0100 Message-ID: <20241120085753.1993436-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The sensorInfo (specifically the crop rectangle of the selected sensor mode) is collected to be passed to the IPA later. In an upcoming patch that data will also be needed for correct ScalerCrop handling. Move the collection of the sensorInfo before the dewarper configuration step and refactor the code a bit. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 6491a48ee83c..4a226d9b809f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -844,6 +844,11 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) << "ISP output pad configured with " << format << " crop " << outputCrop; + IPACameraSensorInfo sensorInfo; + ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) + return ret; + std::map streamConfig; std::vector> outputCfgs; @@ -883,14 +888,9 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* Inform IPA of stream configuration and sensor controls. */ - ipa::rkisp1::IPAConfigInfo ipaConfig{}; - - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); - if (ret) - return ret; - - ipaConfig.sensorControls = data->sensor_->controls(); - ipaConfig.paramFormat = paramFormat.fourcc; + ipa::rkisp1::IPAConfigInfo ipaConfig{ sensorInfo, + data->sensor_->controls(), + paramFormat.fourcc }; ret = data->ipa_->configure(ipaConfig, streamConfig, &data->ipaControls_); if (ret) { From patchwork Wed Nov 20 08:57:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22028 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F73AC330C for ; Wed, 20 Nov 2024 08:58:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1FD3B65F4B; Wed, 20 Nov 2024 09:58:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="R0FvuOse"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B9D1665F40 for ; Wed, 20 Nov 2024 09:58:14 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C494A75A; Wed, 20 Nov 2024 09:57:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093076; bh=xPhnwj2ouJPX8A8sMaZGYgdC+NbcDE6V0SSmFlfMdeo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R0FvuOseKNmcghl6yCcH4nu977/LNYn3L/F4/menvIvkrlM2jvf+oUJvvLk06OBUw jXMEXreaJj1eR1MkiSJYChy2ueT8F8gw6F6WpNylgFz1GeA4gI6Y8PL75M0CHvk+oR 82Y1ljOcNdgzN2p8MFIzThvrjFpJdEro/Tirzr6c= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 6/7] pipeline: rkisp1: Fix ScalerCrop to be in sensor coordinates Date: Wed, 20 Nov 2024 09:57:45 +0100 Message-ID: <20241120085753.1993436-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" ScalerCrop is specified as beeing in sensor coordinates. The current dewarper implementation on the imx8mp handles ScalerCrop in dewarper coordinates. This leads to unexpected results and a unusable ScalerCrop control in camshark. Fix that by transforming back and forth between sensor coordinates and dewarper coordinates. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 49 +++++++++++++++++++----- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4a226d9b809f..c2ce38b1c253 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -204,6 +204,7 @@ private: RkISP1SelfPath selfPath_; std::unique_ptr dewarper_; + Rectangle dewarperSensorCrop_; bool useDewarper_; std::optional activeCrop_; @@ -862,6 +863,14 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) outputCfgs.push_back(const_cast(cfg)); ret = dewarper_->configure(cfg, outputCfgs); useDewarper_ = ret ? false : true; + + /* + * Calculate the crop rectangle of the data + * flowing into the dewarper in sensor + * coordinates. + */ + dewarperSensorCrop_ = outputCrop.mappedBetween(inputCrop, + ipaConfig.sensorInfo.analogCrop); } } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); @@ -1224,10 +1233,21 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) std::pair cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, - cropLimits.second, - cropLimits.second); - activeCrop_ = cropLimits.second; + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform the limits to sensor coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ + Rectangle min = cropLimits.first.mappedBetween(cropLimits.second, + dewarperSensorCrop_); + Rectangle max = cropLimits.second.mappedBetween(cropLimits.second, + dewarperSensorCrop_); + + controls[&controls::ScalerCrop] = ControlInfo(min, + max, + max); + activeCrop_ = max; } /* Add the IPA registered controls to list of camera controls. */ @@ -1478,22 +1498,33 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { - Rectangle appliedRect = crop.value(); + Rectangle rect = crop.value(); + + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform it into dewarper coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ + std::pair cropLimits = + dewarper_->inputCropBounds(&data->mainPathStream_); + rect = rect.mappedBetween(dewarperSensorCrop_, cropLimits.second); int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &appliedRect); - if (!ret && appliedRect != crop.value()) { + &rect); + rect = rect.mappedBetween(cropLimits.second, dewarperSensorCrop_); + if (!ret && rect != crop.value()) { /* * If the rectangle is changed by setInputCrop on the * dewarper, log a debug message and cache the actual * applied rectangle for metadata reporting. */ LOG(RkISP1, Debug) - << "Applied rectangle " << appliedRect.toString() + << "Applied rectangle " << rect.toString() << " differs from requested " << crop.value().toString(); } - activeCrop_ = appliedRect; + activeCrop_ = rect; } /* From patchwork Wed Nov 20 08:57:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22029 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A394EC330D for ; Wed, 20 Nov 2024 08:58:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 31DAC65F51; Wed, 20 Nov 2024 09:58:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jPMhIorM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 00AF565F30 for ; Wed, 20 Nov 2024 09:58:17 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bbd:82cc:f3f3:e12e]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0A43755A; Wed, 20 Nov 2024 09:57:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1732093079; bh=tZphfWIhGEQeLXpOwirq4VVHqyST/gTXN1DCGq09QZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jPMhIorMjrMR3ZKgxC4L1dHPu4TG6srWU8FvNE4zAYhL/UoTcQNS8OTg4zy41nWGZ vh2z8a/7GRurdO+XPJPL1m7aqtJThNBDdV4aprJqG9PlPay2KzTfTJR3kIrz/x3ABL bIjm+MkJfxAaTwjns0f8oUg/BmAc+pBHXaMGhH50= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH 7/7] pipeline: rkisp1: Add ScalerMaximumCrop property Date: Wed, 20 Nov 2024 09:57:46 +0100 Message-ID: <20241120085753.1993436-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> References: <20241120085753.1993436-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ScalerMaximumCrop property holds the biggest allowed ScalerCrop value. Add it to the rkisp1. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c2ce38b1c253..647b5754bd55 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -870,7 +871,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) * coordinates. */ dewarperSensorCrop_ = outputCrop.mappedBetween(inputCrop, - ipaConfig.sensorInfo.analogCrop); + sensorInfo.analogCrop); } } else if (hasSelfPath_) { ret = selfPath_.configure(cfg, format); @@ -1247,6 +1248,7 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) controls[&controls::ScalerCrop] = ControlInfo(min, max, max); + data->properties_.set(properties::ScalerCropMaximum, max); activeCrop_ = max; }