From patchwork Mon Nov 16 17:48:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28914 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A6800C3304 for ; Mon, 16 Nov 2026 17:49:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04D8F6A837; Mon, 16 Nov 2026 18:49:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Im+eLfwv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0726C69089 for ; Mon, 16 Nov 2026 18:49:24 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9835263C; Mon, 16 Nov 2026 18:46:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851217; bh=XyIloSZtlSTOte0wp6uoYaxkjXVhKclxZ7Qz6f6REXQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Im+eLfwvJO51JfrvQHs0qOu1eW2bMJactTOEMM8BzBgE7iJUr5jVZpnUp0/rp3dp+ JFPYrlgABXGiDP47mYmypUqQg/eDwWN5SdduAIJK2kHPd2B1ZyWfVcQ8z8OP2/aIsl J6LVMB36ZDfCxpa6BLGRu7/z4u82vkNI6I/UAwVQ= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Jacopo Mondi Subject: [PATCH v5 01/22] ipa: libipa: fixedpoint: Shift unsigned type for scaling Date: Mon, 16 Nov 2026 18:48:57 +0100 Message-ID: <20261116174919.1056403-2-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Avoid shifting into the potential sign bit of `T`. The same is already done in `toFloat()`. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/fixedpoint.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index ed8aca9f15..537bc774a4 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -78,7 +78,7 @@ public: * properly cast negative values. See * https://embeddeduse.com/2013/08/25/casting-a-negative-float-to-an-unsigned-int/ */ - return static_cast(static_cast(std::round(v * (1 << F)))) & bitMask; + return static_cast(static_cast(std::round(v * (UT{ 1 } << F)))) & bitMask; } }; From patchwork Mon Nov 16 17:48:58 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28915 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 45537C3304 for ; Mon, 16 Nov 2026 17:49:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 000626A857; Mon, 16 Nov 2026 18:49:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PIR3mf07"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4EF2C6A839 for ; Mon, 16 Nov 2026 18:49:24 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 50473175; Mon, 16 Nov 2026 18:46:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851217; bh=7ipo5lgLqCxVElN5BZF9f2Q10EbmQFiYutelpf7H/tg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PIR3mf07vzFEsjxbym0Gx1NUssfi9FNUlS4QO9/VrETWt/vYFCdkE0sM9rbqbSclh be2YBykdGTq6St1CJW/nWfgYIyBgSYiuFl4h5h78Rnyfnfv3/cPnfXrkGPjaqsJLmu s3eTDGJVcZvOkHHxKumFK7z8HguACutFvMRTgRXU= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Jacopo Mondi Subject: [PATCH v5 02/22] ipa: libipa: quantized: Make floating point type customizable Date: Mon, 16 Nov 2026 18:48:58 +0100 Message-ID: <20261116174919.1056403-3-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Adjust the `Quantized` type to retrieve the floating point from the `FloatingType` type alias in the `Traits` type, instead of hard-coding `float`. Additionally, adjust the various traits type, to provide the now required type alias. They are currently hard-coded to use `float`. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/fixedpoint.cpp | 7 ++++++- src/ipa/libipa/fixedpoint.h | 11 ++++++----- src/ipa/libipa/quantized.cpp | 10 +++++++++- src/ipa/libipa/quantized.h | 14 +++++++++----- test/ipa/libipa/quantized.cpp | 8 ++++++-- 5 files changed, 36 insertions(+), 14 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.cpp b/src/ipa/libipa/fixedpoint.cpp index b06ad8d267..8c0e9746c7 100644 --- a/src/ipa/libipa/fixedpoint.cpp +++ b/src/ipa/libipa/fixedpoint.cpp @@ -47,6 +47,11 @@ namespace ipa { * \brief The integral storage type used for the fixed-point representation */ +/** + * \typedef FixedPointQTraits::FloatingType + * \brief The type used for the floating-point representation + */ + /** * \var FixedPointQTraits::qMin * \brief Minimum representable quantized integer value @@ -71,7 +76,7 @@ namespace ipa { */ /** - * \fn FixedPointQTraits::fromFloat(float v) + * \fn FixedPointQTraits::fromFloat(FloatingType v) * \brief Convert a floating-point value to a fixed-point integer * \param[in] v The floating-point value to be converted * \return The quantized fixed-point integer representation diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 537bc774a4..66863075c6 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -37,6 +37,7 @@ private: public: using QuantizedType = UT; + using FloatingType = float; static constexpr UT qMin = std::is_signed_v ? -(UT{ 1 } << (bits - 1)) @@ -49,7 +50,7 @@ public: static constexpr float toFloat(QuantizedType q) { if constexpr (std::is_unsigned_v) - return static_cast(q) / static_cast(UT{ 1 } << F); + return static_cast(q) / static_cast(UT{ 1 } << F); /* * Recreate the upper bits in case of a negative number by @@ -60,16 +61,16 @@ public: */ unsigned int remaining_bits = sizeof(UT) * 8 - (I + F); T t = static_cast(static_cast(q) << remaining_bits) >> remaining_bits; - return static_cast(t) / static_cast(UT{ 1 } << F); + return static_cast(t) / static_cast(UT{ 1 } << F); } - static constexpr float min = toFloat(qMin); - static constexpr float max = toFloat(qMax); + static constexpr FloatingType min = toFloat(qMin); + static constexpr FloatingType max = toFloat(qMax); static_assert(min < max, "FixedPointQTraits: Minimum must be less than maximum"); /* Conversion functions required by Quantized */ - static QuantizedType fromFloat(float v) + static QuantizedType fromFloat(FloatingType v) { v = std::clamp(v, min, max); diff --git a/src/ipa/libipa/quantized.cpp b/src/ipa/libipa/quantized.cpp index 06143a97ab..34cf6b1b97 100644 --- a/src/ipa/libipa/quantized.cpp +++ b/src/ipa/libipa/quantized.cpp @@ -58,7 +58,15 @@ namespace ipa { */ /** - * \fn Quantized::Quantized(float x) + * \typedef Quantized::FloatingType + * \brief The floating type used for the value representation + * + * This alias corresponds to \c TraitsType::FloatingType, as defined by + * the traits class. + */ + +/** + * \fn Quantized::Quantized(FloatingType x) * \brief Construct a Quantized value from a floating-point number * \param[in] x The floating-point value to be quantized * diff --git a/src/ipa/libipa/quantized.h b/src/ipa/libipa/quantized.h index 8d7b06a2df..5c41d0b5dd 100644 --- a/src/ipa/libipa/quantized.h +++ b/src/ipa/libipa/quantized.h @@ -21,13 +21,17 @@ template struct Quantized { using TraitsType = Traits; using QuantizedType = typename Traits::QuantizedType; - static_assert(std::is_arithmetic_v, - "Quantized: QuantizedType must be arithmetic"); + using FloatingType = typename Traits::FloatingType; + + static_assert(std::is_integral_v, + "Quantized: QuantizedType must be integral"); + static_assert(std::is_floating_point_v, + "Quantized: FloatingType must be floating"); constexpr Quantized() - : Quantized(0.0f) {} + : Quantized(FloatingType{}) {} - constexpr Quantized(float x) + constexpr Quantized(FloatingType x) : Quantized(Traits::fromFloat(x)) { } @@ -73,7 +77,7 @@ struct Quantized { private: QuantizedType quantized_; - float value_; + FloatingType value_; }; } /* namespace ipa */ diff --git a/test/ipa/libipa/quantized.cpp b/test/ipa/libipa/quantized.cpp index 18655dac52..4b1d869bf2 100644 --- a/test/ipa/libipa/quantized.cpp +++ b/test/ipa/libipa/quantized.cpp @@ -20,7 +20,9 @@ using namespace ipa; struct BrightnessHueTraits { using QuantizedType = uint8_t; - static QuantizedType fromFloat(float v) + using FloatingType = float; + + static QuantizedType fromFloat(FloatingType v) { int quantized = std::lround(v * 128.0f); return std::clamp(quantized, -128, 127); @@ -35,7 +37,9 @@ using BrightnessHueQuantizer = Quantized; struct ContrastSaturationTraits { using QuantizedType = uint8_t; - static QuantizedType fromFloat(float v) + using FloatingType = float; + + static QuantizedType fromFloat(FloatingType v) { int quantized = std::lround(v * 128.0f); return std::clamp(quantized, 0, 255); From patchwork Mon Nov 16 17:48:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28916 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6721DC3355 for ; Mon, 16 Nov 2026 17:49:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0BA986A861; Mon, 16 Nov 2026 18:49:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UTjamA4k"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B343C6A840 for ; Mon, 16 Nov 2026 18:49:24 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 905A0802; Mon, 16 Nov 2026 18:46:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851217; bh=+ARIYmR/6U1+rHhiq14e5JfJWpeBtpP2WyZ+kOlryVg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UTjamA4ky0If1FBJNu6crmzWaJ7imhDVgftcchUH8KmtB8O+C0Ym1DtM2rEmo/rpC AhjPISRQODsgrSvFu1V6LY06CHqUnVVlUu2uSbF0YFVsHNqSjnRre6FstxJoSZW94w 0ICH/+rdl24tvVCH8wm05ac7wGBcdGqnoFtZlpgU= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham Subject: [PATCH v5 03/22] ipa: libipa: quantized: Use `double` when necessary Date: Mon, 16 Nov 2026 18:48:59 +0100 Message-ID: <20261116174919.1056403-4-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The currently used `float` type (usually IEEE754 binary32) can only represent integers exactly up to 2^24. So automatically use `double` (usually IEEE754 binary64) if more than 24 bits are needed for the fixed-point representation. Also add a static assertion that IEEE754 floating point types are used. But note, that does not guarantee that float and double are binary32 and binary64. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Acked-by: Jacopo Mondi --- src/ipa/libipa/fixedpoint.h | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 66863075c6..0640605bee 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -25,19 +25,20 @@ private: static constexpr unsigned int bits = I + F; static_assert(bits <= sizeof(UT) * 8, "FixedPointQTraits: too many bits for type UT"); - /* - * If fixed point storage is required with more than 24 bits, consider - * updating this implementation to use double-precision floating point. - */ - static_assert(bits <= 24, "Floating point precision may be insufficient for more than 24 bits"); + /* IEEE754 binary64 can faithfully represent integers up to 2^53 */ + static_assert(bits <= 53, "Floating point precision may be insufficient"); static constexpr UT bitMask = bits < sizeof(UT) * 8 ? (UT{ 1 } << bits) - 1 : ~UT{ 0 }; public: + /* IEEE754 binary32 can faithfully represent integers up to 2^24 */ using QuantizedType = UT; - using FloatingType = float; + using FloatingType = std::conditional_t; + + static_assert(std::numeric_limits::is_iec559, + "requires IEEE754 floating point"); static constexpr UT qMin = std::is_signed_v ? -(UT{ 1 } << (bits - 1)) @@ -88,7 +89,7 @@ namespace details { template constexpr auto qtype() { - static_assert(Bits <= 32, + static_assert(Bits <= 64, "Unsupported number of bits for quantized type"); if constexpr (Bits <= 8) @@ -97,6 +98,8 @@ constexpr auto qtype() return int16_t(); else if constexpr (Bits <= 32) return int32_t(); + else if constexpr (Bits <= 64) + return int64_t(); } } /* namespace details */ From patchwork Mon Nov 16 17:49:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28918 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0487CC3304 for ; Mon, 16 Nov 2026 17:49:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D84676A85C; Mon, 16 Nov 2026 18:49:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sDvurNxO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE0E66A851 for ; Mon, 16 Nov 2026 18:49:25 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 50BFF929; Mon, 16 Nov 2026 18:46:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851218; bh=CeRj0042k0gogHjx8dJpuEK+OPXgTngmrKq2+vhNdck=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sDvurNxOxoyxyBEsqLHmCgQKjhmzJ9GsaCEjY6c/b/fb5COXA8p5iIOmjRJJEh0tZ ycmXrC5ucaFgf1b8hsw+2+Ds+HZhIsFgFpCkODpAUoeN4rarvqxEYf/cHnIv76qV8M t+Nz4ijjm2JX+0AnpDb6CYSH0oowJgy/qb4hgLpk= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Jacopo Mondi Subject: [PATCH v5 04/22] ipa: libipa: awb: Log rgb means immediately Date: Mon, 16 Nov 2026 18:49:00 +0100 Message-ID: <20261116174919.1056403-5-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Log the rgb means immediately upon entering `process()`, and also log whether it is valid or not. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/awb.cpp | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/ipa/libipa/awb.cpp b/src/ipa/libipa/awb.cpp index 3b95fecee8..c4153bcf8d 100644 --- a/src/ipa/libipa/awb.cpp +++ b/src/ipa/libipa/awb.cpp @@ -367,7 +367,13 @@ void AwbAlgorithmBase::process(awb::ActiveState &state, const AwbStats &stats, unsigned int lux, ControlList &metadata) { - if (!stats.valid()) + const bool valid = stats.valid(); + + LOG(Awb, Debug) << std::showpoint + << "means: " << stats.rgbMeans() + << (!valid ? " (invalid)" : ""); + + if (!valid) return; auto awbResult = impl_->calculateAwb(stats, lux, { currentMode_->ctLo, @@ -394,8 +400,8 @@ void AwbAlgorithmBase::process(awb::ActiveState &state, static_cast(frameContext.gains.b()) }); metadata.set(controls::ColourTemperature, frameContext.colourTemperature); - LOG(Awb, Debug) << std::showpoint << "Means " << stats.rgbMeans() - << ", gains " << state.automatic.gains + LOG(Awb, Debug) << std::showpoint + << "gains " << state.automatic.gains << ", temp " << state.automatic.colourTemperature << "K"; } From patchwork Mon Nov 16 17:49:01 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28919 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87A16C3304 for ; Mon, 16 Nov 2026 17:49:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AD2356A871; Mon, 16 Nov 2026 18:49:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bv12K8bg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 192E96A853 for ; Mon, 16 Nov 2026 18:49:26 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AA81D175; Mon, 16 Nov 2026 18:46:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851219; bh=d40woxqDsAZGlr0FbSWfVuag/40Za3bodm3Ee4Wnkt4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bv12K8bg7ngTy5DVgc9MRkouSA4b+BlLkOCYLGLtJMDU3GzUB3QXSv/l5cS/Vg0gk i6M/uVYGB7bssV9kKOppvtu5IfwY94xIrWK5X6vtCl1cTiqfIQ+xKi6tsdKnwbDGmk Hrk9aHjz+/9CArnJp/hbFLJPwCLJhUnhG1LyTFzo= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Jacopo Mondi Subject: [PATCH v5 05/22] ipa: libipa: awb_bayes: Fix initial max value Date: Mon, 16 Nov 2026 18:49:01 +0100 Message-ID: <20261116174919.1056403-6-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For floating-point types `std::numeric_limits::min()` is the smallest normal positive value. So `lowest()` needs to be used otherwise e.g. a value of 0 will not be properly handled. This type is only used for logging, so no effects on the algorithm. Fixes: c2059585f3c7 ("libipa: awb_bayes: Add logging of value limits") Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/awb_bayes.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/libipa/awb_bayes.cpp b/src/ipa/libipa/awb_bayes.cpp index 10e05962b8..7f3237c0ab 100644 --- a/src/ipa/libipa/awb_bayes.cpp +++ b/src/ipa/libipa/awb_bayes.cpp @@ -66,7 +66,7 @@ class LimitsRecorder public: LimitsRecorder() : min_(std::numeric_limits::max()), - max_(std::numeric_limits::min()) + max_(std::numeric_limits::lowest()) { } From patchwork Mon Nov 16 17:49:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28917 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 49DE1C3304 for ; Mon, 16 Nov 2026 17:49:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EAC786A869; Mon, 16 Nov 2026 18:49:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pK44LhF3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56AEE6A855 for ; Mon, 16 Nov 2026 18:49:26 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 33C7663C; Mon, 16 Nov 2026 18:46:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851219; bh=uLeQNlNUnjL2g17dXuAePQH3nVvLgyPBp5J1AXGG8Og=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pK44LhF3xo+cMbAgVk7LMFnfWGmLcYn87YpaU6SgkJpXSmYipFuhFQ2nH3t0F/C+y UupEKDszyBPgoGk/6w+/UBGjbTOCOg8MhHxeE9dJsGe+FVM1HUpXB8mB+D55Z1xMxf hJJoKRubmj2+XW1lHyi5wy6+FOiXY8Ys/WsJYpDQ= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham Subject: [PATCH v5 06/22] ipa: libipa: gamma: Static assert lookup node count Date: Mon, 16 Nov 2026 18:49:02 +0100 Message-ID: <20261116174919.1056403-7-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Require at least 2, which is the minimum necessary for the algorithm to function at all. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/gamma.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ipa/libipa/gamma.h b/src/ipa/libipa/gamma.h index 9954f94ba2..e5ac3f96f5 100644 --- a/src/ipa/libipa/gamma.h +++ b/src/ipa/libipa/gamma.h @@ -63,6 +63,8 @@ protected: template class GammaAlgorithm : public GammaAlgorithmBase { + static_assert(NLutNodes >= 2); + public: GammaAlgorithm() : GammaAlgorithmBase(NLutNodes) From patchwork Mon Nov 16 17:49:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28920 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F25DAC3304 for ; Mon, 16 Nov 2026 17:49:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B37B6A876; Mon, 16 Nov 2026 18:49:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bwBHVxCn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8E32C6A856 for ; Mon, 16 Nov 2026 18:49:26 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7A85A9A4; Mon, 16 Nov 2026 18:46:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851219; bh=PVa917y7hCXRg4WsdEjFDtvldwWJoiDt4nnToykjrK0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bwBHVxCnGCBxdcKCaKWEB4bB2js6KYbmsA/WnD8zieg1qUqY+Iero0kI/GCdQHHQb vn4CNtQBDFOyAW7Ra/9TrEl1KUiLSi0+udspCv9157lIlT98RF19itBnJp+vIVgqUG VbXJY541R9hrEEi7cm8guL2r6rCTXzSuW/HcJQBc= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Jacopo Mondi Subject: [PATCH v5 07/22] ipa: libipa: gamma: Accept const segment lengths Date: Mon, 16 Nov 2026 18:49:03 +0100 Message-ID: <20261116174919.1056403-8-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The segment lengths are not modified, so accept `const` qualified arrays, etc. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/gamma.cpp | 2 +- src/ipa/libipa/gamma.h | 2 +- src/ipa/rkisp1/algorithms/goc.cpp | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ipa/libipa/gamma.cpp b/src/ipa/libipa/gamma.cpp index ce66f15d6a..20807ac601 100644 --- a/src/ipa/libipa/gamma.cpp +++ b/src/ipa/libipa/gamma.cpp @@ -109,7 +109,7 @@ const float kDefaultGamma = 2.2f; * @return 0 on success, a negative error code otherwise */ int GammaAlgorithmBase::init(ControlInfoMap::Map &controls, const ValueNode &tuningData, - std::span segments) + std::span segments) { /* * If the caller doesn't pass in a segment list we simply construct the diff --git a/src/ipa/libipa/gamma.h b/src/ipa/libipa/gamma.h index e5ac3f96f5..a8872a6c75 100644 --- a/src/ipa/libipa/gamma.h +++ b/src/ipa/libipa/gamma.h @@ -47,7 +47,7 @@ public: } int init(ControlInfoMap::Map &controls, const ValueNode &tuningData, - std::span segments = {}); + std::span segments = {}); void configure(gamma::ActiveState &state); void queueRequest(gamma::ActiveState &state, const uint32_t frame, diff --git a/src/ipa/rkisp1/algorithms/goc.cpp b/src/ipa/rkisp1/algorithms/goc.cpp index 269ae1ea03..f6961ce17f 100644 --- a/src/ipa/rkisp1/algorithms/goc.cpp +++ b/src/ipa/rkisp1/algorithms/goc.cpp @@ -50,7 +50,7 @@ int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) return -EINVAL; } - std::array segments = { + static constexpr std::array segments = { 64, 64, 64, 64, 128, 128, 128, 128, 256, 256, 256, 512, 512, 512, 512, 512 }; From patchwork Mon Nov 16 17:49:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28921 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26CE3C3355 for ; Mon, 16 Nov 2026 17:49:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 997166A85A; Mon, 16 Nov 2026 18:49:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HkBqaawl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE5B06A842 for ; Mon, 16 Nov 2026 18:49:26 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BD606802 for ; Mon, 16 Nov 2026 18:46:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851220; bh=c2ZxZ/1NdkGQDwE7NcdHFqusBLOKmk+LJc7yQnHH6O8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=HkBqaawlAskfjFvRnKQYI/1uPv7wrMKjc2wr7jDVwDbPlkZMAWFh9VVr9HsrZh3Hw TKfAmvzDQGNSjZLWgD/zgUf7xP/OmPLQUaQF0br7jEz7jzfzCJrx/Zypq2/n++KoWK yv4ODS5Hrvsc4Wrzr3CYyv5QC/8ZMZBq8deCZx3E= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [PATCH v5 08/22] ipa: libipa: gamma: Use `std::optional` for segment lengths Date: Mon, 16 Nov 2026 18:49:04 +0100 Message-ID: <20261116174919.1056403-9-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add an `init()` function to `GammaAlgorithm`, and make the one in `GammaAlgorithmBase` protected. And take the list of segment lengths in an `std::optional`. This allows the use of a constant sized span, and thus compile-time checking when calling `init()`. Additionally, change "@return" to "\return" in the documentation. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/gamma.cpp | 83 +++++++++++++++++++++++----------------- src/ipa/libipa/gamma.h | 15 ++++++-- 2 files changed, 59 insertions(+), 39 deletions(-) diff --git a/src/ipa/libipa/gamma.cpp b/src/ipa/libipa/gamma.cpp index 20807ac601..ad34548454 100644 --- a/src/ipa/libipa/gamma.cpp +++ b/src/ipa/libipa/gamma.cpp @@ -69,44 +69,13 @@ const float kDefaultGamma = 2.2f; */ /** - * \brief Initialise the algorithm with the given tuning data - * \param[out] controls The ControlList into which this algorithm's supported - * controls will be emplaced. - * \param[in] tuningData The tuning data to use with the algorithm - * \param[in] segments A vector of segment spacings to define a custom - * X coordinate system for the curve - * - * Parse \a tuningData and \a segments to initialize the gamma correction curve. - * The tuning data may contain a default gamma value to use; otherwise the value - * of \a kDefaultGamma will be taken as the default. The piecewise linear - * function will be applied on a number of knots whose position is described by - * the optional \a segments argument, which describes each segment's relative - * length. - * - * For example, if the gamma correction has to be applied on 16 equally spaced - * sampling points, a \a segments array like: - * - * [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + * \copybrief GammaAlgorithm::init() * - * would result in evenly spaced knee-points along the X-axis. - * - * Hardware may expect the knee-points to be spaced more densely towards the - * start of the curve and more sparsely towards the end, in which case an - * alternative array might be: - * - * [1, 1, 1, 1, 2, 2, 2, 2, 4, 4, 4, 8, 8, 8, 8, 8] - * - * As the values in \a segments represent the distance between two knee-points - * relative to the total distance between the first and last point, the length - * of \a segments should be equal to the number of knee-points minus one. - *; - * If an IPA implementation doesnt't provide \a segments, the GammaAlgorithm - * class consturcts an evenly-spaced default. + * This function provides the underlying non-templated implementation for + * GammaAlgorithm::init(). Refer to the documentation of that function + * for the details. * - * IPA modules are expected to call this function as part of their - * implementation of Algorithm::init() - * - * @return 0 on success, a negative error code otherwise + * \sa GammaAlgorithm::init() */ int GammaAlgorithmBase::init(ControlInfoMap::Map &controls, const ValueNode &tuningData, std::span segments) @@ -247,6 +216,48 @@ void GammaAlgorithmBase::process(gamma::FrameContext &context, ControlList &meta * - https://en.wikipedia.org/wiki/SRGB */ +/** + * \fn GammaAlgorithm::init() + * \brief Initialise the algorithm with the given tuning data + * \param[out] controls The ControlList into which this algorithm's supported + * controls will be emplaced. + * \param[in] tuningData The tuning data to use with the algorithm + * \param[in] segments A vector of segment spacings to define a custom + * X coordinate system for the curve + * + * Parse \a tuningData and \a segments to initialize the gamma correction curve. + * The tuning data may contain a default gamma value to use; otherwise the value + * of \a kDefaultGamma will be taken as the default. The piecewise linear + * function will be applied on a number of knots whose position is described by + * the optional \a segments argument, which describes each segment's relative + * length. + * + * For example, if the gamma correction has to be applied on 16 equally spaced + * sampling points, a \a segments array like: + * + * [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + * + * would result in evenly spaced knee-points along the X-axis. + * + * Hardware may expect the knee-points to be spaced more densely towards the + * start of the curve and more sparsely towards the end, in which case an + * alternative array might be: + * + * [1, 1, 1, 1, 2, 2, 2, 2, 4, 4, 4, 8, 8, 8, 8, 8] + * + * As the values in \a segments represent the distance between two knee-points + * relative to the total distance between the first and last point, the length + * of \a segments should be equal to the number of knee-points minus one. + *; + * If an IPA implementation doesn't provide \a segments, the GammaAlgorithm + * class constructs an evenly-spaced default. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::init() + * + * \return 0 on success, a negative error code otherwise + */ + /** * \fn GammaAlgorithm::prepare() * \tparam T The type of data expected by the hardware's look-up table diff --git a/src/ipa/libipa/gamma.h b/src/ipa/libipa/gamma.h index a8872a6c75..bf71423848 100644 --- a/src/ipa/libipa/gamma.h +++ b/src/ipa/libipa/gamma.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include @@ -46,15 +47,15 @@ public: { } - int init(ControlInfoMap::Map &controls, const ValueNode &tuningData, - std::span segments = {}); - void configure(gamma::ActiveState &state); void queueRequest(gamma::ActiveState &state, const uint32_t frame, gamma::FrameContext &context, const ControlList &controls); void process(gamma::FrameContext &context, ControlList &metadata); protected: + int init(ControlInfoMap::Map &controls, const ValueNode &tuningData, + std::span segments); + unsigned int nLutNodes_; float defaultGamma_; std::vector kneePoints_; @@ -71,6 +72,14 @@ public: { } + int init(ControlInfoMap::Map &controls, const ValueNode &tuningData, + std::optional> segments = {}) + { + return GammaAlgorithmBase::init( + controls, tuningData, + segments ? *segments : std::span{}); + } + template void prepare(gamma::FrameContext &context, std::span lut) { From patchwork Mon Nov 16 17:49:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28922 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36A29C3304 for ; Mon, 16 Nov 2026 17:49:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55C7B6A860; Mon, 16 Nov 2026 18:49:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dpUgNXli"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6247B6A848 for ; Mon, 16 Nov 2026 18:49:27 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6E26D175; Mon, 16 Nov 2026 18:47:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851220; bh=xmu86H1kwOfr/Rn6BvOhB4CP4pmrshuoZCO5bxWAjF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dpUgNXlifp98puH1cti8S/m0YNUi4YmbN5mjfPzUxXHAiq/LOk1KjlAPM1RIASuS4 qGIUxyZhVHGKuRHRfYL/2Nbfgeuz5xlKt+zpZAhvOqJX3Ky5iJ7PvW0X5FuCf8XH2t sqX8D5GCgmVXGPhQs93lcSKf7fpF5ALwck2i8/wM= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham Subject: [PATCH v5 09/22] ipa: libipa: ccm: Use 32-bit offsets Date: Mon, 16 Nov 2026 18:49:05 +0100 Message-ID: <20261116174919.1056403-10-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The currently used 16-bit offsets are not enough for all platforms. Offsets should ideally be defined as normalized numbers instead of effectively "raw" register values. But that requires breaking the tuning files, so it is postponsed, and this temporary solution is used. Link: https://gitlab.freedesktop.org/camera/libcamera/-/work_items/371 Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/ccm.cpp | 2 +- src/ipa/libipa/ccm.cpp | 2 +- src/ipa/libipa/ccm.h | 4 ++-- src/ipa/rkisp1/algorithms/ccm.cpp | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/ipu3/algorithms/ccm.cpp b/src/ipa/ipu3/algorithms/ccm.cpp index 105d815b03..95ba67f8d1 100644 --- a/src/ipa/ipu3/algorithms/ccm.cpp +++ b/src/ipa/ipu3/algorithms/ccm.cpp @@ -57,7 +57,7 @@ void Ccm::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t fram void Ccm::setParameters(ipu3_uapi_params *params, IPAFrameContext &context) { const Matrix &matrix = context.ccm.ccm; - const Matrix &offsets = context.ccm.offsets; + const Matrix &offsets = context.ccm.offsets; params->use.acc_ccm = 1; diff --git a/src/ipa/libipa/ccm.cpp b/src/ipa/libipa/ccm.cpp index 8f3652f7e3..a19ae9f778 100644 --- a/src/ipa/libipa/ccm.cpp +++ b/src/ipa/libipa/ccm.cpp @@ -83,7 +83,7 @@ int CcmAlgorithmBase::init(const ValueNode &tuningData) << "Failed to parse 'offsets' " << "parameter from tuning file; falling back to zero offsets"; - offsets_.setData({ { 0, Matrix({ 0, 0, 0 }) } }); + offsets_.setData({ { 0, Matrix({ 0, 0, 0 }) } }); } return 0; diff --git a/src/ipa/libipa/ccm.h b/src/ipa/libipa/ccm.h index b72b25e2a3..c17d1ef303 100644 --- a/src/ipa/libipa/ccm.h +++ b/src/ipa/libipa/ccm.h @@ -23,7 +23,7 @@ namespace ccm { struct CcmContext { Matrix ccm; - Matrix offsets; + Matrix offsets; }; struct ActiveState { @@ -50,7 +50,7 @@ public: private: unsigned int ct_; Interpolator> ccm_; - Interpolator> offsets_; + Interpolator> offsets_; }; template diff --git a/src/ipa/rkisp1/algorithms/ccm.cpp b/src/ipa/rkisp1/algorithms/ccm.cpp index 0e54c98710..f54edd44d1 100644 --- a/src/ipa/rkisp1/algorithms/ccm.cpp +++ b/src/ipa/rkisp1/algorithms/ccm.cpp @@ -67,7 +67,7 @@ void Ccm::queueRequest(IPAContext &context, void Ccm::setParameters(RkISP1Params *params, IPAFrameContext &context) { const Matrix &matrix = context.ccm.ccm; - const Matrix &offsets = context.ccm.offsets; + const Matrix &offsets = context.ccm.offsets; auto config = params->block(); config.setEnabled(true); From patchwork Mon Nov 16 17:49:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28923 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EDC61C3360 for ; Mon, 16 Nov 2026 17:49:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 236556A87C; Mon, 16 Nov 2026 18:49:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ElksbwuU"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D97D66A84A for ; Mon, 16 Nov 2026 18:49:27 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DFEFF63C for ; Mon, 16 Nov 2026 18:47:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851221; bh=mjaJdBA5zv+MQbmG/h7w4lc1+yWYwsHwfFDAgJe3O4w=; h=From:To:Subject:Date:In-Reply-To:References:From; b=ElksbwuU1wZvhHKU3FnLBrvcvLIlO9qrCIHj06MPA8zTJcXv9AKB2PukjBcIoFYTe loXc+UDczFjtfPluGr6a0cMDg486YBlwlxXLb8ExTEyiuKI6mgtQ9cNxjVKJr6x1mI i/n/QhZfx0D0embznPGVpdbKXwk1utHDlXgI6J4w= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [PATCH v5 10/22] ipa: libipa: lux: Convert to "algorithm" Date: Mon, 16 Nov 2026 18:49:06 +0100 Message-ID: <20261116174919.1056403-11-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Bring the "Lux" type in-line with other generic algorithm implementations like `CcmAlgorithm`, `GammaAlgorithm`, etc. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/libipa/lux.cpp | 136 +++++++++++++++++++++--------- src/ipa/libipa/lux.h | 36 ++++++-- src/ipa/rkisp1/algorithms/lux.cpp | 34 +++----- src/ipa/rkisp1/algorithms/lux.h | 4 +- src/ipa/rkisp1/ipa_context.h | 9 +- 5 files changed, 142 insertions(+), 77 deletions(-) diff --git a/src/ipa/libipa/lux.cpp b/src/ipa/libipa/lux.cpp index 12ba073a11..2d8c621e5c 100644 --- a/src/ipa/libipa/lux.cpp +++ b/src/ipa/libipa/lux.cpp @@ -3,22 +3,22 @@ * Copyright (C) 2019, Raspberry Pi Ltd * Copyright (C) 2024, Paul Elder * - * Helper class that implements lux estimation + * libIPA lux estimation */ #include "lux.h" -#include -#include - #include +#include +#include + #include "libcamera/internal/value_node.h" #include "histogram.h" /** * \file lux.h - * \brief Helper class that implements lux estimation + * \brief libipa implementation of lux estimation * * Estimating the lux level of an image is a common operation that can for * instance be used to adjust the target Y value in AGC or for Bayesian AWB @@ -33,51 +33,82 @@ LOG_DEFINE_CATEGORY(Lux) namespace ipa { +namespace lux { + /** - * \class Lux - * \brief Class that implements lux estimation + * \class ActiveState + * \brief The active state for the algorithm * - * IPAs that wish to use lux estimation should create a Lux algorithm module - * that lightly wraps this module by providing the platform-specific luminance - * histogram. The Lux entry in the tuning file must then precede the algorithms - * that depend on the estimated lux value. + * \var ActiveState::lux + * \brief The latest lux estimation */ /** - * \var Lux::referenceExposureTime_ + * \class FrameContext + * \brief The frame context for the algorithm + * + * \var FrameContext::lux + * \brief The estimated lux value for the given frame + */ + +} /* namespace lux */ + +/** + * \class LuxAlgorithm + * \brief Lux estimation algorithm + * + * This algorithm provides lux estimation based on tuning data. IPAs that wish to + * use lux estimation should create a Lux algorithm module that lightly wraps this + * module by providing the platform-specific luminance histogram. The Lux entry in + * the tuning file must then precede the algorithms that depend on the estimated lux + * value. + */ + +/** + * \class LuxAlgorithm::ProcessParams + * \brief Parameters for LuxAlgorithm::process() + * + * \var LuxAlgorithm::ProcessParams::exposureTime + * \brief Effective exposure time of the frame + * + * \var LuxAlgorithm::ProcessParams::analogueGain + * \brief Effective analogue gain of the frame + * + * \var LuxAlgorithm::ProcessParams::digitalGain + * \brief Effective digital gain of the frame + * + * \var LuxAlgorithm::ProcessParams::yHist + * \brief Luminance histogram of the frame + */ + +/** + * \var LuxAlgorithm::referenceExposureTime_ * \brief The exposure time of the reference image, in microseconds */ /** - * \var Lux::referenceAnalogueGain_ + * \var LuxAlgorithm::referenceAnalogueGain_ * \brief The analogue gain of the reference image */ /** - * \var Lux::referenceDigitalGain_ + * \var LuxAlgorithm::referenceDigitalGain_ * \brief The analogue gain of the reference image */ /** - * \var Lux::referenceY_ + * \var LuxAlgorithm::referenceY_ * \brief The measured luminance of the reference image, normalized to 1 * */ /** - * \var Lux::referenceLux_ + * \var LuxAlgorithm::referenceLux_ * \brief The estimated lux level of the reference image */ /** - * \brief Construct the Lux helper module - */ -Lux::Lux() -{ -} - -/** - * \brief Parse tuning data + * \brief Initialize algorithm and parse tuning data * \param[in] tuningData The ValueNode representing the tuning data * * This function parses tuning data for the common Lux module. It requires @@ -95,7 +126,7 @@ Lux::Lux() * * \return 0 on success or a negative error code */ -int Lux::parseTuningData(const ValueNode &tuningData) +int LuxAlgorithm::init(const ValueNode &tuningData) { auto value = tuningData["referenceExposureTime"].get(); if (!value) { @@ -141,25 +172,50 @@ int Lux::parseTuningData(const ValueNode &tuningData) } /** - * \brief Estimate lux given runtime values - * \param[in] exposureTime Exposure time applied to the frame - * \param[in] aGain Analogue gain applied to the frame - * \param[in] dGain Digital gain applied to the frame - * \param[in] yHist Histogram from the ISP statistics + * \param[in] state The active state + * \param[in] frameContext The frame context * - * Estimate the lux given the exposure time, gain, and histogram. + * This function finalizes \ref lux::FrameContext::lux "frameContext.lux" + * based on the latest estimation. + */ +void LuxAlgorithm::prepare(const lux::ActiveState &state, lux::FrameContext &frameContext) +{ + frameContext.lux = state.lux; +} + +/** + * \param[in] state The active state + * \param[in] frameContext The frame context + * \param[in] params The processing parameters + * \param[in] metadata The metadata * - * \return Estimated lux value + * This function updates the lux estimation based on the luminance histogram + * and other parameters of a completed frame, and additionally it also updates + * \a metadata for the frame. */ -double Lux::estimateLux(utils::Duration exposureTime, - double aGain, double dGain, - const Histogram &yHist) const +void LuxAlgorithm::process(lux::ActiveState &state, const lux::FrameContext &frameContext, + ControlList &metadata, std::optional &¶ms) +{ + /* + * Report the lux level used by algorithms to prepare this frame + * not the lux level *of* this frame. + */ + metadata.set(controls::Lux, frameContext.lux); + + /* If no statistics are available, keep the latest estimation. */ + if (!params) + return; + + state.lux = estimateLux(*params); +} + +double LuxAlgorithm::estimateLux(const ProcessParams ¶ms) const { - double currentY = yHist.interQuantileMean(0, 1); - double exposureTimeRatio = referenceExposureTime_ / exposureTime; - double aGainRatio = referenceAnalogueGain_ / aGain; - double dGainRatio = referenceDigitalGain_ / dGain; - double yRatio = (currentY / yHist.bins()) / referenceY_; + double currentY = params.yHist.interQuantileMean(0, 1); + double exposureTimeRatio = referenceExposureTime_ / params.exposureTime; + double aGainRatio = referenceAnalogueGain_ / params.analogueGain; + double dGainRatio = referenceDigitalGain_ / params.digitalGain; + double yRatio = (currentY / params.yHist.bins()) / referenceY_; double estimatedLux = exposureTimeRatio * aGainRatio * dGainRatio * yRatio * referenceLux_; diff --git a/src/ipa/libipa/lux.h b/src/ipa/libipa/lux.h index b6837ab729..5bce7855ad 100644 --- a/src/ipa/libipa/lux.h +++ b/src/ipa/libipa/lux.h @@ -8,27 +8,49 @@ #pragma once +#include + #include namespace libcamera { +class ControlList; class ValueNode; namespace ipa { class Histogram; -class Lux +namespace lux { + +struct ActiveState { + double lux; +}; + +struct FrameContext { + double lux; +}; + +} /* namespace lux */ + +class LuxAlgorithm { public: - Lux(); - - int parseTuningData(const ValueNode &tuningData); - double estimateLux(utils::Duration exposureTime, - double aGain, double dGain, - const Histogram &yHist) const; + struct ProcessParams { + utils::Duration exposureTime; + double analogueGain; + double digitalGain; + const Histogram &yHist; + }; + + int init(const ValueNode &tuningData); + void prepare(const lux::ActiveState &state, lux::FrameContext &frameContext); + void process(lux::ActiveState &state, const lux::FrameContext &frameContext, + ControlList &metadata, std::optional &¶ms); private: + double estimateLux(const ProcessParams ¶ms) const; + utils::Duration referenceExposureTime_; double referenceAnalogueGain_; double referenceDigitalGain_; diff --git a/src/ipa/rkisp1/algorithms/lux.cpp b/src/ipa/rkisp1/algorithms/lux.cpp index ce6928a55d..68ad41d6cf 100644 --- a/src/ipa/rkisp1/algorithms/lux.cpp +++ b/src/ipa/rkisp1/algorithms/lux.cpp @@ -31,19 +31,12 @@ namespace ipa::rkisp1::algorithms { * other algorithms (such as AGC) to use. */ -/** - * \brief Construct an rkisp1 Lux algo module - */ -Lux::Lux() -{ -} - /** * \copydoc libcamera::ipa::Algorithm::init */ int Lux::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) { - return lux_.parseTuningData(tuningData); + return lux_.init(tuningData); } /** @@ -53,7 +46,7 @@ void Lux::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, [[maybe_unused]] RkISP1Params *params) { - frameContext.lux.lux = context.activeState.lux.lux; + lux_.prepare(context.activeState.lux, frameContext.lux); } /** @@ -65,25 +58,24 @@ void Lux::process(IPAContext &context, const rkisp1_stat_buffer *stats, ControlList &metadata) { - /* - * Report the lux level used by algorithms to prepare this frame - * not the lux level *of* this frame. - */ - metadata.set(controls::Lux, frameContext.lux.lux); - - if (!stats) + if (!stats) { + lux_.process(context.activeState.lux, frameContext.lux, metadata, {}); return; - - utils::Duration exposureTime = context.configuration.agc.lineDuration * - frameContext.sensor.exposure; - double gain = frameContext.sensor.gain; + } /* \todo Deduplicate the histogram calculation from AGC */ const rkisp1_cif_isp_stat *params = &stats->params; Histogram yHist({ params->hist.hist_bins, context.hw.numHistogramBins }, [](uint32_t x) { return x >> 4; }); - context.activeState.lux.lux = lux_.estimateLux(exposureTime, gain, 1.0, yHist); + lux_.process(context.activeState.lux, frameContext.lux, metadata, + { { + .exposureTime = context.configuration.agc.lineDuration * + frameContext.sensor.exposure, + .analogueGain = frameContext.sensor.gain, + .digitalGain = 1, + .yHist = yHist, + } }); } REGISTER_IPA_ALGORITHM(Lux, "Lux") diff --git a/src/ipa/rkisp1/algorithms/lux.h b/src/ipa/rkisp1/algorithms/lux.h index 8cb35cbae2..f7d2581e57 100644 --- a/src/ipa/rkisp1/algorithms/lux.h +++ b/src/ipa/rkisp1/algorithms/lux.h @@ -20,8 +20,6 @@ namespace ipa::rkisp1::algorithms { class Lux : public Algorithm { public: - Lux(); - int init(IPAContext &context, const ValueNode &tuningData) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, @@ -32,7 +30,7 @@ public: ControlList &metadata) override; private: - ipa::Lux lux_; + ipa::LuxAlgorithm lux_; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index a6db004c16..b34ea53406 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -32,6 +32,7 @@ #include "libipa/fixedpoint.h" #include "libipa/gamma.h" #include "libipa/lsc.h" +#include "libipa/lux.h" namespace libcamera { @@ -105,9 +106,7 @@ struct IPAActiveState { ipa::gamma::ActiveState gamma; - struct { - double lux; - } lux; + ipa::lux::ActiveState lux; struct { controls::WdrModeEnum mode; @@ -162,9 +161,7 @@ struct IPAFrameContext : public FrameContext { ipa::ccm::FrameContext ccm; - struct { - double lux; - } lux; + ipa::lux::FrameContext lux; struct { controls::WdrModeEnum mode; From patchwork Mon Nov 16 17:49:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28924 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 076EFC3355 for ; Mon, 16 Nov 2026 17:49:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C5D3B6A86F; Mon, 16 Nov 2026 18:49:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rkzUpV1P"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50F8B6A845 for ; Mon, 16 Nov 2026 18:49:28 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2BD4B802; Mon, 16 Nov 2026 18:47:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851221; bh=DTDUG4R1lXJF+jVg+zdOe23703N72NnZ38i/OdM8rqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rkzUpV1PHzeGptCuSp5zgIM3b3Kq9QTadlBVCIDR1HWwI4vm1newSHvYeSyhIAUyb OR1oQ/6oIfJIxUIdmzcMMomBlDZpqduhRl85Ka8JTDfUmgQD1O/VLhhrK+Hooh8tV9 olb4BKOiWDS0sKCYuHiAM4PdEE2CnAWhRtfDtdlY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham Subject: [PATCH v5 11/22] include: linux: Update to Linux 7.4-rc3 Date: Mon, 16 Nov 2026 18:49:07 +0100 Message-ID: <20261116174919.1056403-12-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Update the kernel headers to v7.4-rc3 using utils/update-kernel-headers.sh. Preserve the libcamera local modifications manually. This in particular includes the definitions of RPP-X1. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- include/linux/README | 2 +- include/linux/drm.h | 7 + include/linux/drm_fourcc.h | 268 ++++++- include/linux/drm_mode.h | 13 + include/linux/media/arm/mali-c55-config.h | 2 + include/linux/media/dreamchip/rppx1-config.h | 730 +++++++++++++++++++ include/linux/rkisp1-config.h | 113 ++- include/linux/v4l2-controls.h | 23 + include/linux/videodev2.h | 4 + utils/update-kernel-headers.sh | 1 + 10 files changed, 1137 insertions(+), 26 deletions(-) create mode 100644 include/linux/media/dreamchip/rppx1-config.h diff --git a/include/linux/README b/include/linux/README index 9303e0f14a..b086e0099b 100644 --- a/include/linux/README +++ b/include/linux/README @@ -1,4 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 -Files in this directory are imported from v7.1 of the Linux kernel. Do not +Files in this directory are imported from v7.4-rc3 of the Linux kernel. Do not modify them manually. diff --git a/include/linux/drm.h b/include/linux/drm.h index 5eecc8bed5..b051d2da48 100644 --- a/include/linux/drm.h +++ b/include/linux/drm.h @@ -1299,6 +1299,13 @@ extern "C" { */ #define DRM_IOCTL_MODE_GETFB2 DRM_IOWR(0xCE, struct drm_mode_fb_cmd2) +/** + * DRM_IOCTL_SYNCOBJ_EVENTFD - Register an eventfd to be signalled by a syncobj. + * + * This can be used to integrate a syncobj in an event loop. + * + * The IOCTL argument is a struct drm_syncobj_eventfd. + */ #define DRM_IOCTL_SYNCOBJ_EVENTFD DRM_IOWR(0xCF, struct drm_syncobj_eventfd) /** diff --git a/include/linux/drm_fourcc.h b/include/linux/drm_fourcc.h index 8e2a6abae6..51033f08c2 100644 --- a/include/linux/drm_fourcc.h +++ b/include/linux/drm_fourcc.h @@ -224,9 +224,9 @@ extern "C" { * [31:0] sign:exponent:mantissa 1:8:23 */ #define DRM_FORMAT_R32F fourcc_code('R', ' ', ' ', 'F') /* [31:0] R 32 little endian */ -#define DRM_FORMAT_GR3232F fourcc_code('G', 'R', ' ', 'F') /* [63:0] R:G 32:32 little endian */ -#define DRM_FORMAT_BGR323232F fourcc_code('B', 'G', 'R', 'F') /* [95:0] R:G:B 32:32:32 little endian */ -#define DRM_FORMAT_ABGR32323232F fourcc_code('A', 'B', '8', 'F') /* [127:0] R:G:B:A 32:32:32:32 little endian */ +#define DRM_FORMAT_GR3232F fourcc_code('G', 'R', ' ', 'F') /* [63:0] G:R 32:32 little endian */ +#define DRM_FORMAT_BGR323232F fourcc_code('B', 'G', 'R', 'F') /* [95:0] B:G:R 32:32:32 little endian */ +#define DRM_FORMAT_ABGR32323232F fourcc_code('A', 'B', '8', 'F') /* [127:0] A:B:G:R 32:32:32:32 little endian */ /* * RGBA format with 10-bit components packed in 64-bit per pixel, with 6 bits @@ -246,6 +246,7 @@ extern "C" { #define DRM_FORMAT_XVUY8888 fourcc_code('X', 'V', 'U', 'Y') /* [31:0] X:Cr:Cb:Y 8:8:8:8 little endian */ #define DRM_FORMAT_VUY888 fourcc_code('V', 'U', '2', '4') /* [23:0] Cr:Cb:Y 8:8:8 little endian */ #define DRM_FORMAT_VUY101010 fourcc_code('V', 'U', '3', '0') /* Y followed by U then V, 10:10:10. Non-linear modifier only */ +#define DRM_FORMAT_XVUY2101010 fourcc_code('X', 'Y', '3', '0') /* [31:0] x:Cr:Cb:Y 2:10:10:10 little endian */ /* * packed Y2xx indicate for each component, xx valid data occupy msb @@ -361,6 +362,14 @@ extern "C" { */ #define DRM_FORMAT_P030 fourcc_code('P', '0', '3', '0') /* 2x2 subsampled Cr:Cb plane 10 bits per channel packed */ +/* + * 2 plane YCbCr422. + * 3 10 bit components and 2 padding bits packed into 4 bytes. + * index 0 = Y plane, [31:0] x:Y2:Y1:Y0 2:10:10:10 little endian + * index 1 = Cr:Cb plane, [63:0] x:Cr2:Cb2:Cr1:x:Cb1:Cr0:Cb0 [2:10:10:10:2:10:10:10] little endian + */ +#define DRM_FORMAT_P230 fourcc_code('P', '2', '3', '0') /* 2x1 subsampled Cr:Cb plane 10 bits per channel packed */ + /* 3 plane non-subsampled (444) YCbCr * 16 bits per component, but only 10 bits are used and 6 bits are padded * index 0: Y plane, [15:0] Y:x [10:6] little endian @@ -377,6 +386,15 @@ extern "C" { */ #define DRM_FORMAT_Q401 fourcc_code('Q', '4', '0', '1') +/* + * 3 plane non-subsampled (444) YCbCr LSB aligned + * 10 bpc, 30 bits per sample image data in a single contiguous buffer. + * index 0: Y plane, [31:0] x:Y2:Y1:Y0 [2:10:10:10] little endian + * index 1: Cb plane, [31:0] x:Cb2:Cb1:Cb0 [2:10:10:10] little endian + * index 2: Cr plane, [31:0] x:Cr2:Cr1:Cr0 [2:10:10:10] little endian + */ +#define DRM_FORMAT_T430 fourcc_code('T', '4', '3', '0') + /* * 3 plane YCbCr LSB aligned * In order to use these formats in a similar fashion to MSB aligned ones @@ -433,6 +451,17 @@ extern "C" { #define DRM_FORMAT_YUV444 fourcc_code('Y', 'U', '2', '4') /* non-subsampled Cb (1) and Cr (2) planes */ #define DRM_FORMAT_YVU444 fourcc_code('Y', 'V', '2', '4') /* non-subsampled Cr (1) and Cb (2) planes */ +/* + * Y-only (greyscale) formats + * + * The Y-only formats are handled similarly to the YCbCr formats in the display + * pipeline, with the Cb and Cr implicitly neutral (0.0 in nominal values). This + * also means that COLOR_RANGE property applies to the Y-only formats. + */ + +#define DRM_FORMAT_Y8 fourcc_code('G', 'R', 'E', 'Y') /* 8-bit Y-only */ +#define DRM_FORMAT_XYYY2101010 fourcc_code('Y', 'P', 'A', '4') /* [31:0] x:Y2:Y1:Y0 2:10:10:10 little endian */ + /* Compressed formats */ #define DRM_FORMAT_MJPEG fourcc_code('M', 'J', 'P', 'G') /* Motion-JPEG */ @@ -1674,6 +1703,30 @@ drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier) #define DRM_FORMAT_MOD_APPLE_GPU_TILED fourcc_mod_code(APPLE, 1) #define DRM_FORMAT_MOD_APPLE_GPU_TILED_COMPRESSED fourcc_mod_code(APPLE, 2) +/* + * Apple Interchange layout + * + * Apple Interchange is a compressed framebuffer format used exclusively for + * sharing buffers between hardware blocks (GPU, display controller, video + * codecs) on Apple platforms. It is similar to and uses the same compression + * scheme as GPU_TILED_COMPRESSED. There is no known uncompressed variant + * of Apple Interchange. + * + * Unlike the GPU layouts, tile size is not dependent on bit depth. RGB + * images and the chroma components of YUV images use 16x16 tiles. The + * luma component of YUV images uses 32x32 tiles. Compression tiles + * are once again 16x16 with 8 bytes of metadata per compression subtile. + * + * Following the example set by GPU_TILED_{,COMPRESSED}, Interchange images + * are treated as a single plane by software and do not have a meaningful + * stride. + * + * AGX has limited support for Apple Interchange. It cannot be used for Z/S + * buffers or array images, for example. It is intended for scanout buffer + * sharing only. + */ +#define DRM_FORMAT_MOD_APPLE_INTERCHANGE fourcc_mod_code(APPLE, 3) + /* * AMD modifiers * @@ -1694,36 +1747,69 @@ drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier) * For multi-plane formats the above surfaces get merged into one plane for * each format plane, based on the required alignment only. * - * Bits Parameter Notes - * ----- ------------------------ --------------------------------------------- + * Bits Parameter Notes + * ------- ------------------------ --------------------------------------------- + * + * DRM format modifier fields on AMD GPUs: + * 7:0 TILE_VERSION Values are AMD_FMT_MOD_TILE_VER_* + * 12:8 TILE Values are AMD_FMT_MOD_TILE__* + * 13 DCC Delta Color Compression, supported on GFX8 and newer + * 55:14 (chip specific) See below for details, depends on GFX block version + * 63:56 Vendor Value is DRM_FORMAT_MOD_VENDOR_AMD + * + * Chip specific fields on Gfx9 and newer: + * 14 DCC_RETILE + * 15 DCC_PIPE_ALIGN + * 16 DCC_INDEPENDENT_64B + * 17 DCC_INDEPENDENT_128B + * 19:18 DCC_MAX_COMPRESSED_BLOCK Values are AMD_FMT_MOD_DCC_BLOCK_* + * 20 DCC_CONSTANT_ENCODE + * 23:21 PIPE_XOR_BITS Only for some chips + * 26:24 BANK_XOR_BITS Only for some chips + * 29:27 PACKERS Only for some chips + * 32:30 RB Only for some chips + * 35:33 PIPE Only for some chips + * 55:36 - Reserved for future use, must be zero + * + * Chip specific fields on Gfx6-8: + * 16:14 MICROTILE Micro tile format + * 21:17 PIPE_CONFIG Number of pipes and how pipes are interleaved + * 24:22 TILE_SPLIT Tile split size + * 26:25 BANK_WIDTH Number of tiles in the X direction in the same bank + * 28:27 BANK_HEIGHT Number of tiles in the Y direction in the same bank + * 30:29 MACRO_TILE_ASPECT Macro tile aspect ratio + * 32:31 NUM_BANKS Number of banks + * 55:33 - Reserved for future use, must be zero * - * 7:0 TILE_VERSION Values are AMD_FMT_MOD_TILE_VER_* - * 12:8 TILE Values are AMD_FMT_MOD_TILE__* - * 13 DCC - * 14 DCC_RETILE - * 15 DCC_PIPE_ALIGN - * 16 DCC_INDEPENDENT_64B - * 17 DCC_INDEPENDENT_128B - * 19:18 DCC_MAX_COMPRESSED_BLOCK Values are AMD_FMT_MOD_DCC_BLOCK_* - * 20 DCC_CONSTANT_ENCODE - * 23:21 PIPE_XOR_BITS Only for some chips - * 26:24 BANK_XOR_BITS Only for some chips - * 29:27 PACKERS Only for some chips - * 32:30 RB Only for some chips - * 35:33 PIPE Only for some chips - * 55:36 - Reserved for future use, must be zero */ #define AMD_FMT_MOD fourcc_mod_code(AMD, 0) #define IS_AMD_FMT_MOD(val) (((val) >> 56) == DRM_FORMAT_MOD_VENDOR_AMD) -/* Reserve 0 for GFX8 and older */ +#define AMD_FMT_MOD_TILE_VER_GFX6 0 #define AMD_FMT_MOD_TILE_VER_GFX9 1 #define AMD_FMT_MOD_TILE_VER_GFX10 2 #define AMD_FMT_MOD_TILE_VER_GFX10_RBPLUS 3 #define AMD_FMT_MOD_TILE_VER_GFX11 4 #define AMD_FMT_MOD_TILE_VER_GFX12 5 +/* + * Gfx6-8 tiling modes. + * A complete reference implementation is found in addrlib in the Mesa code base. + * + * - Microtiled modes (1D): + * Pixel data is organized into micro tiles of 8x8 pixels. + * + * - Macrotiled modes (2D): + * Micro tiles are further organized into macro tiles. + * These are optimized for even load distribution among memory channels. + * + * Note that only THIN1 modes are exposed here. + * THICK and XTHICK are for 3D images and not relevant to DRM format modifiers. + */ +#define AMD_FMT_MOD_TILE_GFX6_1D_TILED_THIN1 0x2 +#define AMD_FMT_MOD_TILE_GFX6_2D_TILED_THIN1 0x4 + /* * 64K_S is the same for GFX9/GFX10/GFX10_RBPLUS and hence has GFX9 as canonical * version. @@ -1822,6 +1908,146 @@ drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier) #define AMD_FMT_MOD_PIPE_SHIFT 33 #define AMD_FMT_MOD_PIPE_MASK 0x7 +/* + * MICRO_TILE_MODE, 3 bits. Determines the micro tile format. + * Only relevant to Gfx6-8. + * + * DISPLAY - Displayable tiling + * THIN - Non-displayable tiling, a.k.a thin micro tiling + * DEPTH, THICK - not exposed, not relevant to DRM format modifier use cases + * ROTATED - not exposed, not implemented in Linux or Mesa + */ +#define AMD_FMT_MOD_MICROTILE_SHIFT 14ULL +#define AMD_FMT_MOD_MICROTILE_MASK 0x7 + +#define AMD_FMT_MOD_MICROTILE_DISPLAY 0x0 +#define AMD_FMT_MOD_MICROTILE_THIN 0x1 + +/* + * PIPE_CONFIG, 5 bits. Number of pipes and how pipes are interleaved on the surface, + * which means the shader engine tile size and packer tile size. + * Typically matches the number of memory channels, or number of RBs. + * Only relevant to Gfx6-8 macro tiled modes. + * + * P_x_x + * where: + * - number of pipes + * x - shader engine tile size + * x - packer tile size + */ +#define AMD_FMT_MOD_PIPE_CONFIG_SHIFT 17ULL +#define AMD_FMT_MOD_PIPE_CONFIG_MASK 0x1f + +#define AMD_FMT_MOD_PIPE_CONFIG_P2 0x0 +#define AMD_FMT_MOD_PIPE_CONFIG_P4_8x16 0x4 +#define AMD_FMT_MOD_PIPE_CONFIG_P4_16x16 0x5 +#define AMD_FMT_MOD_PIPE_CONFIG_P4_16x32 0x6 +#define AMD_FMT_MOD_PIPE_CONFIG_P4_32x32 0x7 +#define AMD_FMT_MOD_PIPE_CONFIG_P8_16x16_8x16 0x8 +#define AMD_FMT_MOD_PIPE_CONFIG_P8_16x32_8x16 0x9 +#define AMD_FMT_MOD_PIPE_CONFIG_P8_32x32_8x16 0xa +#define AMD_FMT_MOD_PIPE_CONFIG_P8_16x32_16x16 0xb +#define AMD_FMT_MOD_PIPE_CONFIG_P8_32x32_16x16 0xc +#define AMD_FMT_MOD_PIPE_CONFIG_P8_32x32_16x32 0xd +#define AMD_FMT_MOD_PIPE_CONFIG_P8_32x64_32x32 0xe +#define AMD_FMT_MOD_PIPE_CONFIG_P16_32x32_8x16 0x10 +#define AMD_FMT_MOD_PIPE_CONFIG_P16_32x32_16x16 0x11 + +/* + * TILE_SPLIT, 3 bits. + * Only relevant to Gfx6-8 macro tiled modes. + * + * On GFX6 (or with depth tiling modes on GFX7 and newer), + * the GFX block uses the GB_TILE_MODE.TILE_SPLIT field directly. + * + * On GFX7 and newer with non-depth tiling modes, the GFX block uses a + * split factor which is stored in the GB_TILE_MODE.SAMPLE_SPLIT field. + * SAMPLE_SPLIT may be: 0 - 1 byte; 1 - 2 bytes; 2 - 4 bytes; 3 - 8 bytes. + * The actual tile size and tile split bytes are calculated as follows: + * + * bpp = ... <- bits per pixel in the current image + * thickness = ... <- depends on array mode; may be: 1, 4, 8 + * num_samples = ... <- number of samples in the current image + * tile_size_pixels = 8 * 8 + * tile_bytes_1x = thickness * tile_size_pixels * bpp / 8 + * sample_split_factor = 1 << SAMPLE_SPLIT + * tile_split_bytes = clamp(tile_bytes_1x * sample_split_factor, 256, dram_row_size_bytes) + * tile_bytes = clamp(tile_bytes_1x * num_samples, 64, tile_split_bytes) + * + * In both cases, the display block (DCE) has no SAMPLE_SPLIT + * and just needs the tile split bytes in the GRPH_CONTROL.GRPH_TILE_SPLIT field. + * To maximize compatibility between GFX6-7, we don't include the SAMPLE_SPLIT + * in the format modifiers. + * + * The actual tile split in bytes is: 64 << field value + * Possible values of this field: + * + * 0 - Tile split is 64 bytes + * 1 - Tile split is 128 bytes + * 2 - Tile split is 256 bytes + * 3 - Tile split is 512 bytes + * 4 - Tile split is 1 KiB + * 5 - Tile split is 2 KiB + * 6 - Tile split is 4 KiB + */ +#define AMD_FMT_MOD_TILE_SPLIT_SHIFT 22ULL +#define AMD_FMT_MOD_TILE_SPLIT_MASK 0x7 + +/* + * BANK_WIDTH, 2 bits. Number of tiles in the X direction in the same bank. + * Only relevant to Gfx6-8 macro tiled modes. + * The actual bank width is: 1 << field value + * Possible values: + * + * 0 - bank width is 1 + * 1 - bank width is 2 + * 2 - bank width is 4 + * 3 - bank width is 8 + */ +#define AMD_FMT_MOD_BANK_WIDTH_SHIFT 25ULL +#define AMD_FMT_MOD_BANK_WIDTH_MASK 0x3 + +/* + * BANK_HEIGHT, 2 bits. Number of tiles in the Y direction in the same bank. + * Only relevant to Gfx6-8 macro tiled modes. + * The actual bank height is: 1 << field value + * Possible values: + * + * 0 - bank height is 1 + * 1 - bank height is 2 + * 2 - bank height is 4 + * 3 - bank height is 8 + */ +#define AMD_FMT_MOD_BANK_HEIGHT_SHIFT 27ULL +#define AMD_FMT_MOD_BANK_HEIGHT_MASK 0x3 + +/* + * MACRO_TILE_ASPECT, 2 bits. Macro tile aspect ratio. + * Only relevant to Gfx6-8 macro tiled modes. + * Possible values: + * + * 0 - aspect ratio is 1:1 + * 1 - aspect ratio is 4:1 + * 2 - aspect ratio is 16:1 + * 3 - aspect ratio is 64:1 + */ +#define AMD_FMT_MOD_MACRO_TILE_ASPECT_SHIFT 29ULL +#define AMD_FMT_MOD_MACRO_TILE_ASPECT_MASK 0x3 + +/* + * NUM_BANKS, 2 bits. Number of banks. + * Only relevant to Gfx6-8 macro tiled modes. + * The actual number of banks is: 2 << field value + * Possible values: + * + * 0 - number of banks is 2 + * 1 - number of banks is 4 + * 2 - number of banks is 8 + * 3 - number of banks is 16 + */ +#define AMD_FMT_MOD_NUM_BANKS_SHIFT 31ULL +#define AMD_FMT_MOD_NUM_BANKS_MASK 0x3 + #define AMD_FMT_MOD_SET(field, value) \ ((__u64)(value) << AMD_FMT_MOD_##field##_SHIFT) #define AMD_FMT_MOD_GET(field, value) \ diff --git a/include/linux/drm_mode.h b/include/linux/drm_mode.h index 381a3e857d..ed1a660a3d 100644 --- a/include/linux/drm_mode.h +++ b/include/linux/drm_mode.h @@ -155,6 +155,7 @@ extern "C" { /* Panel type property */ #define DRM_MODE_PANEL_TYPE_UNKNOWN 0 #define DRM_MODE_PANEL_TYPE_OLED 1 +#define DRM_MODE_PANEL_TYPE_LCD 2 /* * DRM_MODE_ROTATE_ @@ -954,6 +955,18 @@ enum drm_colorop_type { * color = lut3d[index] */ DRM_COLOROP_3D_LUT, + + /** + * @DRM_COLOROP_FIXED_MATRIX: + * + * enum string "Fixed Matrix" + * + * A Colorop block that performs a pre-defined matrix operation selected + * via the FIXED_MATRIX_TYPE enum property. The driver advertises the supported + * operations through this property. + */ + DRM_COLOROP_FIXED_MATRIX, + }; /** diff --git a/include/linux/media/arm/mali-c55-config.h b/include/linux/media/arm/mali-c55-config.h index ef0fe17df9..3ba416d81c 100644 --- a/include/linux/media/arm/mali-c55-config.h +++ b/include/linux/media/arm/mali-c55-config.h @@ -794,12 +794,14 @@ struct mali_c55_params_mesh_shading_selection { * Q4.8 format * @gains: Gains for red, green and blue channels in unsigned Q4.8 format * @offs: Offsets for red, green and blue channels + * @__pad: Reserved bytes */ struct mali_c55_params_ccm { struct v4l2_isp_params_block_header header; __u16 coeffs[3][3]; __u16 gains[3]; __u16 offs[3]; + __u16 __pad; }; /** diff --git a/include/linux/media/dreamchip/rppx1-config.h b/include/linux/media/dreamchip/rppx1-config.h new file mode 100644 index 0000000000..3507a9ab03 --- /dev/null +++ b/include/linux/media/dreamchip/rppx1-config.h @@ -0,0 +1,730 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Dreamchip RPP-X1 ISP Driver - Userspace API + * + * Copyright (C) 2026 Renesas Electronics Corp. + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2026 Ragnatech AB + */ + +#ifndef __UAPI_RPP_X1_CONFIG_H +#define __UAPI_RPP_X1_CONFIG_H + +#include + +/** + * struct rppx1_window - Measurement window + * + * RPP-X1 measurement window. Different blocks use a window or multiple + * windows for measurement purposes. This defines a common type for all of + * them. The number of relevant bits depends on the block where the window is + * used and is specified in the per-block description + * + * @h_offs: horizontal offset from the left of the frame in pixels + * @v_offs: vertical offset from the top of the frame in pixels + * @h_size: horizontal size of the window in pixels + * @v_size: vertical size of the window in pixels + */ +struct rppx1_window { + __u16 h_offs; + __u16 v_offs; + __u16 h_size; + __u16 v_size; +}; + +/** + * enum rppx1_meas_chan - Measurement point for the Histogram and EXM Modules + * + * Measurement points for the RPP-X1 Histogram measurement module and Exposure + * measurement module. + * + * All tap points are available for the PRE1/PRE2 pipes. Only + * RPPX1_MEAS_CHAN_SEL4 and RPPX1_MEAS_CHAN_SEL7 are available for the + * MAIN_POST pipe. + * + * @RPPX1_MEAS_CHAN_SEL0: after input acquisition + * @RPPX1_MEAS_CHAN_SEL1: after black level subtraction + * @RPPX1_MEAS_CHAN_SEL2: after sensor gamma linearization + * @RPPX1_MEAS_CHAN_SEL3: after lens shading correction + * @RPPX1_MEAS_CHAN_SEL4: after auto white balance gains + * @RPPX1_MEAS_CHAN_SEL5: after defect pixel correction + * @RPPX1_MEAS_CHAN_SEL6: after denoise pre-filter + * @RPPX1_MEAS_CHAN_SEL7: after demosaicing + */ +enum rppx1_meas_chan { + RPPX1_MEAS_CHAN_SEL0, + RPPX1_MEAS_CHAN_SEL1, + RPPX1_MEAS_CHAN_SEL2, + RPPX1_MEAS_CHAN_SEL3, + RPPX1_MEAS_CHAN_SEL4, + RPPX1_MEAS_CHAN_SEL5, + RPPX1_MEAS_CHAN_SEL6, + RPPX1_MEAS_CHAN_SEL7, +}; + +/* --------------------------------------------------------------------------- + * Parameter Structures + * + * The same ISP block might be instantiated in multiple pipeliness and operate + * on a different bitdepth/precision. For fields of varying length among + * different instances of the same block, use a data type that can accommodate + * the larger bitdepth/precision. + */ + +/** + * enum rppx1_params_block_type - RPP-X1 extensible params block types + * + * NOTE: Only append to the enumeration as the numbers are uAPI. + * + * @RPPX1_PARAMS_BLOCK_TYPE_WBMEAS_POST: AWB Measurement Configuration + * @RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1: PRE1 pipe White Balance Gains + * @RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2: PRE2 White Balance Gains + * @RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST: MAIN_POST White Balance Gains + * @RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1: PRE1 pipe Exposure Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2: PRE2 pipe Exposure Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1: PRE1 pipe Histogram Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2: PRE2 pipe Histogram Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_HIST_POST: POST pipe Histogram Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1: PRE1 pipe Black Level Subtraction + * @RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2: PRE2 pipe Black Level Subtraction + * @RPPX1_PARAMS_BLOCK_TYPE_CCOR_POST: POST pipe Color Correction + * @RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE1: PRE1 pipe Lens Shading Correction + * @RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE2: PRE2 pipe Lens Shading Correction + * @RPPX1_PARAMS_BLOCK_TYPE_GA_HV: Human Vision Pipe Gamma Out Correction + * @RPPX1_PARAMS_BLOCK_TYPE_GA_MV: Machine Vision Gamma Out Correction + * @RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1: PRE1 pipe Linearization (Sensor De-gamma) + * @RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2: PRE2 pipe Linearization (Sensor De-gamma) + */ +enum rppx1_params_block_type { + RPPX1_PARAMS_BLOCK_TYPE_WBMEAS_POST, + RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST, + RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_HIST_POST, + RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_CCOR_POST, + RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_GA_HV, + RPPX1_PARAMS_BLOCK_TYPE_GA_MV, + RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2, +}; + +/** + * enum rppx1_wbmeas_mode - AWB measurement mode + * + * @RPPX1_WBMEAS_MODE_YCBCR: YCbCr measurement mode + * @RPPX1_WBMEAS_MODE_RGB: RGB measurement mode + */ +enum rppx1_wbmeas_mode { + RPPX1_WBMEAS_MODE_YCBCR, + RPPX1_WBMEAS_MODE_RGB, +}; + +/** + * struct rppx1_wbmeas_params - AWB measurement configuration + * + * The Auto-White Balance measurement module is available on the MAIN_POST pipe. + * It supports two measurement modes, selected by the @mode field. The + * measurement window is programmed through the @wnd field. + * + * To support measurement in YCbCr mode a color conversion matrix with + * programmable offset is available in the @ccor_coeff and @ccor_offs fields. + * The color conversion matrix coefficients are represented as 16 bits signed + * Q4.12 numbers ranging from -8 to +7.99. The per-color channel offsets are + * represented as 25 bits 2's complement integer numbers ranging from -16777216 + * to +16777215. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_WBMEAS_POST) + * @wnd: measurement window + * @mode: measurement mode (from enum rppx1_wbmeas_mode) + * @ymax_cmp: enable Y_MAX compare using @max_y + * @frames: number of frames for mean value calculation (0 = 1 frame) + * @reserved: padding + * @ref_cr_max_r: reference Cr or max red value in RGB mode, 24 bits + * @ref_cb_max_b: reference Cb or max blue value in RGB mode, 24 bits + * @min_y_max_g: luminance minimum value or max green value in RGB mode, 24 bits + * @max_y: luminance maximum value, only valid if @mode is set to YCbCr and + * @ymax_cmp is set to enabled, 24 bits + * @max_csum: chrominance sum maximum value, 24 bits + * @min_c: chrominance minimum value, 24 bits + * @ccor_coeff: coefficients for color conversion matrix, signed 16 bits Q4.6 + * @reserved2: padding + * @ccor_offs: R-G-B color conversion coefficients, signed 25 bits 2's complement + * @reserved3: padding + */ +struct rppx1_wbmeas_params { + struct v4l2_isp_params_block_header header; + struct rppx1_window wnd; + __u8 mode; + __u8 ymax_cmp; + __u8 frames; + __u8 reserved; + __u32 ref_cr_max_r; + __u32 ref_cb_max_b; + __u32 min_y_max_g; + __u32 max_y; + __u32 max_csum; + __u32 min_c; + __u16 ccor_coeff[3][3]; + __u16 reserved2; + __u32 ccor_offs[3]; + __u32 reserved3; +}; + +/** + * struct rppx1_awbg_params - WB gain configuration + * + * The RPP-X1 White Balance Gain module is available in the PRE1 and PRE2 + * pre-fusion pipes and in the MAIN_POST post-fusion pipe. Userspace selects + * which pipe to operate by setting the @header.type field to + * RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1, RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2 + * or RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST. + * + * The White Balance module allows to specify per-color channel gains, expressed + * as unsigned fixed-point values as 18 bits unsigned integers in Q6.12 format + * with a maximum of 63.999. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1 or + * type = RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2 or + * type = RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST) + * @gain_red: gain for red component, 18-bit (unsigned Q6.12) + * @gain_green_r: gain for green component in red lines, 18-bit (unsigned Q6.12) + * @gain_blue: gain for blue component, 18-bit (unsigned Q6.12) + * @gain_green_b: gain for green component in blue lines, 18-bit (unsigned Q6.12) + */ +struct rppx1_awbg_params { + struct v4l2_isp_params_block_header header; + __u32 gain_red; + __u32 gain_green_r; + __u32 gain_blue; + __u32 gain_green_b; +}; + +/** + * enum rppx1_exm_mode - Exposure measurement mode + * + * Exaposure measurement mode selection (RGB/Bayer). + * + * @RPPX1_EXP_MEASURING_MODE_DISABLED: no measurement + * @RPPX1_EXP_MEASURING_MODE_RGB: Y/R/G/B measurement + * @RPPX1_EXP_MEASURING_MODE_BAYER: Bayer RGB measurement + */ +enum rppx1_exm_mode { + RPPX1_EXP_MEASURING_MODE_DISABLED, + RPPX1_EXP_MEASURING_MODE_RGB, + RPPX1_EXP_MEASURING_MODE_BAYER, +}; + +/** + * struct rppx1_exm_params - Exposure measurement configuration + * + * The RPP-X1 Exposure measurement unit is available on the PRE1 and PRE2 + * pre-fusion pipes. Userspace selects which pipe to operate by setting + * the @header.type field to RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2. + * + * Exposure measurement is performed in the RGB or Bayer domain, according to + * the setting of the @mode field. The exposure measurement tap point is + * selected according to the value of @channel_sel. + * + * The exposure measurement is performed on an input window specified in @wnd. + * To each color component a programmable weight coefficient is associated. + * Coefficients are represented as unsigned 8 bits integer values in Q1.7 format + * ranging from 0 to 1.992. + * + * The @last_line fields controls when the exposure measurement completes. It + * is usually programmed to the value of (@wnd.v_offs + @wnd.v_size + 1). + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1 or + * type = RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2) + * @wnd: measurement window coordinates + * @mode: exposure measure mode (from enum rppx1_exm_mode) + * @last_line: line number for which the exposure measurement completes + * @channel_sel: exposure measurement point (see enum rppx1_meas_chan) + * @coeff_r: coefficient for the red Bayer sample or red color channel, Q1.7 + * @coeff_g_gr: coefficient for the green/red Bayer sample or green color channel, Q1.7 + * @coeff_b: coefficient for the blue Bayer sample or blue color channel, Q1.7 + * @coeff_gb: coefficient for the green/blue Bayer sample, unused in RGB mode, Q1.7 + * @reserved: padding + */ +struct rppx1_exm_params { + struct v4l2_isp_params_block_header header; + struct rppx1_window wnd; + __u32 mode; + __u32 last_line; + __u8 channel_sel; + __u8 coeff_r; + __u8 coeff_g_gr; + __u8 coeff_b; + __u8 coeff_gb; + __u8 reserved[3]; +}; + +/* Histogram */ +#define RPPX1_HIST_WEIGHT_GRIDS_SIZE 25 + +/** + * enum rppx1_hist_mode - Histogram measurement mode + * + * Histogram measurement mode. Select which channel or combination of channels + * the histogram measurement is performed on. + * + * @RPPX1_HIST_MODE_DISABLE: histogram disabled + * @RPPX1_HIST_MODE_RGB_COMBINED: combined RGB histogram + * @RPPX1_HIST_MODE_R_HISTOGRAM: red channel histogram + * @RPPX1_HIST_MODE_GR_HISTOGRAM: green/red channel histogram + * @RPPX1_HIST_MODE_B_HISTOGRAM: blue channel histogram + * @RPPX1_HIST_MODE_GB_HISTOGRAM: green/blue histogram + */ +enum rppx1_hist_mode { + RPPX1_HIST_MODE_DISABLE, + RPPX1_HIST_MODE_RGB_COMBINED, + RPPX1_HIST_MODE_R_HISTOGRAM, + RPPX1_HIST_MODE_GR_HISTOGRAM, + RPPX1_HIST_MODE_B_HISTOGRAM, + RPPX1_HIST_MODE_GB_HISTOGRAM, +}; + +/** + * struct rppx1_hist_params - Histogram measurement configuration + * + * The RPP-X1 Histogram measurement unit is available on the PRE1, PRE2 and + * MAIN_POST pipes. Userspace selects which pipe to operate by setting the + * @header.type field to RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1, + * RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2 or + * RPPX1_PARAMS_BLOCK_TYPE_HIST_POST. + * + * The histogram measurement point is selected using the @channel field while + * histogram measurement mode is selected using the @mode field. + * + * Histogram measurement is performed by programming subsampling factors using + * the @v_stepsize and @h_step_inc fields and by weighted windowing, by + * programming the size of the measurement window @wnd with @weights associated + * to each cell of the 5x5 measurement grid. Weights are represented as 5 bits + * integer values ranging from 0 to 16. + * + * The @last_line fields controls when the histogram measurement completes. It + * is usually programmed to the value of (@wnd.v_offs + @wnd.v_size - 1). + * + * Histogram values are calculated by applying a per-color channel coefficient + * represented as an 8 bits unsigned Q1.7 integer value. The @sample_offs and + * @sample_shift fields allow to reduce the color dynamic range on which + * histogram data are produced. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1, + * type = RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2 or + * type = RPPX1_PARAMS_BLOCK_TYPE_HIST_POST) + * @wnd: measurement window coordinates + * @last_line: line number for which the histogram measurement completes + * @v_stepsize: vertical subsampling divider, 7 bits + * @h_step_inc: horizontal subsampling step counter, 17 bits + * @sample_offs: sample offset, 24 bits + * @mode: histogram measurement mode (from enum rppx1_hist_mode) + * @channel_sel: histogram measurement point (see enum rppx1_meas_chan) + * @weights: weighting factors for each sub-window (5x5 grid) + * @coeff: R-G-B coefficients, 8 bits unsigned Q1.7 + * @sample_shift: sample shift, 4 bits + * @reserved: padding + */ +struct rppx1_hist_params { + struct v4l2_isp_params_block_header header; + struct rppx1_window wnd; + __u32 last_line; + __u32 v_stepsize; + __u32 h_step_inc; + __u32 sample_offs; + __u8 mode; + __u8 channel_sel; + __u8 weights[RPPX1_HIST_WEIGHT_GRIDS_SIZE]; + __u8 coeff[3]; + __u8 sample_shift; + __u8 reserved; +}; + +/** + * struct rppx1_bls_fixed - BLS fixed subtraction values + * + * Fixed black level values subtracted from sensor data per Bayer channel. + * Negative values result in addition. + * + * The PRE1 pipe BLS module operates on a 24-bits input data and fixed black + * levels are stored as a signed 2's complement representation ranging from + * -2^24 to 2^24-1. + * + * The PRE2 pipe BLS module operates on a 12-bits input data and fixed black + * levels are stored as a signed 2's complement representation ranging from + * -2^12 to 2^12-1. + * + * Userspace is expected to provide fixed black level values with a bit-depth + * matching the one of pipe in use. + * + * These subtraction values are matched with the sensor native Bayer components + * ordering according to the cropping configuration on the input port. + * + * @a: subtraction value for channel A + * @b: subtraction value for channel B + * @c: subtraction value for channel C + * @d: subtraction value for channel D + */ +struct rppx1_bls_fixed { + __u32 a; + __u32 b; + __u32 c; + __u32 d; +}; + +/** + * enum rppx1_bls_mode - BLS subtraction mode + * + * Select if subtracted black level come from fixed or measured values. + * + * @RPPX1_BLS_MODE_FIXED: subtract fixed values + * @RPPX1_BLS_MODE_MEAS: subtract measured values + */ +enum rppx1_bls_mode { + RPPX1_BLS_MODE_FIXED, + RPPX1_BLS_MODE_MEAS, +}; + +/** + * enum rppx1_bls_win_en: BLS measurement configuration + * + * Select the measurement window to use for measured black level values. + * + * @RPPX1_BLS_WIN_EN_OFF: disable measurement + * @RPPX1_BLS_WIN_EN_WIN1: Enable measurement from window 1 + * @RPPX1_BLS_WIN_EN_WIN2: enable measurement from window 2 + * @RPPX1_BLS_WIN_EN_WIN12: enable measurement from window 1 and window 2 + */ +enum rppx1_bls_win_en { + RPPX1_BLS_WIN_EN_OFF, + RPPX1_BLS_WIN_EN_WIN1, + RPPX1_BLS_WIN_EN_WIN2, + RPPX1_BLS_WIN_EN_WIN12, +}; + +/** + * struct rppx1_bls_params - RPP-X1 Black Level Subtraction Module + * + * The RPP-X1 Black Level Subtraction module is available on the PRE1 and PRE2 + * pre-fusion pipes. Userspace selects which pipe to operate by setting the + * @header.type field to RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2. + * + * The BLS module operates on fixed or measured data according to the setting of + * the @mode field. When RPPX1_BLS_MODE_FIXED is used userspace shall provide + * the per-channel black levels in @fixed. When RPPX1_BLS_MODE_MEAS is used + * userspace shall configure the measurement windows @window1 and optionally + * @window2 to select the optically black pixels region in the input frame. The + * @samples fields controls how many measure samples are used for averaging the + * measured black levels. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1 or + * type == RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2) + * @window1: BLS measurement window 1 (14 bits) + * @window2: BLS measurement window 2 (14 bits) + * @fixed: fixed subtraction values (see enum rppx1_bls_fixed) + * @mode: BLS subtraction mode (see enum rppx1_bls_mode) + * @en_windows: BLS measurement mode (see rppx1_bls_win_en) + * @samples: log2 of the number of measured pixels per Bayer position + * @reserved: padding + */ +struct rppx1_bls_params { + struct v4l2_isp_params_block_header header; + struct rppx1_window window1; + struct rppx1_window window2; + struct rppx1_bls_fixed fixed; + __u8 mode; + __u8 en_windows; + __u8 samples; + __u8 reserved[5]; +}; + +/** + * struct rppx1_ccor_params - Color CORrection configuration + * + * The CCOR (Color Correction) module is available on the MAIN_POST pipe. It + * performs color space correction on a pixel-per-pixel basis using a 3x3 matrix + * of coefficients and per-color channel offsets. + * + * The matrix coefficients are represented as 16 bits signed fixed point values + * in Q4.12 format ranging from -8 to +7.999. + * + * The per-channel color offsets are represented as 2's complement values + * stored in 25 bits ranging from -16777216 to 16777215. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_CCOR_POST) + * @coeff: color correction matrix coefficients, 16 bits signed Q4.12 + * @reserved: padding + * @offset: R, G, B offsets, 2's complement 25 bits + */ +struct rppx1_ccor_params { + struct v4l2_isp_params_block_header header; + __u16 coeff[3][3]; + __u16 reserved; + __u32 offset[3]; +}; + +/* Lens Shade Correction */ +#define RPPX1_LSC_SAMPLES_MAX 17 +#define RPPX1_LSC_NUM_SECTORS 16 + +/** + * struct rppx1_lsc_params - Lens Shading Correction configuration + * + * The RPP-X1 Lens shading correction module is available on the PRE1 and PRE2 + * pre-fusion pipes. Userspace selects which pipe to operate by setting the + * @header.type field to RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE2. + * + * The module applies per-color channel correction factors @r_data, @gr_data, + * @gb_data and @b_data as a 16x16 grid mapped on the image. The size of each + * grid segment is expressed by the @x_sect_size and @y_sect_size arrays. Each + * segment shall be at least 8 pixels in size and the sum of all horizontal + * segments @x_sect_size shall match the input frame size width. + * + * The correction factors values are expressed as unsigned Q2.10 integers + * ranging from 1 to 3.999. + * + * Pre-calculated interpolation factors shall be provided in the @x_grad + * and @y_grad fields, expressed as 12 bits integer values. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_LSC) + * @r_data: correction factors for the red channel in Q2.10 format + * @gr_data: correction factors for the green (red) channel in Q2.10 format + * @gb_data: correction factors for the green (blue) channel in Q2.10 format + * @b_data: correction factors for the blue channel in Q2.10 format + * @x_grad: Interpolation gradients for each horizontal sector (12 bits) + * @y_grad: Interpolation gradients for each vertical sector (12 bits) + * @x_sect_size: Horizontal sectors sizes + * @y_sect_size: Vertical sectors sizes + */ +struct rppx1_lsc_params { + struct v4l2_isp_params_block_header header; + __u16 r_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 gr_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 gb_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 b_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 x_grad[RPPX1_LSC_NUM_SECTORS]; + __u16 y_grad[RPPX1_LSC_NUM_SECTORS]; + __u16 x_sect_size[RPPX1_LSC_NUM_SECTORS]; + __u16 y_sect_size[RPPX1_LSC_NUM_SECTORS]; +}; + +/* Gamma Out */ +#define RPPX1_GA_MAX_SAMPLES 17 + +/** + * enum rppx1_ga_seg_mode - Gamma out curve segmentation mode + * + * Segmentation mode of the 16 input sampling points for the Gamma Out + * Correction module. + * + * @RPPX1_GA_SEG_MODE_LOGARITHMIC: logarithmic-like segmentation mode + * @RPPX1_GA_SEG_MODE_EQUIDISTANT: equidistant segmentation mode + */ +enum rppx1_ga_seg_mode { + RPPX1_GA_SEG_MODE_LOGARITHMIC, + RPPX1_GA_SEG_MODE_EQUIDISTANT +}; + +/** + * struct rppx1_ga_params - Gamma Out Correction configuration + * + * The Gamma Out Correction module is available on the Human Vision Output + * Pipe (HV) and the Machine Vision Output Pipe (MV). Userspace selects + * which pipe to operate by setting the @header.type field to + * RPPX1_PARAMS_BLOCK_TYPE_GA_HV or RPPX1_PARAMS_BLOCK_TYPE_GA_MV. + * + * The module allows to apply a @gamma_y gamma correction curve to RGB data + * represented as a table of 16 entries. The 16 input sampling points can be + * equidistant or segmented using a logarithmic scale according to the value of + * @mode. + * + * The gamma curve values are 12 bits on the HV output pipe and 24 bits on the + * MV output pipe. Userspace is expected to provide the curve values with a + * bit-depth matching the one of pipe in use. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_GA_HV or + * type = RPPX1_PARAMS_BLOCK_TYPE_GA_MV) + * @gamma_y: gamma out curve y-axis values + * @mode: gamma curve input segmentation mode (see rppx1_ga_seg_mode) + * @reserved: padding + */ +struct rppx1_ga_params { + struct v4l2_isp_params_block_header header; + __u32 gamma_y[RPPX1_GA_MAX_SAMPLES]; + __u8 mode; + __u8 reserved[3]; +}; + +/* Linearization (Sensor De-gamma) */ +#define RPPX1_LIN_SAMPLE_POINTS_NUM 16 +#define RPPX1_LIN_DEGAMMA_CURVE_NUM 17 + +/** + * struct rppx1_lin_params - Linearization (Sensor De-gamma) configuration + * + * The RPP-X1 linearization module is available on the PRE1 and PRE2 pre-fusion + * pipes. Userspace selects which pipe to operate by setting the @header.type + * field to RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2. + * + * The LIN module applies the per-color channel de-gamma linearization curves + * @curve_r, @curve_g and @curve_b defined on the input sampling points @dx. + * + * For the PRE1 pipe the de-gamma curves values are 24-bits, for the PRE2 pipe + * the de-gamma curve values are 12-bits. + * + * For the PRE1 pipe de-gamma module sampling points @dx values are in the range + * [0, 15] (4 bits). For the PRE2 pipe de-gamma module sampling points values + * are in the range [0, 7] (3 bits). + * + * Userspace is expected to provide the curve values and sampling points with a + * bit-depth matching the one of pipe in use. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2) + * @curve_r: de-gamma linearization curve for red channel + * @curve_g: de-gamma linearization curve for green channel + * @curve_b: de-gamma linearization curve for blue channel + * @dx: input sampling points + * @reserved: padding + */ +struct rppx1_lin_params { + struct v4l2_isp_params_block_header header; + __u32 curve_r[RPPX1_LIN_DEGAMMA_CURVE_NUM]; + __u32 curve_g[RPPX1_LIN_DEGAMMA_CURVE_NUM]; + __u32 curve_b[RPPX1_LIN_DEGAMMA_CURVE_NUM]; + __u8 dx[RPPX1_LIN_SAMPLE_POINTS_NUM]; + __u32 reserved; +}; + +/** + * RPPX1_PARAMS_MAX_SIZE - Maximum size of all RPP-X1 parameter blocks + * + * Some types are reported twice as the same block might be instantiated in + * multiple pipes. + */ +#define RPPX1_PARAMS_MAX_SIZE \ + (sizeof(struct rppx1_wbmeas_params) + \ + sizeof(struct rppx1_awbg_params) + \ + sizeof(struct rppx1_awbg_params) + \ + sizeof(struct rppx1_awbg_params) + \ + sizeof(struct rppx1_exm_params) + \ + sizeof(struct rppx1_exm_params) + \ + sizeof(struct rppx1_hist_params) + \ + sizeof(struct rppx1_hist_params) + \ + sizeof(struct rppx1_hist_params) + \ + sizeof(struct rppx1_bls_params) + \ + sizeof(struct rppx1_bls_params) + \ + sizeof(struct rppx1_ccor_params) + \ + sizeof(struct rppx1_lsc_params) + \ + sizeof(struct rppx1_lsc_params) + \ + sizeof(struct rppx1_ga_params) + \ + sizeof(struct rppx1_ga_params) + \ + sizeof(struct rppx1_lin_params) + \ + sizeof(struct rppx1_lin_params)) + +/* --------------------------------------------------------------------------- + * Statistics Structures + * + * The same ISP block might be instantiated in multiple pipeliness and operate + * on a different bitdepth/precision. For fields of varying length among + * different instances of the same block, use a data type that can accommodate + * the larger bitdepth/precision. + */ + +/** + * enum rppx1_stats_block_type - RPP-X1 extensible stats block types + * + * NOTE: Only append to the enumeration as the numbers are uAPI. + * + * @RPPX1_STATS_BLOCK_TYPE_WBMEAS_POST: post-fusion white-balance measurement + * @RPPX1_STATS_BLOCK_TYPE_EXM_PRE1: pre-fusion pipe1 exposure measurement + * @RPPX1_STATS_BLOCK_TYPE_EXM_PRE2: pre-fusion pipe2 exposure measurement + * @RPPX1_STATS_BLOCK_TYPE_HIST_PRE1: pre-fusion pipe1 histogram + * @RPPX1_STATS_BLOCK_TYPE_HIST_PRE2: pre-fusion pipe2 histogram + * @RPPX1_STATS_BLOCK_TYPE_HIST_POST: post-fusion histogram + */ +enum rppx1_stats_block_type { + RPPX1_STATS_BLOCK_TYPE_WBMEAS_POST, + RPPX1_STATS_BLOCK_TYPE_EXM_PRE1, + RPPX1_STATS_BLOCK_TYPE_EXM_PRE2, + RPPX1_STATS_BLOCK_TYPE_HIST_PRE1, + RPPX1_STATS_BLOCK_TYPE_HIST_PRE2, + RPPX1_STATS_BLOCK_TYPE_HIST_POST, +}; + +/** + * struct rppx1_wbmeas_stats - AWB statistics + * + * @header: block header (type = RPPX1_STATS_BLOCK_TYPE_WBMEAS_POST) + * @cnt: Number of pixels matched + * @mean_y_or_g: mean Y (or G in RGB mode) value, 24-bit + * @mean_cb_or_b: mean Cb (or B in RGB mode) value, 24-bit + * @mean_cr_or_r: mean Cr (or R in RGB mode) value, 24-bit + */ +struct rppx1_wbmeas_stats { + struct v4l2_isp_block_header header; + __u32 cnt; + __u32 mean_y_or_g; + __u32 mean_cb_or_b; + __u32 mean_cr_or_r; +}; + +/* Exposure Measurement */ +#define RPPX1_EXM_NUM_WIN 25 + +/** + * struct rppx1_exm_stats - Exposure measurement + * + * RPP-X1 exposure measurement calculates the mean value on 25 programmable + * windows on the input picture. + * + * @header: block header (type = RPPX1_STATS_BLOCK_TYPE_EXM_PRE1) + * @exp_mean: mean luminance values per block, up to 20-bit + * @reserved: padding + */ +struct rppx1_exm_stats { + struct v4l2_isp_block_header header; + __u32 exp_mean[RPPX1_EXM_NUM_WIN]; + __u32 reserved; +}; + +/* Histogram */ +#define RPPX1_HIST_NUM_BINS 32 + +/** + * struct rppx1_hist_stats - Histogram statistics + * + * @header: block header (type = RPPX1_STATS_BLOCK_TYPE_HIST_POST) + * @hist_bins: accumulation histogram results in unsigned 20-bit Q16.4 format + */ +struct rppx1_hist_stats { + struct v4l2_isp_block_header header; + __u32 hist_bins[RPPX1_HIST_NUM_BINS]; +}; + +/** + * RPPX1_STATS_MAX_SIZE - Maximum size of all RPP-X1 statistics + * + * Some types are reported twice as the same block might be instantiated in + * multiple pipes. + */ +#define RPPX1_STATS_MAX_SIZE \ + (sizeof(struct rppx1_wbmeas_stats) + \ + sizeof(struct rppx1_exm_stats) + \ + sizeof(struct rppx1_exm_stats) + \ + sizeof(struct rppx1_hist_stats) + \ + sizeof(struct rppx1_hist_stats) + \ + sizeof(struct rppx1_hist_stats)) + +#endif /* __UAPI_RPP_X1_CONFIG_H */ diff --git a/include/linux/rkisp1-config.h b/include/linux/rkisp1-config.h index 3e8aa77fde..e2c755e1a9 100644 --- a/include/linux/rkisp1-config.h +++ b/include/linux/rkisp1-config.h @@ -964,6 +964,92 @@ struct rkisp1_cif_isp_wdr_config { __u8 use_iref; }; +/* + * enum rkisp1_cif_isp_cac_h_clip_mode - horizontal clipping mode + * + * @RKISP1_CIF_ISP_CAC_H_CLIP_MODE_4PX: +/- 4 pixels + * @RKISP1_CIF_ISP_CAC_H_CLIP_MODE_4_5PX: +/- 4/5 pixels depending on bayer position + */ +enum rkisp1_cif_isp_cac_h_clip_mode { + RKISP1_CIF_ISP_CAC_H_CLIP_MODE_4PX = 0, + RKISP1_CIF_ISP_CAC_H_CLIP_MODE_4_5PX = 1, +}; + +/** + * enum rkisp1_cif_isp_cac_v_clip_mode - vertical clipping mode + * + * @RKISP1_CIF_ISP_CAC_V_CLIP_MODE_2PX: +/- 2 pixels + * @RKISP1_CIF_ISP_CAC_V_CLIP_MODE_3PX: +/- 3 pixels + * @RKISP1_CIF_ISP_CAC_V_CLIP_MODE_3_4PX: +/- 3/4 pixels depending on bayer position + */ +enum rkisp1_cif_isp_cac_v_clip_mode { + RKISP1_CIF_ISP_CAC_V_CLIP_MODE_2PX = 0, + RKISP1_CIF_ISP_CAC_V_CLIP_MODE_3PX = 1, + RKISP1_CIF_ISP_CAC_V_CLIP_MODE_3_4PX = 2, +}; + +/** + * struct rkisp1_cif_isp_cac_config - chromatic aberration correction configuration + * + * The correction is carried out by shifting the red and blue pixels relative + * to the green ones, depending on the distance from the optical center: + * + * @h_count_start: horizontal coordinate of the optical center (13-bit unsigned integer; [1,8191]) + * @v_count_start: vertical coordinate of the optical center (13-bit unsigned integer; [1,8191]) + * + * For each pixel, the x/y distances from the optical center are calculated and + * then transformed into the [0,255] range based on the following formula: + * + * (((d << 4) >> ns) * nf) >> 5 + * + * where `d` is the distance, `ns` and `nf` are the normalization parameters: + * + * @x_nf: horizontal normalization scale parameter (5-bit unsigned integer; [0,31]) + * @x_ns: horizontal normalization shift parameter (4-bit unsigned integer; [0,15]) + * + * @y_nf: vertical normalization scale parameter (5-bit unsigned integer; [0,31]) + * @y_ns: vertical normalization shift parameter (4-bit unsigned integer; [0,15]) + * + * These parameters should be chosen based on the image resolution, the position + * of the optical center, and the shape of pixels, so that no normalized distance + * is larger than 255. If the pixels have square shape, the two sets of parameters + * should be equal. + * + * The actual amount of correction is calculated with a third degree polynomial: + * + * c[0] * r + c[1] * r^2 + c[2] * r^3 + * + * where `c` is the set of coefficients for the given color, and `r` is distance: + * + * @red: red coefficients (5.4 two's complement; [-16,15.9375]) + * @blue: blue coefficients (5.4 two's complement; [-16,15.9375]) + * + * Finally, the amount is clipped as requested: + * + * @h_clip_mode: maximum horizontal shift (from enum rkisp1_cif_isp_cac_h_clip_mode) + * @v_clip_mode: maximum vertical shift (from enum rkisp1_cif_isp_cac_v_clip_mode) + * + * A positive result will shift away from the optical center, while a negative + * one will shift towards the optical center. In the latter case, the pixel + * values at the edges are duplicated. + */ +struct rkisp1_cif_isp_cac_config { + __u8 h_clip_mode; + __u8 v_clip_mode; + + __u16 h_count_start; + __u16 v_count_start; + + __u16 red[3]; + __u16 blue[3]; + + __u8 x_nf; + __u8 x_ns; + + __u8 y_nf; + __u8 y_ns; +}; + /*---------- PART2: Measurement Statistics ------------*/ /** @@ -1135,6 +1221,7 @@ struct rkisp1_stat_buffer { * @RKISP1_EXT_PARAMS_BLOCK_TYPE_COMPAND_EXPAND: Companding expand curve * @RKISP1_EXT_PARAMS_BLOCK_TYPE_COMPAND_COMPRESS: Companding compress curve * @RKISP1_EXT_PARAMS_BLOCK_TYPE_WDR: Wide dynamic range + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CAC: Chromatic aberration correction */ enum rkisp1_ext_params_block_type { RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS, @@ -1158,6 +1245,7 @@ enum rkisp1_ext_params_block_type { RKISP1_EXT_PARAMS_BLOCK_TYPE_COMPAND_EXPAND, RKISP1_EXT_PARAMS_BLOCK_TYPE_COMPAND_COMPRESS, RKISP1_EXT_PARAMS_BLOCK_TYPE_WDR, + RKISP1_EXT_PARAMS_BLOCK_TYPE_CAC, }; /* For backward compatibility */ @@ -1504,6 +1592,22 @@ struct rkisp1_ext_params_wdr_config { struct rkisp1_cif_isp_wdr_config config; } __attribute__((aligned(8))); +/** + * struct rkisp1_ext_params_cac_config - RkISP1 extensible params CAC config + * + * RkISP1 extensible parameters CAC block. + * Identified by :c:type:`RKISP1_EXT_PARAMS_BLOCK_TYPE_CAC`. + * + * @header: The RkISP1 extensible parameters header, see + * :c:type:`rkisp1_ext_params_block_header` + * @config: CAC configuration, see + * :c:type:`rkisp1_cif_isp_cac_config` + */ +struct rkisp1_ext_params_cac_config { + struct rkisp1_ext_params_block_header header; + struct rkisp1_cif_isp_cac_config config; +} __attribute__((aligned(8))); + /* * The rkisp1_ext_params_compand_curve_config structure is counted twice as it * is used for both the COMPAND_EXPAND and COMPAND_COMPRESS block types. @@ -1529,14 +1633,15 @@ struct rkisp1_ext_params_wdr_config { sizeof(struct rkisp1_ext_params_compand_bls_config) +\ sizeof(struct rkisp1_ext_params_compand_curve_config) +\ sizeof(struct rkisp1_ext_params_compand_curve_config) +\ - sizeof(struct rkisp1_ext_params_wdr_config)) + sizeof(struct rkisp1_ext_params_wdr_config) +\ + sizeof(struct rkisp1_ext_params_cac_config)) /** - * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version + * enum rkisp1_ext_param_buffer_version - RkISP1 extensible parameters version * * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters */ -enum rksip1_ext_param_buffer_version { +enum rkisp1_ext_param_buffer_version { RKISP1_EXT_PARAM_BUFFER_V1 = V4L2_ISP_PARAMS_VERSION_V1, }; @@ -1598,7 +1703,7 @@ enum rksip1_ext_param_buffer_version { * +---------------------------------------------------------------------+ * * @version: The RkISP1 extensible parameters buffer version, see - * :c:type:`rksip1_ext_param_buffer_version` + * :c:type:`rkisp1_ext_param_buffer_version` * @data_size: The RkISP1 configuration data effective size, excluding this * header * @data: The RkISP1 extensible configuration data blocks diff --git a/include/linux/v4l2-controls.h b/include/linux/v4l2-controls.h index c0d9cab3c7..9834301d7d 100644 --- a/include/linux/v4l2-controls.h +++ b/include/linux/v4l2-controls.h @@ -121,6 +121,15 @@ enum v4l2_colorfx { */ #define V4L2_CID_USER_MEYE_BASE (V4L2_CID_USER_BASE + 0x1000) +/* + * The base for the vim2m driver controls. + * We reserve 16 controls for this driver. + * The vim2m control range clashed with the meye control range, which was not + * intended, but since the meye driver has been removed, we can just keep the + * vim2m control range. + */ +#define V4L2_CID_USER_VIM2M_BASE (V4L2_CID_USER_BASE + 0x1000) + /* The base for the bttv driver controls. * We reserve 32 controls for this driver. */ #define V4L2_CID_USER_BTTV_BASE (V4L2_CID_USER_BASE + 0x1010) @@ -236,6 +245,18 @@ enum v4l2_colorfx { */ #define V4L2_CID_USER_MALI_C55_BASE (V4L2_CID_USER_BASE + 0x1230) +/* + * The base for the ST DCMIPP driver controls. + * We reserve 16 controls for this driver + */ +#define V4L2_CID_USER_DCMIPP_BASE (V4L2_CID_USER_BASE + 0x1240) + +/* + * The base for IT6625/IT6626 driver controls. + * We reserve 16 controls for this driver. + */ +#define V4L2_CID_USER_IT6625_BASE (V4L2_CID_USER_BASE + 0x1250) + /* MPEG-class control IDs */ /* The MPEG controls are applicable to all codec controls * and the 'MPEG' part of the define is historical */ @@ -464,6 +485,8 @@ enum v4l2_mpeg_video_intra_refresh_period_type { V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC = 1, }; +#define V4L2_CID_MPEG_VIDEO_BACKGROUND_DETECTION (V4L2_CID_CODEC_BASE + 238) + /* CIDs for the MPEG-2 Part 2 (H.262) codec */ #define V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL (V4L2_CID_CODEC_BASE+270) enum v4l2_mpeg_video_mpeg2_level { diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 8bccad84ff..c823409c7f 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -864,6 +864,10 @@ struct v4l2_pix_format { #define V4L2_META_FMT_MALI_C55_PARAMS v4l2_fourcc('C', '5', '5', 'P') /* ARM Mali-C55 Parameters */ #define V4L2_META_FMT_MALI_C55_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */ +/* Vendor specific - used for Dreamchip RPP-X1 ISP */ +#define V4L2_META_FMT_RPPX1_PARAMS v4l2_fourcc('D', 'R', '1', 'P') /* Dreamchip RPP-X1 Parameters */ +#define V4L2_META_FMT_RPPX1_STATS v4l2_fourcc('D', 'R', '1', 'S') /* Dreamchip RPP-X1 Statistics */ + /* * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when * adding new ones! diff --git a/utils/update-kernel-headers.sh b/utils/update-kernel-headers.sh index 3fdea5f1eb..488b87de59 100755 --- a/utils/update-kernel-headers.sh +++ b/utils/update-kernel-headers.sh @@ -56,6 +56,7 @@ headers=" linux/media-bus-format.h linux/media.h linux/media/arm/mali-c55-config.h + linux/media/dreamchip/rppx1-config.h linux/media/v4l2-isp.h linux/rkisp1-config.h linux/stddef.h From patchwork Mon Nov 16 17:49:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28926 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 129E1C3360 for ; Mon, 16 Nov 2026 17:49:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D3646A87A; Mon, 16 Nov 2026 18:49:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b5LL7f6U"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E0856A85F for ; Mon, 16 Nov 2026 18:49:29 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 11F7E929; Mon, 16 Nov 2026 18:47:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851222; bh=BxaJoGI1cYsRMGTZyHTz7jkwzMm5Cp8qtdG1RA+rlbs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b5LL7f6UpxIvzRr0LEcoG2C4Y/y3yAURlIwFGhH8kZ67aoIGsYabgsGHYXjtaSaP1 QFcDWsmOWVspxgW3x05FrTvkl4mt/tK5HyOtWkBqMbLbybkY3al3UoFlmQbCwc/Tps K4fOGHVAXc9hEmgFdkfLPbUthX2gqHWW735OiSu8= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Niklas_S=C3=B6derlund?= Subject: [PATCH v5 12/22] libcamera: pipeline: Add R-Car Gen4 ISP pipeline Date: Mon, 16 Nov 2026 18:49:08 +0100 Message-ID: <20261116174919.1056403-13-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Niklas Söderlund Add a pipeline handler for R-Car Gen4. The pipeline will make use of the new "rppx1" ipa module that will be introduced momentarily. There are two distinct parts of the pipeline: one capture block (VIN) that is responsible for acquiring the raw images from the sensor and supplying them to user-space, and the ISP operating in memory-to-memory mode that produces the processed images and statistics from the raw image. Notably, there is no usable scaler in this configuration (the VIN device can only scale non-raw images), so the resolutions need to match in the whole pipeline beginning with the sensor. The pipeline handler supports a maximum of one raw and one processed streams at a time, both with the same resolution. Tested on R-Car V4H Sparrow-Hawk together with IMX219 and IMX708 sensors. Signed-off-by: Niklas Söderlund Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- changes in v3: * rebased * uses new ipa interface (rppx1) instead of the rkisp1 * remove the queue of pending requests and use the queueing feature of the PipelineHandler base class * reworked format handling * improved orientation handling wrt. raw formats v2: https://patchwork.libcamera.org/cover/24304/ v1: https://patchwork.libcamera.org/cover/23591/ --- * Changes since v1 - Fix camera names. Was 'imx462 2-001a', is now '/base/soc/i2c@e6510000/cam@1a'. - Fix a compiler issue with some versions of gcc. - Add dependency on rkisp1.mojom. - Rebase on latest master branch which requires reworking some interfaces. --- Documentation/Doxyfile-common.in | 1 + include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/rppx1.mojom | 40 + meson.build | 1 + meson_options.txt | 1 + src/libcamera/pipeline/rcar-gen4/frames.cpp | 264 ++++++ src/libcamera/pipeline/rcar-gen4/frames.h | 84 ++ src/libcamera/pipeline/rcar-gen4/isp.cpp | 194 ++++ src/libcamera/pipeline/rcar-gen4/isp.h | 39 + src/libcamera/pipeline/rcar-gen4/meson.build | 8 + .../pipeline/rcar-gen4/rcar-gen4.cpp | 880 ++++++++++++++++++ src/libcamera/pipeline/rcar-gen4/vin.cpp | 177 ++++ src/libcamera/pipeline/rcar-gen4/vin.h | 54 ++ 13 files changed, 1744 insertions(+) create mode 100644 include/libcamera/ipa/rppx1.mojom create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.cpp create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.h create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.cpp create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.h create mode 100644 src/libcamera/pipeline/rcar-gen4/meson.build create mode 100644 src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.cpp create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.h diff --git a/Documentation/Doxyfile-common.in b/Documentation/Doxyfile-common.in index f11ec593d5..131fdcc608 100644 --- a/Documentation/Doxyfile-common.in +++ b/Documentation/Doxyfile-common.in @@ -37,6 +37,7 @@ EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \ @TOP_BUILDDIR@/include/libcamera/ipa/mali-c55_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/raspberrypi_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/rkisp1_*.h \ + @TOP_BUILDDIR@/include/libcamera/ipa/rppx1_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/vimc_*.h EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \ diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index fb518526a2..7dcefc8b26 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -65,6 +65,7 @@ libcamera_ipa_headers += custom_target('core_ipa_serializer_h', pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'mali-c55': 'mali-c55.mojom', + 'rcar-gen4': 'rppx1.mojom', 'rkisp1': 'rkisp1.mojom', 'rpi/pisp': 'raspberrypi.mojom', 'rpi/vc4': 'raspberrypi.mojom', diff --git a/include/libcamera/ipa/rppx1.mojom b/include/libcamera/ipa/rppx1.mojom new file mode 100644 index 0000000000..6d280b9ef1 --- /dev/null +++ b/include/libcamera/ipa/rppx1.mojom @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + */ + +module ipa.rppx1; + +import "include/libcamera/ipa/core.mojom"; + +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; +}; + +interface IPARppX1Interface { + init(libcamera.IPASettings settings, + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers); + unmapBuffers(array ids); + + [async] queueRequest(uint32 frame, libcamera.ControlList reqControls); + [async] computeParams(uint32 frame, uint32 bufferId); + [async] processStats(uint32 frame, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPARppX1EventInterface { + paramsComputed(uint32 frame, uint32 bytesused); + setSensorControls(uint32 frame, libcamera.ControlList sensorControls); + metadataReady(uint32 frame, libcamera.ControlList metadata); +}; diff --git a/meson.build b/meson.build index 6d54fc7a93..265a825ffa 100644 --- a/meson.build +++ b/meson.build @@ -218,6 +218,7 @@ pipelines_support = { 'imx8-isi': arch_arm, 'ipu3': arch_x86, 'mali-c55': arch_arm, + 'rcar-gen4': arch_arm, 'rkisp1': arch_arm, 'rpi/pisp': arch_arm, 'rpi/vc4': arch_arm, diff --git a/meson_options.txt b/meson_options.txt index e829ebe604..3c4bc9f8e7 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -81,6 +81,7 @@ option('pipelines', 'imx8-isi', 'ipu3', 'mali-c55', + 'rcar-gen4', 'rkisp1', 'rpi/pisp', 'rpi/vc4', diff --git a/src/libcamera/pipeline/rcar-gen4/frames.cpp b/src/libcamera/pipeline/rcar-gen4/frames.cpp new file mode 100644 index 0000000000..b94bbf8e59 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/frames.cpp @@ -0,0 +1,264 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#include "frames.h" + +#include +#include + +#include +#include + +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/pipeline_handler.h" + +#include "isp.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RCar4) + +int RCar4Frames::start(RCarISPDevice *isp, ipa::rppx1::IPAProxyRppX1 *ipa, + unsigned int bufferCount) +{ + unsigned int ipaBufferId = 1; + + auto pushBuffers = [&](const std::vector> &buffers, + std::queue &queue) { + for (const std::unique_ptr &buffer : buffers) { + std::span planes = buffer->planes(); + + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), + std::vector{ planes.begin(), + planes.end() }); + queue.push(buffer.get()); + } + }; + + frameInfo_.clear(); + + utils::scope_exit stopGuard([&] { stop(isp, ipa); }); + + int ret = isp->input_->exportBuffers(bufferCount, &inputBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP input buffers"; + return ret; + } + + ret = isp->param_->allocateBuffers(bufferCount, ¶mBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP param buffers"; + return ret; + } + + ret = isp->stat_->allocateBuffers(bufferCount, &statBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP stat buffers"; + return ret; + } + + ret = isp->output_->exportBuffers(bufferCount, &outputBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP output buffers"; + return ret; + } + + for (const std::unique_ptr &buffer : inputBuffers_) + availableInputBuffers_.push(buffer.get()); + + pushBuffers(paramBuffers_, availableParamBuffers_); + pushBuffers(statBuffers_, availableStatBuffers_); + + for (const std::unique_ptr &buffer : outputBuffers_) + availableOutputBuffers_.push(buffer.get()); + + ipa->mapBuffers(ipaBuffers_); + + stopGuard.release(); + return 0; +} + +void RCar4Frames::stop(RCarISPDevice *isp, ipa::rppx1::IPAProxyRppX1 *ipa) +{ + std::vector ids; + + availableInputBuffers_ = {}; + availableParamBuffers_ = {}; + availableStatBuffers_ = {}; + availableOutputBuffers_ = {}; + + outputBuffers_.clear(); + statBuffers_.clear(); + paramBuffers_.clear(); + inputBuffers_.clear(); + + for (IPABuffer &ipabuf : ipaBuffers_) + ids.push_back(ipabuf.id); + + ipa->unmapBuffers(ids); + ipaBuffers_.clear(); + + if (isp->stat_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP stat buffers"; + + if (isp->param_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP param buffers"; +} + +RCar4Frames::Info *RCar4Frames::create(Request *request) +{ + unsigned int frame = request->sequence(); + + /* Try to get input and output buffers from request. */ + FrameBuffer *inputBuffer = request->findBuffer(&rawStream_); + FrameBuffer *outputBuffer = request->findBuffer(&outputStream_); + + /* Make sure we have enough internal buffers. */ + if (!inputBuffer && availableInputBuffers_.empty()) { + LOG(RCar4, Debug) << "Input buffer underrun"; + return nullptr; + } + + if (availableParamBuffers_.empty()) { + LOG(RCar4, Debug) << "Parameters buffer underrun"; + return nullptr; + } + + if (availableStatBuffers_.empty()) { + LOG(RCar4, Debug) << "Statistics buffer underrun"; + return nullptr; + } + + if (!outputBuffer && availableOutputBuffers_.empty()) { + LOG(RCar4, Debug) << "Output buffer underrun"; + return nullptr; + } + + /* Select buffers to use. */ + if (!inputBuffer) { + inputBuffer = availableInputBuffers_.front(); + availableInputBuffers_.pop(); + } + + FrameBuffer *paramBuffer = availableParamBuffers_.front(); + availableParamBuffers_.pop(); + + FrameBuffer *statBuffer = availableStatBuffers_.front(); + availableStatBuffers_.pop(); + + if (!outputBuffer) { + outputBuffer = availableOutputBuffers_.front(); + availableOutputBuffers_.pop(); + } + + /* Record the info needed to process one frame. */ + auto [it, inserted] = frameInfo_.try_emplace(frame); + if (!inserted) + return nullptr; + + auto &info = it->second; + + info.frame = frame; + info.request = request; + info.inputBuffer = inputBuffer; + info.paramBuffer = paramBuffer; + info.statBuffer = statBuffer; + info.outputBuffer = outputBuffer; + info.rawDequeued = false; + info.paramDequeued = false; + info.metadataProcessed = false; + info.outputDequeued = false; + + return &info; +} + +void RCar4Frames::remove(RCar4Frames::Info *info) +{ + /* If internal input buffer used, return for reuse. */ + for (const std::unique_ptr &buf : inputBuffers_) { + if (info->inputBuffer == buf.get()) { + availableInputBuffers_.push(info->inputBuffer); + break; + } + } + + /* Return param and stat buffer for reuse. */ + availableParamBuffers_.push(info->paramBuffer); + availableStatBuffers_.push(info->statBuffer); + + /* If internal output buffer used, return for reuse. */ + for (const std::unique_ptr &buf : outputBuffers_) { + if (info->outputBuffer == buf.get()) { + availableOutputBuffers_.push(info->outputBuffer); + break; + } + } + + /* Delete the extended frame information. */ + frameInfo_.erase(info->frame); +} + +bool RCar4Frames::tryComplete(RCar4Frames::Info *info) +{ + Request *request = info->request; + + if (request->hasPendingBuffers()) + return false; + + if (!info->rawDequeued) + return false; + + if (!info->metadataProcessed) + return false; + + if (!info->paramDequeued) + return false; + + if (!info->outputDequeued) + return false; + + remove(info); + + return true; +} + +RCar4Frames::Info *RCar4Frames::find(unsigned int frame) +{ + const auto &itInfo = frameInfo_.find(frame); + + if (itInfo != frameInfo_.end()) + return &itInfo->second; + + LOG(RCar4, Fatal) << "Can't find tracking information for frame " << frame; + + return nullptr; +} + +RCar4Frames::Info *RCar4Frames::find(FrameBuffer *buffer) +{ + for (auto &itInfo : frameInfo_) { + Info *info = &itInfo.second; + + for (const auto &[stream, fb] : info->request->buffers()) + if (buffer == fb) + return info; + + if (info->inputBuffer == buffer || + info->paramBuffer == buffer || + info->statBuffer == buffer || + info->outputBuffer == buffer) + return info; + } + + LOG(RCar4, Fatal) << "Can't find tracking information from buffer"; + + return nullptr; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/frames.h b/src/libcamera/pipeline/rcar-gen4/frames.h new file mode 100644 index 0000000000..79365f38e5 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/frames.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#pragma once + +#include +#include +#include +#include + +#include + +#include +#include + +#include + +#include "isp.h" + +namespace libcamera { + +class RCarISPDevice; +class FrameBuffer; +class Request; + +class RCar4Frames +{ +public: + struct Info { + unsigned int frame; + Request *request; + + FrameBuffer *inputBuffer; + FrameBuffer *paramBuffer; + FrameBuffer *statBuffer; + FrameBuffer *outputBuffer; + + ControlList effectiveSensorControls; + + bool rawDequeued; + bool paramDequeued; + bool metadataProcessed; + bool outputDequeued; + }; + + int start(RCarISPDevice *isp, ipa::rppx1::IPAProxyRppX1 *ipa, + unsigned int bufferCount); + void stop(RCarISPDevice *isp, ipa::rppx1::IPAProxyRppX1 *ipa); + + Info *create(Request *request); + void remove(Info *info); + bool tryComplete(Info *info); + + Info *find(unsigned int frame); + Info *find(FrameBuffer *buffer); + + Stream rawStream_; + Stream outputStream_; + +private: + std::map frameInfo_; + + /* Buffers for internal use, if none is provided in request. */ + std::vector> inputBuffers_; + std::vector> paramBuffers_; + std::vector> statBuffers_; + std::vector> outputBuffers_; + + /* Queues of available internal buffers. */ + std::queue availableInputBuffers_; + std::queue availableParamBuffers_; + std::queue availableStatBuffers_; + std::queue availableOutputBuffers_; + + /* Buffers mapped and shared with IPA. */ + std::vector ipaBuffers_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/isp.cpp b/src/libcamera/pipeline/rcar-gen4/isp.cpp new file mode 100644 index 0000000000..5b1472f374 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/isp.cpp @@ -0,0 +1,194 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 ISP pipeline + */ + +#include "isp.h" + +#include + +#include +#include + +#include +#include + +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_subdevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RCar4) + +int RCarISPDevice::init(const MediaDevice *media, const std::string &pipeId) +{ + const MediaEntity *entity; + const MediaPad *pad, *next; + + /* Locate IPSCORE, e.g. rcar_isp fed00000.isp core */ + std::unique_ptr core = + V4L2Subdevice::fromEntityName(media, pipeId + " core"); + if (!core) { + LOG(RCar4, Error) << "Failed to find ISPCORE " << pipeId; + return -EINVAL; + } + + entity = core->entity(); + + /* Use the media links to find all video devices. */ + pad = entity->getPadByIndex(0); + next = pad->links()[0]->source(); + input_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!input_) { + LOG(RCar4, Error) << "Failed to find ISP input entity"; + return -EINVAL; + } + + pad = entity->getPadByIndex(1); + next = pad->links()[0]->source(); + param_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!param_) { + LOG(RCar4, Error) << "Failed to find ISP param entity"; + return -EINVAL; + } + + pad = entity->getPadByIndex(2); + next = pad->links()[0]->sink(); + stat_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!stat_) { + LOG(RCar4, Error) << "Failed to find ISP stat entity"; + return -EINVAL; + } + + pad = entity->getPadByIndex(3); + next = pad->links()[0]->sink(); + output_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!output_) { + LOG(RCar4, Error) << "Failed to find ISP output entity"; + return -EINVAL; + } + + /* Open all devices. */ + int ret = input_->open(); + if (ret) + return ret; + + ret = param_->open(); + if (ret) + return ret; + + ret = stat_->open(); + if (ret) + return ret; + + ret = output_->open(); + if (ret) + return ret; + + return 0; +} + +int RCarISPDevice::configure(const V4L2DeviceFormat &sensorFormat, + const PixelFormat &outputPixelFormat) +{ + auto inputFormat = sensorFormat; + + /* Configure the RAW input. */ + int ret = input_->setFormat(&inputFormat); + if (ret) + return ret; + + if (inputFormat.fourcc != sensorFormat.fourcc || inputFormat.size != sensorFormat.size) + return -EINVAL; + + /* Configure the image output. */ + V4L2DeviceFormat outputFormat = {}; + V4L2PixelFormat outputPf = output_->toV4L2PixelFormat(outputPixelFormat); + outputFormat.fourcc = outputPf; + outputFormat.size = inputFormat.size; + ret = output_->setFormat(&outputFormat); + if (ret) + return ret; + + if (outputFormat.fourcc != outputPf || outputFormat.size != inputFormat.size) + return -EINVAL; + + /* Configure paramaters. */ + V4L2DeviceFormat paramFormat = {}; + paramFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPPX1_PARAMS); + ret = param_->setFormat(¶mFormat); + if (ret) + return ret; + + /* Configure statistics. */ + V4L2DeviceFormat statFormat = {}; + statFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPPX1_STATS); + ret = stat_->setFormat(&statFormat); + if (ret) + return ret; + + return 0; +} + +int RCarISPDevice::start(unsigned int bufferCount) +{ + utils::scope_exit stopGuard([&] { stop(); }); + + int ret = input_->importBuffers(bufferCount); + if (ret) { + LOG(RCar4, Error) << "Failed to import ISP input buffers"; + return ret; + } + + ret = output_->importBuffers(bufferCount); + if (ret) { + LOG(RCar4, Error) << "Failed to import ISP output buffers"; + return ret; + } + + ret = output_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP output"; + return ret; + } + + ret = param_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP param"; + return ret; + } + + ret = stat_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP stat"; + return ret; + } + + ret = input_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP input"; + return ret; + } + + stopGuard.release(); + return 0; +} + +void RCarISPDevice::stop() +{ + input_->streamOff(); + stat_->streamOff(); + param_->streamOff(); + output_->streamOff(); + + if (output_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP output buffers"; + + if (input_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP input buffers"; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/isp.h b/src/libcamera/pipeline/rcar-gen4/isp.h new file mode 100644 index 0000000000..55987f1f51 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/isp.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 ISP pipeline + */ + +#pragma once + +#include +#include + +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class MediaDevice; +class Size; +struct StreamConfiguration; + +class RCarISPDevice +{ +public: + int init(const MediaDevice *media, const std::string &pipeId); + + int configure(const V4L2DeviceFormat &inputFormat, + const PixelFormat &outputPixelFormat); + + int start(unsigned int bufferCount); + void stop(); + + std::unique_ptr input_; + std::unique_ptr param_; + std::unique_ptr stat_; + std::unique_ptr output_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/meson.build b/src/libcamera/pipeline/rcar-gen4/meson.build new file mode 100644 index 0000000000..431eb54e28 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'frames.cpp', + 'isp.cpp', + 'rcar-gen4.cpp', + 'vin.cpp', +]) diff --git a/src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp b/src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp new file mode 100644 index 0000000000..40d4c8dbbc --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp @@ -0,0 +1,880 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 ISP pipeline + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include + +#include + +#include +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +#include "frames.h" +#include "isp.h" +#include "vin.h" + +namespace libcamera { + +namespace { + +static constexpr unsigned int kMaxRequests = 4; +static constexpr unsigned int kDefaultBufferCount = kMaxRequests; + +} /* namespace */ + +LOG_DEFINE_CATEGORY(RCar4) + +/* ----------------------------------------------------------------------------- + * Camera Data + */ + +class RCar4CameraData final : public Camera::Private +{ +public: + RCar4CameraData(PipelineHandler *pipe) + : Camera::Private(pipe) + { + } + + int init(const MediaDevice *mdev, const std::string &pipeId); + + [[nodiscard]] + bool populateFormats(); + + [[nodiscard]] + std::tuple + findSensorFormat(PixelFormat pixelFormat, Size size, Transform transform) const; + + /* Slots for processing ready buffers. */ + void vinBufferReady(FrameBuffer *buffer); + void inputBufferReady(FrameBuffer *buffer); + void paramBufferReady(FrameBuffer *buffer); + void statBufferReady(FrameBuffer *buffer); + void outputBufferReady(FrameBuffer *buffer); + + /* Slots for processing IPA interactions. */ + void paramsComputed(unsigned int frame, unsigned int bytesused); + void setSensorControls(unsigned int frame, + const ControlList &sensorControls); + void metadataReady(unsigned int frame, const ControlList &metadata); + + RCarVINDevice vin_; + RCarISPDevice isp_; + std::unique_ptr ipa_; + + RCar4Frames frames_; + std::unique_ptr delayedCtrls_; + + std::map> rawFormats_; + std::map> outputFormats_; +}; + +int RCar4CameraData::init(const MediaDevice *mdev, const std::string &pipeId) +{ + int ret = vin_.init(mdev, pipeId); + if (ret) + return ret; + + ret = isp_.init(mdev, pipeId); + if (ret) + return ret; + + ipa_ = pipe()->createIPA(1, 1); + if (!ipa_) { + LOG(RCar4, Error) << "No IPA module found"; + return -ENOENT; + } + + /* The IPA tuning file is made from the sensor name. */ + std::string ipaTuningFile = ipa_->configurationFile( + vin_.sensor()->model() + ".yaml", "uncalibrated.yaml"); + + IPACameraSensorInfo sensorInfo; + ret = vin_.sensor()->sensorInfo(&sensorInfo); + if (ret) { + LOG(RCar4, Error) << "Camera sensor information not available"; + return ret; + } + + IPASettings settings{ + std::move(ipaTuningFile), + vin_.sensor()->model(), + }; + + ret = ipa_->init(std::move(settings), sensorInfo, + vin_.sensor()->controls(), &controlInfo_); + if (ret < 0) { + LOG(RCar4, Error) << "IPA initialization failure"; + return ret; + } + + /* + * Initialize the camera properties. + */ + properties_ = vin_.sensor()->properties(); + const CameraSensorProperties::SensorDelays &delays = vin_.sensor()->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + { V4L2_CID_VBLANK, { delays.vblankDelay, true } }, + }; + + delayedCtrls_ = std::make_unique( + vin_.sensor()->device(), params); + + /* Connect bufferReady for each video device to a handler. */ + vin_.bufferReady().connect(this, &RCar4CameraData::vinBufferReady); + isp_.input_->bufferReady.connect(this, &RCar4CameraData::inputBufferReady); + isp_.param_->bufferReady.connect(this, &RCar4CameraData::paramBufferReady); + isp_.stat_->bufferReady.connect(this, &RCar4CameraData::statBufferReady); + isp_.output_->bufferReady.connect(this, &RCar4CameraData::outputBufferReady); + + /* Connect IPA signals. */ + ipa_->setSensorControls.connect(this, &RCar4CameraData::setSensorControls); + ipa_->paramsComputed.connect(this, &RCar4CameraData::paramsComputed); + ipa_->metadataReady.connect(this, &RCar4CameraData::metadataReady); + + /* Apply controls at start of frame. */ + vin_.frameStart().connect(delayedCtrls_.get(), &DelayedControls::applyControls); + + if (!populateFormats()) { + LOG(RCar4, Error) + << "Sensor " << vin_.sensor()->entity()->name() + << " has no format and size compatible with the VIN and ISP"; + return -ENOTSUP; + } + + return 0; +} + +namespace { + +/* + * \todo This should obviously be common code. + */ +void filterSizes(std::vector &sizes, std::span filter) +{ + for (auto it = sizes.begin(); it != sizes.end();) { + bool accept = false; + + for (const auto &range : filter) { + accept = range.contains(*it); + if (accept) + break; + } + + if (!accept) + it = sizes.erase(it); + else + ++it; + } +} + +} /* namespace */ + +/* + * \todo This should obviously be common code. + */ +bool RCar4CameraData::populateFormats() +{ + auto vinFormats = vin_.output()->formats(); + const auto &inputFormats = isp_.input_->formats(); + std::set outputSizes; + + rawFormats_.clear(); + outputFormats_.clear(); + + /* Handle missing `VIDIOC_ENUM_FRMSIZES`. */ + if (vinFormats.empty()) { + static const SizeRange fallbackRange = { + { 5, 2 }, + { 4096, 4096 }, + 1, + 1, + }; + + vinFormats = { + { V4L2PixelFormat(V4L2_PIX_FMT_SBGGR10), std::vector{ fallbackRange } }, + { V4L2PixelFormat(V4L2_PIX_FMT_SGBRG10), std::vector{ fallbackRange } }, + { V4L2PixelFormat(V4L2_PIX_FMT_SGRBG10), std::vector{ fallbackRange } }, + { V4L2PixelFormat(V4L2_PIX_FMT_SRGGB10), std::vector{ fallbackRange } }, + }; + } + + for (unsigned int mbusCode : vin_.sensor()->mbusCodes()) { + auto v4pf = BayerFormat::fromMbusCode(mbusCode).toV4L2PixelFormat(); + + auto it = vinFormats.find(v4pf); + if (it == vinFormats.end()) + continue; + + auto it2 = inputFormats.find(v4pf); + if (it2 == inputFormats.end()) + continue; + + auto sizes = vin_.sensor()->sizes(mbusCode); + filterSizes(sizes, it->second); + filterSizes(sizes, it2->second); + + if (sizes.empty()) + continue; + + /* + * \todo This assumes any input size is accepted as output size + * for all output formats. + */ + outputSizes.insert(sizes.begin(), sizes.end()); + + rawFormats_.try_emplace(mbusCode, std::move(sizes)); + } + + for (const auto &[v4pf, sizes] : isp_.output_->formats()) { + auto pf = v4pf.toPixelFormat(); + if (!pf.isValid()) + continue; + + outputFormats_.try_emplace(pf, outputSizes.begin(), outputSizes.end()); + } + + return !rawFormats_.empty() && !outputFormats_.empty(); +} + +/* + * \todo This should obviously be common code. + * + * CameraSensor::getFormat() is not adequate as it cannot take + * specific requirements along a pipeline into account. + */ +std::tuple +RCar4CameraData::findSensorFormat(PixelFormat targetFormat, Size targetSize, + Transform transform) const +{ + struct { + unsigned int mbusCode; + PixelFormat pf; + Size size; + unsigned bpp; + uint64_t areaDiff = -1; + } best = {}; + + const auto targetArea = uint64_t(targetSize.width) * targetSize.height; + + for (const auto &[mbusCode, sizes] : rawFormats_) { + ASSERT(!sizes.empty()); + + auto bayerFormat = BayerFormat::fromMbusCode(mbusCode); + ASSERT(bayerFormat.isValid()); + bayerFormat.order = vin_.sensor()->bayerOrder(transform); + + auto pf = bayerFormat.toPixelFormat(); + ASSERT(pf.isValid()); + + const auto &info = PixelFormatInfo::info(pf); + + for (const Size &size : sizes) { + const auto area = uint64_t(size.width) * size.height; + const auto areaDiff = utils::abs_diff(targetArea, area); + + if ((pf == targetFormat && best.pf != targetFormat) || + areaDiff < best.areaDiff || + (areaDiff == best.areaDiff && info.bitsPerPixel > best.bpp)) + best = { mbusCode, pf, size, info.bitsPerPixel, areaDiff }; + } + + if (targetFormat.isValid() && best.pf == targetFormat) + break; + } + + LOG(RCar4, Debug) + << "format: " << best.pf << ", " + << "size: " << best.size; + + /* + * The un-transformed mbus code is returned as it is expected + * that the sensor driver handles that correctly. + */ + + return { best.pf, best.mbusCode, best.size }; +} + +void RCar4CameraData::vinBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + /* If the buffer is cancelled force a complete of the whole request. */ + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + frames_.remove(info); + request->_d()->cancel(); + pipe()->completeRequest(request); + return; + } + + /* Record the sensor's timestamp in the request metadata. */ + request->_d()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + ipa_->computeParams(info->frame, info->paramBuffer->cookie()); +} + +void RCar4CameraData::inputBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + if (request->findBuffer(&frames_.rawStream_)) + pipe()->completeBuffer(request, buffer); + + info->rawDequeued = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +void RCar4CameraData::paramBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + info->paramDequeued = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +void RCar4CameraData::statBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + info->metadataProcessed = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); + + return; + } + + ipa_->processStats(info->frame, info->statBuffer->cookie(), + delayedCtrls_->get(buffer->metadata().sequence)); +} + +void RCar4CameraData::outputBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + if (request->findBuffer(&frames_.outputStream_)) + pipe()->completeBuffer(request, buffer); + + request->_d()->metadata().set(controls::draft::PipelineDepth, 3); + + info->outputDequeued = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +void RCar4CameraData::paramsComputed(unsigned int frame, unsigned int bytesused) +{ + RCar4Frames::Info *info = frames_.find(frame); + if (!info) + return; + + info->paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; + + isp_.output_->queueBuffer(info->outputBuffer); + isp_.param_->queueBuffer(info->paramBuffer); + isp_.stat_->queueBuffer(info->statBuffer); + isp_.input_->queueBuffer(info->inputBuffer); +} + +void RCar4CameraData::setSensorControls([[maybe_unused]] unsigned int frame, + const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + +void RCar4CameraData::metadataReady(unsigned int frame, const ControlList &metadata) +{ + RCar4Frames::Info *info = frames_.find(frame); + if (!info) + return; + + Request *request = info->request; + + request->_d()->metadata().merge(metadata); + info->metadataProcessed = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +/* ----------------------------------------------------------------------------- + * Camera Configuration + */ + +class RCar4CameraConfiguration final : public CameraConfiguration +{ +public: + RCar4CameraConfiguration(RCar4CameraData *data); + + Status validate() override; + + const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + const Transform &combinedTransform() { return combinedTransform_; } + const PixelFormat &ispOutputFormat() { return ispOutputFormat_; } + +private: + std::shared_ptr data_; + + V4L2SubdeviceFormat sensorFormat_ = {}; + Transform combinedTransform_ = Transform::Identity; + PixelFormat ispOutputFormat_ = {}; +}; + +RCar4CameraConfiguration::RCar4CameraConfiguration(RCar4CameraData *data) + : CameraConfiguration(), data_(data->_o()->shared_from_this(), data) +{ +} + +CameraConfiguration::Status RCar4CameraConfiguration::validate() +{ + if (config_.empty()) + return Invalid; + + if (sensorConfig) { + LOG(RCar4, Error) + << "Setting sensor configuration is not implemented"; + return Invalid; + } + + Status status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + + /* Cap the number of entries to the available streams. */ + if (config_.size() > 2) { + config_.resize(2); + status = Adjusted; + } + + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->vin_.sensor()->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + + StreamConfiguration *rawCfg = nullptr; + StreamConfiguration *processedCfg = nullptr; + + for (size_t i = 0; i < config_.size(); i++) { + StreamConfiguration &cfg = config_.at(i); + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { + if (rawCfg) { + LOG(RCar4, Error) + << "Camera configuration supports only one RAW stream"; + return Invalid; + } + + rawCfg = &cfg; + } else { + if (processedCfg) { + LOG(RCar4, Error) + << "Camera configuration supports only one processed stream"; + return Invalid; + } + + processedCfg = &cfg; + } + + if (cfg.bufferCount == 0) { + cfg.bufferCount = kDefaultBufferCount; + status = Adjusted; + } + } + + ASSERT(rawCfg || processedCfg); + + auto [sensorFormat, sensorCode, sensorSize] = data_->findSensorFormat( + rawCfg ? rawCfg->pixelFormat : PixelFormat{}, + rawCfg ? rawCfg->size : processedCfg->size, + combinedTransform_); + + V4L2DeviceFormat vinFormat = {}; + const auto vinPf = data_->vin_.output()->toV4L2PixelFormat(sensorFormat); + vinFormat.fourcc = vinPf; + vinFormat.size = sensorSize; + + if (data_->vin_.output()->tryFormat(&vinFormat)) + return Invalid; + + /* The format is expected to be accepted without adjustments. */ + if (vinFormat.fourcc != vinPf || vinFormat.size != sensorSize) + return Invalid; + + ispOutputFormat_ = data_->outputFormats_.begin()->first; + sensorFormat_ = { + .code = sensorCode, + .size = sensorSize, + .colorSpace = ColorSpace::Raw, + }; + + /* + * \todo `rvin_mc_try_format()` always sets it to `V4L2_COLORSPACE_SRGB`, + * so overwrite it with the actually expected color space. + */ + vinFormat.colorSpace = ColorSpace::Raw; + + if (rawCfg) { + if (rawCfg->pixelFormat != sensorFormat) + status = Adjusted; + if (rawCfg->size != sensorSize) + status = Adjusted; + if (rawCfg->colorSpace != vinFormat.colorSpace) + status = Adjusted; + + rawCfg->pixelFormat = sensorFormat; + rawCfg->size = vinFormat.size; + rawCfg->stride = vinFormat.planes[0].bpl; + rawCfg->frameSize = vinFormat.planes[0].size; + rawCfg->colorSpace = vinFormat.colorSpace; + + rawCfg->setStream(&data_->frames_.rawStream_); + } + + if (processedCfg) { + V4L2DeviceFormat ispFormat = {}; + ispFormat.fourcc = data_->isp_.output_->toV4L2PixelFormat( + processedCfg->pixelFormat); + ispFormat.size = sensorSize; + + if (data_->isp_.output_->tryFormat(&ispFormat)) + return Invalid; + + auto pf = ispFormat.fourcc.toPixelFormat(); + if (!pf.isValid()) + return Invalid; + + if (ispFormat.size != vinFormat.size) + return Invalid; + + if (processedCfg->pixelFormat != pf) + status = Adjusted; + if (processedCfg->size != ispFormat.size) + status = Adjusted; + if (processedCfg->colorSpace != ispFormat.colorSpace) + status = Adjusted; + + processedCfg->pixelFormat = pf; + processedCfg->size = ispFormat.size; + processedCfg->stride = ispFormat.planes[0].bpl; + processedCfg->frameSize = ispFormat.planes[0].size; + processedCfg->colorSpace = ispFormat.colorSpace; + + processedCfg->setStream(&data_->frames_.outputStream_); + + ispOutputFormat_ = processedCfg->pixelFormat; + } + + return status; +} + +/* ----------------------------------------------------------------------------- + * Pipeline Handler + */ + +class PipelineHandlerRCar4 final : public PipelineHandler +{ +public: + PipelineHandlerRCar4(CameraManager *manager); + + std::unique_ptr generateConfiguration(Camera *camera, + std::span roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + RCar4CameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + int createCamera(const MediaDevice *mdev, const std::string &pipeId); +}; + +PipelineHandlerRCar4::PipelineHandlerRCar4(CameraManager *manager) + : PipelineHandler(manager, kMaxRequests) +{ +} + +std::unique_ptr +PipelineHandlerRCar4::generateConfiguration(Camera *camera, + std::span roles) +{ + RCar4CameraData *data = cameraData(camera); + auto config = std::make_unique(data); + + if (roles.empty()) + return config; + + for (const StreamRole role : roles) { + std::map> formats; + PixelFormat pixelFormat; + + switch (role) { + case StreamRole::Raw: + for (const auto &[mbusCode, sizes] : data->rawFormats_) { + auto pf = BayerFormat::fromMbusCode(mbusCode).toPixelFormat(); + ASSERT(pf.isValid()); + formats.try_emplace(pf, sizes.begin(), sizes.end()); + } + + pixelFormat = formats.begin()->first; + break; + default: { + for (const auto &[pf, sizes] : data->outputFormats_) + formats.try_emplace(pf, sizes.begin(), sizes.end()); + + pixelFormat = formats.begin()->first; + break; + } + } + + ASSERT(!formats.empty()); + StreamConfiguration cfg = { std::move(formats) }; + + cfg.pixelFormat = pixelFormat; + cfg.size = { -1u, -1u }; + + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) + return {}; + + return config; +} + +int PipelineHandlerRCar4::configure(Camera *camera, CameraConfiguration *c) +{ + RCar4CameraConfiguration *config = static_cast(c); + RCar4CameraData *data = cameraData(camera); + V4L2DeviceFormat vinFormat; + + /* Configure VIN and propagate format to ISP. */ + int ret = data->vin_.configure(config->sensorFormat(), + config->combinedTransform(), &vinFormat); + if (ret) + return ret; + + ret = data->isp_.configure(vinFormat, config->ispOutputFormat()); + if (ret) + return ret; + + /* Inform IPA of stream configuration and sensor controls. */ + IPACameraSensorInfo sensorInfo; + ret = data->vin_.sensor()->sensorInfo(&sensorInfo); + if (ret) + return ret; + + ipa::rppx1::IPAConfigInfo ipaConfig{ + std::move(sensorInfo), + data->vin_.sensor()->controls(), + }; + + ret = data->ipa_->configure(std::move(ipaConfig), &data->controlInfo_); + if (ret) { + LOG(RCar4, Error) << "failed configuring IPA (" << ret << ")"; + return ret; + } + + return 0; +} + +int PipelineHandlerRCar4::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + RCar4CameraData *data = cameraData(camera); + unsigned int count = stream->configuration().bufferCount; + + if (stream == &data->frames_.outputStream_) + return data->isp_.output_->exportBuffers(count, buffers); + + if (stream == &data->frames_.rawStream_) + return data->isp_.input_->exportBuffers(count, buffers); + + return -EINVAL; +} + +int PipelineHandlerRCar4::start(Camera *camera, + [[maybe_unused]] const ControlList *controls) +{ + utils::scope_exit stopGuard([&] { stop(camera); }); + RCar4CameraData *data = cameraData(camera); + + data->delayedCtrls_->reset(); + + /* \todo Handle `controls`. */ + + int ret = data->frames_.start(&data->isp_, data->ipa_.get(), kMaxRequests); + if (ret) + return ret; + + ret = data->vin_.start(kMaxRequests); + if (ret) + return ret; + + ret = data->isp_.start(kMaxRequests); + if (ret) + return ret; + + ret = data->ipa_->start(); + if (ret) + return ret; + + stopGuard.release(); + return 0; +} + +void PipelineHandlerRCar4::stopDevice(Camera *camera) +{ + RCar4CameraData *data = cameraData(camera); + + data->ipa_->stop(); + data->isp_.stop(); + data->vin_.stop(); + + data->frames_.stop(&data->isp_, data->ipa_.get()); +} + +int PipelineHandlerRCar4::queueRequestDevice(Camera *camera, Request *request) +{ + RCar4CameraData *data = cameraData(camera); + + RCar4Frames::Info *info = data->frames_.create(request); + + /* Always expected to have buffers for `kMaxRequests` in-flight requests. */ + ASSERT(info); + + int ret = data->vin_.queueBuffer(info->inputBuffer); + if (ret) { + data->frames_.remove(info); + return ret; + } + + data->ipa_->queueRequest(info->frame, request->controls()); + + return 0; +} + +int PipelineHandlerRCar4::createCamera(const MediaDevice *mdev, + const std::string &pipeId) +{ + auto data = std::make_unique(this); + + int ret = data->init(mdev, pipeId); + if (ret) + return ret; + + const std::string &id = data->vin_.sensor()->id(); + std::set streams{ + &data->frames_.rawStream_, + &data->frames_.outputStream_, + }; + + registerCamera(Camera::create(std::move(data), id, streams)); + + return 0; +} + +bool PipelineHandlerRCar4::match(DeviceEnumerator *enumerator) +{ + DeviceMatch dm("rcar_vin"); + + auto media = acquireMediaDevice(enumerator, dm); + if (!media) + return false; + + bool registered = false; + for (const MediaEntity *entity : media->entities()) { + if (!entity->name().starts_with("rcar_isp")) + continue; + if (entity->name().rfind("core") == std::string::npos) + continue; + + /* + * Isolate the unit address that identifies one ISP + * instance. pipeId will look like + * 'rcar_isp fed00000.isp'. + */ + constexpr size_t prefix = + std::string_view("rcar_isp fed00000.isp").length(); + + std::string pipeId = entity->name().substr(0, prefix); + if (!createCamera(media.get(), pipeId)) + registered = true; + } + + return registered; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerRCar4, "rcar-gen4") + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/vin.cpp b/src/libcamera/pipeline/rcar-gen4/vin.cpp new file mode 100644 index 0000000000..1dc883412a --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/vin.cpp @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#include "vin.h" + +#include + +#include + +#include +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_subdevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RCar4) + +int RCarVINDevice::init(const MediaDevice *media, const std::string &pipeId) +{ + const MediaEntity *entity; + const MediaPad *pad, *next; + + /* + * The current kernel driver connects each sensor to a particular VIN + * device, so that one has to be used, even though technically any + * VIN device could be used if proper link handling is implemented. + */ + + /* Locate IPS Channel Selector, e.g. rcar_isp fed00000.isp */ + csisp_ = V4L2Subdevice::fromEntityName(media, pipeId); + if (!csisp_) { + LOG(RCar4, Error) << "Failed to find Channel Selector " << pipeId; + return -EINVAL; + } + + /* Use the Channel Selector links to find CSI-2 Rx and Sensor. */ + entity = csisp_->entity(); + pad = entity->getPadByIndex(0); + next = pad->links()[0]->source(); + csi2_ = V4L2Subdevice::fromEntityName(media, next->entity()->name()); + if (!csi2_) { + LOG(RCar4, Error) << "Failed to find CSI-2 Rx entity"; + return -EINVAL; + } + + entity = csi2_->entity(); + pad = entity->getPadByIndex(0); + next = pad->links()[0]->source(); + sensor_ = CameraSensorFactoryBase::create(next->entity()); + if (!sensor_) { + LOG(RCar4, Error) << "Failed to find sensor entity"; + return -EINVAL; + } + + /* + * Use the Channel Selector links to find VIN. Currently the first + * VIN device is used unconditionally. + */ + entity = csisp_->entity(); + pad = entity->getPadByIndex(1); + next = pad->links()[0]->sink(); + output_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!output_) { + LOG(RCar4, Error) << "Failed to find VIN entity"; + return -EINVAL; + } + + /* Open all devices. */ + int ret = csi2_->open(); + if (ret) + return ret; + + ret = csisp_->open(); + if (ret) + return ret; + + ret = output_->open(); + if (ret) + return ret; + + return 0; +} + +int RCarVINDevice::configure(const V4L2SubdeviceFormat &format, Transform transform, + V4L2DeviceFormat *outputFormat) +{ + auto sensorFormat = format; + + /* Configure sensor */ + int ret = sensor_->setFormat(&sensorFormat, transform); + if (ret) + return ret; + + /* Configure CSI-2 */ + ret = csi2_->setFormat(0, &sensorFormat); + if (ret) + return ret; + + /* Configure Channel selector. */ + ret = csisp_->setFormat(0, &sensorFormat); + if (ret) + return ret; + + auto bayerFormat = BayerFormat::fromMbusCode(sensorFormat.code); + if (!bayerFormat.isValid()) + return -ENOTSUP; + + /* Transform already applied to format by `CameraSensor::setFormat()`. */ + auto v4pf = bayerFormat.toV4L2PixelFormat(); + + /* Configure VIN */ + outputFormat->fourcc = v4pf; + outputFormat->size = sensorFormat.size; + outputFormat->planesCount = 1; + outputFormat->colorSpace = sensorFormat.colorSpace; + + ret = output_->setFormat(outputFormat); + if (ret) + return ret; + + LOG(RCar4, Debug) + << "sensor: " << sensorFormat << ", " + << "VIN: " << *outputFormat; + + if (outputFormat->size != format.size || outputFormat->fourcc != v4pf) + return -EINVAL; + + return 0; +} + +int RCarVINDevice::start(unsigned int bufferCount) +{ + utils::scope_exit stopGuard([&] { stop(); }); + + int ret = output_->importBuffers(bufferCount); + if (ret) { + LOG(RCar4, Error) << "Failed to import VIN buffers"; + return ret; + } + + ret = output_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start VIN"; + return ret; + } + + ret = output_->setFrameStartEnabled(true); + if (ret) { + LOG(RCar4, Error) << "Failed to enable Frame Start"; + return ret; + } + + stopGuard.release(); + return 0; +} + +void RCarVINDevice::stop() +{ + output_->setFrameStartEnabled(false); + + output_->streamOff(); + + if (output_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release VIN buffers"; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/vin.h b/src/libcamera/pipeline/rcar-gen4/vin.h new file mode 100644 index 0000000000..0168b1ddee --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/vin.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#pragma once + +#include +#include + +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CameraSensor; +class FrameBuffer; +class MediaDevice; +enum class Transform; + +class RCarVINDevice +{ +public: + int init(const MediaDevice *media, const std::string &pipeId); + int configure(const V4L2SubdeviceFormat &format, Transform transform, + V4L2DeviceFormat *outputFormat); + + int start(unsigned int bufferCount); + void stop(); + + CameraSensor *sensor() { return sensor_.get(); } + const CameraSensor *sensor() const { return sensor_.get(); } + V4L2VideoDevice *output() { return output_.get(); } + const V4L2VideoDevice *output() const { return output_.get(); } + + int queueBuffer(FrameBuffer *buffer) + { + return output_->queueBuffer(buffer); + } + + Signal &bufferReady() { return output_->bufferReady; } + Signal &frameStart() { return output_->frameStart; } + +private: + std::unique_ptr sensor_; + std::unique_ptr csi2_; + std::unique_ptr csisp_; + std::unique_ptr output_; +}; + +} /* namespace libcamera */ From patchwork Mon Nov 16 17:49:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28925 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09C6DC3304 for ; Mon, 16 Nov 2026 17:49:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A47426A85D; Mon, 16 Nov 2026 18:49:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="f3oYx5Ev"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A9F56A84E for ; Mon, 16 Nov 2026 18:49:30 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D928D802; Mon, 16 Nov 2026 18:47:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851223; bh=Rig98cdPtWK/OpHU2/cKHL1YaK0pQ/Rh5PPg6IKebIc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f3oYx5EvpHBtHgu9A2hztCHrG+16Ij9A+xoVL5GT2THz/xiN2W1/MdGnACZLb9jtU T4gVpMyh+gWLUISGFOk6nz5rm0vp2LU6EEzcCwgIA+f4WBbvIg9RK9NYiJiVBaacVb du9rjVf/DiajUIGeo+/f1GsVJ9weKsGOWcbRtIug= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jai Luthra , Jacopo Mondi Subject: [PATCH v5 13/22] ipa: rppx1: Add IPA module Date: Mon, 16 Nov 2026 18:49:09 +0100 Message-ID: <20261116174919.1056403-14-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jai Luthra Copy the rkisp1 IPA module and remove the unnecessary parts, and adjust it as necessary to create the base for the new rppx1 ipa module. Co-developed-by: Jacopo Mondi Signed-off-by: Jai Luthra Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- meson_options.txt | 2 +- src/ipa/meson.build | 1 + src/ipa/rppx1/algorithms/algorithm.h | 22 +++ src/ipa/rppx1/algorithms/meson.build | 4 + src/ipa/rppx1/data/meson.build | 8 + src/ipa/rppx1/ipa_context.cpp | 37 ++++ src/ipa/rppx1/ipa_context.h | 58 ++++++ src/ipa/rppx1/meson.build | 31 +++ src/ipa/rppx1/module.h | 29 +++ src/ipa/rppx1/params.h | 89 +++++++++ src/ipa/rppx1/rppx1.cpp | 280 +++++++++++++++++++++++++++ src/ipa/rppx1/stats.h | 55 ++++++ 12 files changed, 615 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rppx1/algorithms/algorithm.h create mode 100644 src/ipa/rppx1/algorithms/meson.build create mode 100644 src/ipa/rppx1/data/meson.build create mode 100644 src/ipa/rppx1/ipa_context.cpp create mode 100644 src/ipa/rppx1/ipa_context.h create mode 100644 src/ipa/rppx1/meson.build create mode 100644 src/ipa/rppx1/module.h create mode 100644 src/ipa/rppx1/params.h create mode 100644 src/ipa/rppx1/rppx1.cpp create mode 100644 src/ipa/rppx1/stats.h diff --git a/meson_options.txt b/meson_options.txt index 3c4bc9f8e7..2d98955dcf 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -49,7 +49,7 @@ option('gstreamer', option('ipas', type : 'array', choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/pisp', 'rpi/vc4', - 'softisp', 'vimc'], + 'rppx1', 'softisp', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', diff --git a/src/ipa/meson.build b/src/ipa/meson.build index ab4a1897f5..17ca211957 100644 --- a/src/ipa/meson.build +++ b/src/ipa/meson.build @@ -27,6 +27,7 @@ ipa_sign = files('ipa-sign.sh') supported_ipas = { 'ipu3': 'ipu3', 'mali-c55': 'mali-c55', + 'rcar-gen4': 'rppx1', 'rkisp1': 'rkisp1', 'rpi/pisp': 'rpi/pisp', 'rpi/vc4': 'rpi/vc4', diff --git a/src/ipa/rppx1/algorithms/algorithm.h b/src/ipa/rppx1/algorithms/algorithm.h new file mode 100644 index 0000000000..f5f002a0af --- /dev/null +++ b/src/ipa/rppx1/algorithms/algorithm.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 control algorithm interface + */ + +#pragma once + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::rppx1 { + +using Algorithm = libcamera::ipa::Algorithm; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build new file mode 100644 index 0000000000..5fed9e9d50 --- /dev/null +++ b/src/ipa/rppx1/algorithms/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +rppx1_ipa_algorithms = files([ +]) diff --git a/src/ipa/rppx1/data/meson.build b/src/ipa/rppx1/data/meson.build new file mode 100644 index 0000000000..d1eb321758 --- /dev/null +++ b/src/ipa/rppx1/data/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'rppx1', + install_tag : 'runtime') diff --git a/src/ipa/rppx1/ipa_context.cpp b/src/ipa/rppx1/ipa_context.cpp new file mode 100644 index 0000000000..2cd435d980 --- /dev/null +++ b/src/ipa/rppx1/ipa_context.cpp @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::rppx1 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + */ + +/** + * \struct IPAActiveState + * \brief Active state for algorithms + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + */ + +} /* namespace libcamera::ipa::rppx1 */ diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h new file mode 100644 index 0000000000..079fcab782 --- /dev/null +++ b/src/ipa/rppx1/ipa_context.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 IPA Context + */ + +#pragma once + +#include + +#include + +#include +#include +#include + +#include + +#include +#include + +namespace libcamera { + +namespace ipa::rppx1 { + +struct IPASessionConfiguration { +}; + +struct IPAActiveState { +}; + +struct IPAFrameContext : public FrameContext { +}; + +struct IPAContext { + IPAContext(unsigned int frameContextSize) + : frameContexts(frameContextSize) + { + } + + IPACameraSensorInfo sensorInfo; + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; + + ControlInfoMap sensorControls; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; +}; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rppx1/meson.build b/src/ipa/rppx1/meson.build new file mode 100644 index 0000000000..98499c3f78 --- /dev/null +++ b/src/ipa/rppx1/meson.build @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('algorithms') +subdir('data') + +ipa_name = 'ipa_rppx1' + +rppx1_ipa_sources = files([ + 'ipa_context.cpp', + 'rppx1.cpp', +]) + +rppx1_ipa_sources += rppx1_ipa_algorithms + +mod = shared_module(ipa_name, rppx1_ipa_sources, + name_prefix : '', + include_directories : [ipa_includes], + dependencies : [libcamera_private, libipa_dep], + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/rppx1/module.h b/src/ipa/rppx1/module.h new file mode 100644 index 0000000000..04902eae10 --- /dev/null +++ b/src/ipa/rppx1/module.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 IPA Module + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" +#include "params.h" +#include "stats.h" + +namespace libcamera { + +namespace ipa::rppx1 { + +using Module = ipa::Module; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h new file mode 100644 index 0000000000..315c8a9785 --- /dev/null +++ b/src/ipa/rppx1/params.h @@ -0,0 +1,89 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 ISP Parameters + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa::rppx1 { + +enum class BlockType : uint16_t { +}; + +namespace details { + +template +struct block_type { +}; + +#define RPPX1_DEFINE_BLOCK_TYPE(blkType, cfgType, id) \ + template<> \ + struct block_type { \ + using type = struct rppx1_##cfgType##_params; \ + static constexpr rppx1_params_block_type blockType = \ + RPPX1_PARAMS_BLOCK_TYPE_##id; \ + }; + +struct params_traits { + using id_type = BlockType; + + template + using id_to_details = block_type; +}; + +} /* namespace details */ + +template +class RppX1ParamsBlock final : public V4L2ParamsBlock +{ +public: + RppX1ParamsBlock(const std::span data) + : V4L2ParamsBlock(data), + configData_(data.subspan(sizeof(v4l2_isp_block_header))) + { + } + + const T *operator->() const override + { + return reinterpret_cast(configData_.data()); + } + + T *operator->() override + { + return reinterpret_cast(configData_.data()); + } + + const T &operator*() const override + { + return *reinterpret_cast(configData_.data()); + } + + T &operator*() override + { + return *reinterpret_cast(configData_.data()); + } + +private: + std::span configData_; +}; + +class RppX1Params : public V4L2Params +{ +public: + RppX1Params(std::span data) + : V4L2Params(data, V4L2_ISP_PARAMS_VERSION_V1) + { + } +}; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/rppx1.cpp b/src/ipa/rppx1/rppx1.cpp new file mode 100644 index 0000000000..4f1b449398 --- /dev/null +++ b/src/ipa/rppx1/rppx1.cpp @@ -0,0 +1,280 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy + * + * Based on rkisp1.cpp + * Copyright (C) 2019, Google Inc. + * + * RPP-X1 Image Processing Algorithms + */ + +#include +#include +#include +#include + +#include + +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" + +#include "ipa_context.h" +#include "module.h" +#include "params.h" +#include "stats.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPARppX1) + +namespace ipa::rppx1 { + +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPARppX1 final : public IPARppX1Interface, public Module +{ +public: + IPARppX1(); + + int init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + + int configure(const IPAConfigInfo &ipaConfig, ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void computeParams(const uint32_t frame, const uint32_t bufferId) override; + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateControls(ControlInfoMap *ipaControls); + void setControls(unsigned int frame); + + std::map mappedBuffers_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +IPARppX1::IPARppX1() + : context_(kMaxFrameContexts) +{ +} + +std::string IPARppX1::logPrefix() const +{ + return "rppx1"; +} + +int IPARppX1::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + context_.sensorInfo = sensorInfo; + context_.sensorControls = sensorControls; + + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { + LOG(IPARppX1, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + /* Load the tuning data file. */ + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPARppX1, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + unsigned int version = (*data)["version"].get(0); + if (version != 1) { + LOG(IPARppX1, Error) + << "Invalid tuning file version " << version; + return -EINVAL; + } + + if (!data->contains("algorithms")) { + LOG(IPARppX1, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + updateControls(ipaControls); + + return 0; +} + +int IPARppX1::start() +{ + /* \todo Properly handle startup controls. */ + return 0; +} + +void IPARppX1::stop() +{ + context_.frameContexts.clear(); +} + +int IPARppX1::configure(const IPAConfigInfo &ipaConfig, ControlInfoMap *ipaControls) +{ + context_.sensorInfo = ipaConfig.sensorInfo; + context_.sensorControls = ipaConfig.sensorControls; + + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, context_.sensorInfo); + if (ret) + return ret; + } + + updateControls(ipaControls); + + return 0; +} + +void IPARppX1::mapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + FrameBuffer fb(buffer.planes); + + auto [it, inserted] = mappedBuffers_.try_emplace( + buffer.id, &fb, MappedFrameBuffer::MapFlag::ReadWrite); + ASSERT(inserted); + + if (!it->second.isValid()) { + LOG(IPARppX1, Fatal) + << "Failed to mmap buffer: " + << strerror(it->second.error()); + } + } +} + +void IPARppX1::unmapBuffers(const std::vector &ids) +{ + for (unsigned int id : ids) + mappedBuffers_.erase(id); +} + +void IPARppX1::queueRequest(const uint32_t frame, const ControlList &controls) +{ + IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); + + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + algo->queueRequest(context_, frame, frameContext, controls); + } +} + +void IPARppX1::computeParams(const uint32_t frame, const uint32_t bufferId) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + + RppX1Params params(mappedBuffers_.at(bufferId).planes()[0]); + + for (const auto &algo : algorithms()) + algo->prepare(context_, frame, frameContext, ¶ms); + + paramsComputed.emit(frame, params.bytesused()); +} + +void IPARppX1::processStats(const uint32_t frame, const uint32_t bufferId, + [[maybe_unused]] const ControlList &sensorControls) +{ + auto stats = RppX1Stats(mappedBuffers_.at(bufferId).planes()[0]); + if (!stats) + return; + + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + ControlList metadata(controls::controls); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + algo->process(context_, frame, frameContext, &stats, metadata); + } + + setControls(frame); + + metadataReady.emit(frame, metadata); +} + +void IPARppX1::updateControls(ControlInfoMap *ipaControls) +{ + *ipaControls = { + { context_.ctrlMap.begin(), context_.ctrlMap.end() }, + controls::controls, + }; +} + +void IPARppX1::setControls(unsigned int frame) +{ + [[maybe_unused]] IPAFrameContext &frameContext = context_.frameContexts.get(frame); + + ControlList ctrls(context_.sensorControls); + + setSensorControls.emit(frame, ctrls); +} + +} /* namespace ipa::rppx1 */ + +/* + * External IPA module interface + */ + +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "rppx1", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::rppx1::IPARppX1(); +} +} + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/stats.h b/src/ipa/rppx1/stats.h new file mode 100644 index 0000000000..39dc132bac --- /dev/null +++ b/src/ipa/rppx1/stats.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 ISP Statistics + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa::rppx1 { + +enum class StatsType : uint16_t { +}; + +namespace details { + +template +struct stats_type { +}; + +#define RPPX1_DEFINE_STATS_TYPE(blkType, cfgType, id) \ + template<> \ + struct stats_type { \ + using type = struct rppx1_##cfgType##_stats; \ + static constexpr rppx1_stats_block_type blockType = \ + RPPX1_STATS_BLOCK_TYPE_##id; \ + }; + +struct stats_traits { + using id_type = StatsType; + + template + using id_to_details = stats_type; +}; + +} /* namespace details */ + +class RppX1Stats : public V4L2Stats +{ +public: + RppX1Stats(std::span data) + : V4L2Stats(data, V4L2_ISP_VERSION_V1) + { + } +}; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera */ From patchwork Mon Nov 16 17:49:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28927 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09E05C3361 for ; Mon, 16 Nov 2026 17:49:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1D4AC6A889; Mon, 16 Nov 2026 18:49:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TcpjWuT/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 11C686A862 for ; Mon, 16 Nov 2026 18:49:31 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EDD6DC36; Mon, 16 Nov 2026 18:47:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851224; bh=J3vF6Ojjqfjd0rTEoQ/YHRBKnjQiu37Iovdr1UFto6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TcpjWuT/mzRS09u2Ckdpy8wTZkY4YMwMojKKBZc570EictTulTvz77BXzsYWTmFO1 RT/2j9IWnDiGa7smKCU/4EJ4t1TRBt1hZXHH9tIfOr/yOfbASBsVVFhW6VO4tmbM2T /mFYx2w2sd+dPBLl2znn34cnrDf2fy14ZAMQ8CyU= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham Subject: [PATCH v5 14/22] ipa: rppx1: Add black level correction Date: Mon, 16 Nov 2026 18:49:10 +0100 Message-ID: <20261116174919.1056403-15-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/blc.cpp | 92 ++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/blc.h | 39 ++++++++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/params.h | 3 + 4 files changed, 135 insertions(+) create mode 100644 src/ipa/rppx1/algorithms/blc.cpp create mode 100644 src/ipa/rppx1/algorithms/blc.h diff --git a/src/ipa/rppx1/algorithms/blc.cpp b/src/ipa/rppx1/algorithms/blc.cpp new file mode 100644 index 0000000000..e78fede73b --- /dev/null +++ b/src/ipa/rppx1/algorithms/blc.cpp @@ -0,0 +1,92 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 black level correction algorithm + */ + +#include "blc.h" + +#include + +#include + +#include "libcamera/internal/value_node.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +LOG_DEFINE_CATEGORY(RppX1Blc) + +int BlackLevelCorrection::init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) +{ + auto blackLevel = context.camHelper->blackLevel(); + if (!blackLevel) { + LOG(RppX1Blc, Error) + << "No black levels provided by camera sensor helper"; + return -ENOTSUP; + } + + blackLevelRed_ = *blackLevel; + blackLevelGreenR_ = *blackLevel; + blackLevelGreenB_ = *blackLevel; + blackLevelBlue_ = *blackLevel; + + LOG(RppX1Blc, Debug) + << "Black levels: red " << blackLevelRed_ + << ", green (red) " << blackLevelGreenR_ + << ", green (blue) " << blackLevelGreenB_ + << ", blue " << blackLevelBlue_; + + return 0; +} + +void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RppX1Params *params) +{ + if (frame > 0) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->mode = RPPX1_BLS_MODE_FIXED; + config->en_windows = RPPX1_BLS_WIN_EN_OFF; + + /* + * A separate black level subtraction value can be specified + * for each channel. The corresponding color depends on the + * input configuration. + * + * \todo Separate black level for each channel. + * + * Black level values are 16 bits, scale to the 24-bit domain. + */ + + config->fixed.a = static_cast(blackLevelRed_) << 8; + config->fixed.b = static_cast(blackLevelGreenR_) << 8; + config->fixed.c = static_cast(blackLevelGreenB_) << 8; + config->fixed.d = static_cast(blackLevelBlue_) << 8; +} + +void BlackLevelCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const RppX1Stats *stats, + ControlList &metadata) +{ + metadata.set(controls::SensorBlackLevels, + { static_cast(blackLevelRed_), + static_cast(blackLevelGreenR_), + static_cast(blackLevelGreenB_), + static_cast(blackLevelBlue_) }); +} + +REGISTER_IPA_ALGORITHM(BlackLevelCorrection, "BlackLevelCorrection") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/blc.h b/src/ipa/rppx1/algorithms/blc.h new file mode 100644 index 0000000000..91b22b6b97 --- /dev/null +++ b/src/ipa/rppx1/algorithms/blc.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 black level correction algorithm + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class BlackLevelCorrection : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + int16_t blackLevelRed_; + int16_t blackLevelGreenR_; + int16_t blackLevelGreenB_; + int16_t blackLevelBlue_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 5fed9e9d50..165f0ce054 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 rppx1_ipa_algorithms = files([ + 'blc.cpp', ]) diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index 315c8a9785..05db309914 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -16,6 +16,7 @@ namespace libcamera { namespace ipa::rppx1 { enum class BlockType : uint16_t { + BlsPre1, }; namespace details { @@ -32,6 +33,8 @@ struct block_type { RPPX1_PARAMS_BLOCK_TYPE_##id; \ }; +RPPX1_DEFINE_BLOCK_TYPE(BlsPre1, bls, BLS_PRE1) + struct params_traits { using id_type = BlockType; From patchwork Mon Nov 16 17:49:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28928 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 000FBC3355 for ; Mon, 16 Nov 2026 17:49:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 379C76A88C; Mon, 16 Nov 2026 18:49:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bybPkjQk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A83296A863 for ; Mon, 16 Nov 2026 18:49:31 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 565F7802; Mon, 16 Nov 2026 18:47:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851224; bh=TzwwljRmfcgupX0uDSPt+/jqMjyLTWdwvznS3ALuhho=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bybPkjQkizWhzCjNZ0MUL9wWJABR4QA04tdCThtF6Cwv0V/YZD9oiw/FVo0Eh03Wr DGJvRfzivGbfHeNEhJUGRBhGPHDInuD9N5dQ2zvlm7H+daS5iuqTmdcBD4H7HjDUw1 fk14hOd+6JERFSOwu5NkVNVf0xQbo/8erFkHj4Oo= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham Subject: [PATCH v5 15/22] ipa: rppx1: Add auto gain/exposure control Date: Mon, 16 Nov 2026 18:49:11 +0100 Message-ID: <20261116174919.1056403-16-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/agc.cpp | 329 +++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/agc.h | 47 ++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.h | 19 +- src/ipa/rppx1/params.h | 4 + src/ipa/rppx1/rppx1.cpp | 17 +- src/ipa/rppx1/stats.h | 5 + 7 files changed, 418 insertions(+), 4 deletions(-) create mode 100644 src/ipa/rppx1/algorithms/agc.cpp create mode 100644 src/ipa/rppx1/algorithms/agc.h diff --git a/src/ipa/rppx1/algorithms/agc.cpp b/src/ipa/rppx1/algorithms/agc.cpp new file mode 100644 index 0000000000..22a3c7467b --- /dev/null +++ b/src/ipa/rppx1/algorithms/agc.cpp @@ -0,0 +1,329 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 auto exposure/gain algorithm + */ + +#include "agc.h" + +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/fixedpoint.h" +#include "libipa/histogram.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +using WindowWeightQ = UQ<1, 4>; +using MeasureCoeffQ = UQ<1, 7>; + +LOG_DEFINE_CATEGORY(RppX1Agc) + +int Agc::parseMeteringModes(IPAContext &context, const ValueNode &tuningData) +{ + if (!tuningData.isDictionary()) + LOG(RppX1Agc, Warning) + << "'AeMeteringMode' parameter not found in tuning file"; + + for (const auto &[key, value] : tuningData.asDict()) { + auto it = controls::AeMeteringModeNameValueMap.find(key); + if (it == controls::AeMeteringModeNameValueMap.end()) { + LOG(RppX1Agc, Warning) + << "Skipping unknown metering mode '" << key << "'"; + continue; + } + + auto weights = value.get>().value_or(utils::defopt); + if (weights.size() != RPPX1_HIST_WEIGHT_GRIDS_SIZE) { + LOG(RppX1Agc, Error) + << "Invalid 'AeMeteringMode:" << key << "': " + << "expected " << RPPX1_HIST_WEIGHT_GRIDS_SIZE << " elements, " + << "got " << weights.size(); + return -EINVAL; + } + + for (const auto &x : weights) { + if (x < 0 || x > 1) { + LOG(RppX1Agc, Error) + << "Invalid 'AeMeteringMode:" << key << "' values: " + << "elements must be in [0;1]"; + return -EINVAL; + } + } + + meteringModes_.try_emplace(it->second, std::move(weights)); + } + + if (meteringModes_.empty()) { + LOG(RppX1Agc, Warning) + << "No metering modes read from tuning file; defaulting to matrix"; + + meteringModes_.try_emplace( + controls::AeMeteringModeMatrix, + RPPX1_HIST_WEIGHT_GRIDS_SIZE, 1.0f); + } + + std::vector meteringModes; + std::vector meteringModeKeys = utils::map_keys(meteringModes_); + std::transform(meteringModeKeys.begin(), meteringModeKeys.end(), + std::back_inserter(meteringModes), + [](int x) { return ControlValue(x); }); + context.ctrlMap[&controls::AeMeteringMode] = ControlInfo(meteringModes); + + return 0; +} + +int Agc::init(IPAContext &context, const ValueNode &tuningData) +{ + int ret; + + ret = agc_.init(tuningData, context.camHelper.get(), { + .sensorInfo = context.sensorInfo, + .sensorControls = context.sensorControls, + .ctrlMap = context.ctrlMap, + }); + if (ret) + return ret; + + const ValueNode &meteringModes = tuningData["AeMeteringMode"]; + ret = parseMeteringModes(context, meteringModes); + if (ret) + return ret; + + return 0; +} + +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + int ret = agc_.configure(context.configuration.agc, context.activeState.agc, { + .sensorInfo = context.sensorInfo, + .sensorControls = context.sensorControls, + .ctrlMap = context.ctrlMap, + }); + if (ret) + return ret; + + context.activeState.agc.meteringMode = + static_cast(meteringModes_.begin()->first); + + context.configuration.agc.measureWindow.h_offs = 0; + context.configuration.agc.measureWindow.v_offs = 0; + context.configuration.agc.measureWindow.h_size = configInfo.outputSize.width; + context.configuration.agc.measureWindow.v_size = configInfo.outputSize.height; + + return 0; +} + +void Agc::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + agc_.queueRequest(context.configuration.agc, agc, frameContext.agc, controls); + + const auto &meteringMode = controls.get(controls::AeMeteringMode); + if (meteringMode) { + frameContext.agc.updateMetering = agc.meteringMode != *meteringMode; + agc.meteringMode = + static_cast(*meteringMode); + } + frameContext.agc.meteringMode = agc.meteringMode; +} + +void Agc::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RppX1Params *params) +{ + agc_.prepare(context.activeState.agc, frameContext.agc); + + if (frame > 0 && !frameContext.agc.updateMetering) + return; + + auto exmConfig = params->block(); + exmConfig.setEnabled(true); + + exmConfig->wnd = context.configuration.agc.measureWindow; + exmConfig->mode = RPPX1_EXP_MEASURING_MODE_BAYER; + + /* `rppx1_exm_params::last_line` can be programmed but the driver overrides it. */ + exmConfig->channel_sel = RPPX1_MEAS_CHAN_SEL6; + exmConfig->coeff_r = MeasureCoeffQ(1.f).quantized(); + exmConfig->coeff_g_gr = MeasureCoeffQ(1.f).quantized(); + exmConfig->coeff_b = MeasureCoeffQ(1.f).quantized(); + exmConfig->coeff_gb = MeasureCoeffQ(1.f).quantized(); + + auto hst = params->block(); + hst.setEnabled(true); + + /* + * Configure the histogram measurement. Set the window, produce a + * luminance histogram, and set the weights and predivider. + */ + hst->wnd = context.configuration.agc.measureWindow; + + /* + * In RGB mode, a linear combination of the RGB components is used + * for the histogram. Use the ITU-R BT.709 RGB -> Y coefficients. + */ + hst->mode = RPPX1_HIST_MODE_RGB_COMBINED; + hst->channel_sel = RPPX1_MEAS_CHAN_SEL7; + hst->coeff[0] = MeasureCoeffQ(0.2126f).quantized(); + hst->coeff[1] = MeasureCoeffQ(0.7152f).quantized(); + hst->coeff[2] = MeasureCoeffQ(0.0722f).quantized(); + + std::span weights = meteringModes_.at(frameContext.agc.meteringMode); + ASSERT(weights.size() == std::size(hst->weights)); + for (size_t i = 0; i < weights.size(); i++) { + WindowWeightQ w = weights[i]; + hst->weights[i] = w.quantized(); + + LOG(RppX1Agc, Debug) + << "weights[" << i << "]: " << w; + } + + /* + * Each bin has a 16-bit integral component. So the maximum number in it + * is 65535. The worst case is when all pixels are in the same bin. + * + * To ensure no overflow even in that case, one pixel may be sampled out + * of every `skip` pixel. + */ + constexpr uint32_t kBinMax = (1u << 16) - 1; + const double skip = double(hst->wnd.h_size) * hst->wnd.v_size / kBinMax; + + /* + * There is a separate vertical/horizontal step setting, + * but let's try to find something square-ish. + * + * Subsampling by `prediv` vertically and horizontally should + * ensure that no more that `kBinMax` pixels are sampled. + * + * \todo Take weights into account? + */ + const auto prediv = std::clamp(std::ceil(std::sqrt(skip)), 1, 127); + hst->v_stepsize = prediv; + hst->h_step_inc = 65536 / prediv; + + hst->sample_offs = 0; + hst->sample_shift = 0; + + LOG(RppX1Agc, Debug) + << "window: " << Rectangle(hst->wnd.h_offs, hst->wnd.v_offs, hst->wnd.h_size, hst->wnd.v_size) << ", " + << "v_stepsize: " << hst->v_stepsize << ", " + << "h_step_inc: " << hst->h_step_inc; +} + +namespace { + +class AgcTraits final : public AgcMeanLuminance::Traits +{ +public: + AgcTraits(std::span expMeans, std::span weights) + : expMeans_(expMeans), weights_(weights) + { + ASSERT(expMeans_.size() == weights_.size()); + } + + /** + * \brief Estimate the relative luminance of the frame with a given gain + * \param[in] gain The gain to apply to the frame + * + * This function estimates the average relative luminance of the frame that + * would be output by the sensor if an additional \a gain was applied. + * + * The estimation is based on the AE statistics for the current frame. Y + * averages for all cells are first multiplied by the gain, and then saturated + * to approximate the sensor behaviour at high brightness values. The + * approximation is quite rough, as it doesn't take into account non-linearities + * when approaching saturation. + * + * The values are normalized to the [0.0, 1.0] range, where 1.0 corresponds to a + * theoretical perfect reflector of 100% reference white. + * + * More detailed information can be found in: + * https://en.wikipedia.org/wiki/Relative_luminance + * + * \return The relative luminance + */ + double estimateLuminance(double gain) const override + { + constexpr double kMaxExpMean = (1u << 20) - 1; + + double ySum = 0.0; + double wSum = 0.0; + + /* Sum the averages, saturated to the maximum. */ + for (unsigned i = 0; i < expMeans_.size(); i++) { + double w = weights_[i]; + ySum += std::min(expMeans_[i] * gain, kMaxExpMean) * w; + wSum += w; + } + + /* \todo Weight with the AWB gains */ + + return ySum / wSum / kMaxExpMean; + } + +private: + std::span expMeans_; + std::span weights_; +}; + +} /* namespace */ + +void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, const RppX1Stats *stats, + ControlList &metadata) +{ + const auto histPost = stats->block(); + const auto exmPre1 = stats->block(); + + if (!histPost) + LOG(RppX1Agc, Error) << "HIST_POST data is missing in statistics"; + if (!exmPre1) + LOG(RppX1Agc, Error) << "EXM_PRE1 data is missing in statistics"; + + if (histPost && exmPre1) { + static_assert(RPPX1_EXM_NUM_WIN == RPPX1_HIST_WEIGHT_GRIDS_SIZE); + + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {{ + .traits = AgcTraits{ + exmPre1->exp_mean, + meteringModes_.at(frameContext.agc.meteringMode), + }, + .yHist = { + /* The lower 4 bits are fractional and meant to be discarded. */ + histPost->hist_bins, + [](uint32_t x) { return x >> 4; }, + }, + .exposure = frameContext.sensor.exposure, + .gain = frameContext.sensor.gain, + }}, metadata); + } else { + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {}, metadata); + } + + metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/agc.h b/src/ipa/rppx1/algorithms/agc.h new file mode 100644 index 0000000000..03f870955f --- /dev/null +++ b/src/ipa/rppx1/algorithms/agc.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 auto exposure/gain algorithm + */ + +#pragma once + +#include +#include + +#include "libipa/agc.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class Agc : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + int parseMeteringModes(IPAContext &context, const ValueNode &tuningData); + + std::map> meteringModes_; + AgcAlgorithm agc_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 165f0ce054..394ea9510b 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 rppx1_ipa_algorithms = files([ + 'agc.cpp', 'blc.cpp', ]) diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index 079fcab782..f6408e73bb 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -9,14 +9,14 @@ #include -#include +#include #include #include -#include #include +#include #include #include @@ -25,12 +25,27 @@ namespace libcamera { namespace ipa::rppx1 { struct IPASessionConfiguration { + struct Agc : ipa::agc::Session { + rppx1_window measureWindow; + } agc; }; struct IPAActiveState { + struct Agc : ipa::agc::ActiveState { + controls::AeMeteringModeEnum meteringMode; + } agc; }; struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + } sensor; + + struct Agc : ipa::agc::FrameContext { + controls::AeMeteringModeEnum meteringMode; + bool updateMetering; + } agc; }; struct IPAContext { diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index 05db309914..45adf43613 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -17,6 +17,8 @@ namespace ipa::rppx1 { enum class BlockType : uint16_t { BlsPre1, + ExmPre1, + HistPost, }; namespace details { @@ -34,6 +36,8 @@ struct block_type { }; RPPX1_DEFINE_BLOCK_TYPE(BlsPre1, bls, BLS_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) struct params_traits { using id_type = BlockType; diff --git a/src/ipa/rppx1/rppx1.cpp b/src/ipa/rppx1/rppx1.cpp index 4f1b449398..4a494a57e9 100644 --- a/src/ipa/rppx1/rppx1.cpp +++ b/src/ipa/rppx1/rppx1.cpp @@ -30,6 +30,8 @@ #include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/yaml_parser.h" +#include + #include "algorithms/algorithm.h" #include "ipa_context.h" @@ -222,7 +224,7 @@ void IPARppX1::computeParams(const uint32_t frame, const uint32_t bufferId) } void IPARppX1::processStats(const uint32_t frame, const uint32_t bufferId, - [[maybe_unused]] const ControlList &sensorControls) + const ControlList &sensorControls) { auto stats = RppX1Stats(mappedBuffers_.at(bufferId).planes()[0]); if (!stats) @@ -231,6 +233,9 @@ void IPARppX1::processStats(const uint32_t frame, const uint32_t bufferId, IPAFrameContext &frameContext = context_.frameContexts.get(frame); ControlList metadata(controls::controls); + std::tie(frameContext.sensor.exposure, frameContext.sensor.gain) = + agc::extractControls(sensorControls, context_.camHelper.get()); + for (auto const &a : algorithms()) { Algorithm *algo = static_cast(a.get()); algo->process(context_, frame, frameContext, &stats, metadata); @@ -251,9 +256,17 @@ void IPARppX1::updateControls(ControlInfoMap *ipaControls) void IPARppX1::setControls(unsigned int frame) { - [[maybe_unused]] IPAFrameContext &frameContext = context_.frameContexts.get(frame); + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + + uint32_t exposure = frameContext.agc.exposure; + uint32_t vblank = frameContext.agc.vblank; + + LOG(IPARppX1, Debug) + << "Set controls for frame " << frame << ": exposure " << exposure + << ", gain " << frameContext.agc.gain << ", vblank " << vblank; ControlList ctrls(context_.sensorControls); + agc::prepareControls(ctrls, context_.camHelper.get(), frameContext.agc); setSensorControls.emit(frame, ctrls); } diff --git a/src/ipa/rppx1/stats.h b/src/ipa/rppx1/stats.h index 39dc132bac..2925e2bbfc 100644 --- a/src/ipa/rppx1/stats.h +++ b/src/ipa/rppx1/stats.h @@ -16,6 +16,8 @@ namespace libcamera { namespace ipa::rppx1 { enum class StatsType : uint16_t { + ExmPre1, + HistPost, }; namespace details { @@ -32,6 +34,9 @@ struct stats_type { RPPX1_STATS_BLOCK_TYPE_##id; \ }; +RPPX1_DEFINE_STATS_TYPE(ExmPre1, exm, EXM_PRE1) +RPPX1_DEFINE_STATS_TYPE(HistPost, hist, HIST_POST) + struct stats_traits { using id_type = StatsType; From patchwork Mon Nov 16 17:49:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28929 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C5709C3362 for ; Mon, 16 Nov 2026 17:49:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1CBE96A868; Mon, 16 Nov 2026 18:49:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YQxWIj/e"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F3E6D6A86A for ; Mon, 16 Nov 2026 18:49:31 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 00EAE16F9; Mon, 16 Nov 2026 18:47:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851225; bh=Ef6aJbW0kXlLdUEJyhirCT4Z6yNtTTCz0lzfeo7dsiQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YQxWIj/eEoyBaQqoDv37hfyjNjEEFdm0qYlRaBrio0nMoKOxpy8uznEnk25044KCS gapOmcP49U14soOaPu+tnJJnwxdFFtycAGxqYSDd11WTKUvRaxBl+FqkLuWnbuorPS CaoH2aYK0D8s03jX9xolkvvgcz1q5vrZrecQDDnM= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v5 16/22] ipa: rppx1: Add auto white balance Date: Mon, 16 Nov 2026 18:49:12 +0100 Message-ID: <20261116174919.1056403-17-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/awb.cpp | 247 +++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/awb.h | 48 ++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.h | 10 ++ src/ipa/rppx1/params.h | 4 + src/ipa/rppx1/stats.h | 2 + 6 files changed, 312 insertions(+) create mode 100644 src/ipa/rppx1/algorithms/awb.cpp create mode 100644 src/ipa/rppx1/algorithms/awb.h diff --git a/src/ipa/rppx1/algorithms/awb.cpp b/src/ipa/rppx1/algorithms/awb.cpp new file mode 100644 index 0000000000..bd59d13ad5 --- /dev/null +++ b/src/ipa/rppx1/algorithms/awb.cpp @@ -0,0 +1,247 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 auto white balance algorithm + */ + +#include "awb.h" + +#include + +#include + +#include + +#include "libcamera/internal/vector.h" + +#include "libipa/fixedpoint.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +LOG_DEFINE_CATEGORY(RppX1Awb) + +class RppX1AwbStats final : public AwbStats +{ +public: + RppX1AwbStats(const RGB &rgbMeans) + : rgbMeans_(rgbMeans) + { + rg_ = rgbMeans_.r() / rgbMeans_.g(); + bg_ = rgbMeans_.b() / rgbMeans_.g(); + } + + double computeColourError(const RGB &gains) const override + { + /* + * Compute the sum of the squared colour error (non-greyness) as + * it appears in the log likelihood equation. + */ + double deltaR = gains.r() * rg_ - 1.0; + double deltaB = gains.b() * bg_ - 1.0; + double delta2 = deltaR * deltaR + deltaB * deltaB; + + return delta2; + } + + RGB rgbMeans() const override + { + return rgbMeans_; + } + + bool valid() const override + { + /* Minimum mean value below which AWB can't operate. */ + constexpr double minValue = 0.01; + + return rgbMeans_.r() > minValue || rgbMeans_.g() > minValue || + rgbMeans_.b() > minValue; + } + +private: + RGB rgbMeans_; + double rg_; + double bg_; +}; + +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + +int Awb::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + awbAlgo_.configure(context.activeState.awb); + + /* + * Define the measurement window for AWB as a centered rectangle + * covering 3/4 of the image width and height. + */ + context.configuration.awb.measureWindow.h_offs = configInfo.outputSize.width / 8; + context.configuration.awb.measureWindow.v_offs = configInfo.outputSize.height / 8; + context.configuration.awb.measureWindow.h_size = 3 * configInfo.outputSize.width / 4; + context.configuration.awb.measureWindow.v_size = 3 * configInfo.outputSize.height / 4; + + context.configuration.awb.enabled = true; + + return 0; +} + +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + +void Awb::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RppX1Params *params) +{ + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + + auto gain = params->block(); + gain.setEnabled(true); + + gain->gain_green_b = GainQ(static_cast(frameContext.awb.gains.g())).quantized(); + gain->gain_blue = GainQ(static_cast(frameContext.awb.gains.b())).quantized(); + gain->gain_red = GainQ(static_cast(frameContext.awb.gains.r())).quantized(); + gain->gain_green_r = GainQ(static_cast(frameContext.awb.gains.g())).quantized(); + + /* If we have already set the AWB measurement parameters, return. */ + if (frame > 0) + return; + + auto awb = params->block(); + awb.setEnabled(true); + + /* Configure the measure window for AWB. */ + awb->wnd = context.configuration.awb.measureWindow; + + /* Number of frames to use to estimate the means (0 means 1 frame). */ + awb->frames = 0; + + awb->mode = RPPX1_WBMEAS_MODE_YCBCR; + + /* Set the reference Cr and Cb (AWB target) to white. */ + awb->ref_cb_max_b = UQ<0, 24>(0.5f).quantized(); + awb->ref_cr_max_r = UQ<0, 24>(0.5f).quantized(); + + /* + * Filter out pixels based on luminance and chrominance values. + * The acceptable luma values are specified as a [16, 250] + * range, while the acceptable chroma values are specified with + * a minimum of 16 and a maximum Cb+Cr sum of 250. + */ + awb->ymax_cmp = true; + awb->min_y_max_g = UQ<0, 24>(16.f / 255.f).quantized(); + awb->max_y = UQ<0, 24>(250.f / 255.f).quantized(); + + awb->min_c = UQ<0, 24>(16.f / 255.f).quantized(); + awb->max_csum = UQ<0, 24>(250.f / 255.f).quantized(); + + /* + * The suggested coefficients in YCbCr mode are those set forth + * in ITU-R BT.709. + * + * \todo These should be in a common place like colours.{h,cpp}. + * \todo The ordering of the coefficients should be investigated more. + */ + + /* Y coefficients */ + awb->ccor_coeff[0][2] = Q<4, 12>(0.2126f).quantized(); + awb->ccor_coeff[0][0] = Q<4, 12>(0.7152f).quantized(); + awb->ccor_coeff[0][1] = Q<4, 12>(0.0722f).quantized(); + /* Cb coefficients */ + awb->ccor_coeff[1][2] = Q<4, 12>(-0.1146f).quantized(); + awb->ccor_coeff[1][0] = Q<4, 12>(-0.3854f).quantized(); + awb->ccor_coeff[1][1] = Q<4, 12>(0.5f).quantized(); + /* Cr coefficients */ + awb->ccor_coeff[2][2] = Q<4, 12>(0.5f).quantized(); + awb->ccor_coeff[2][0] = Q<4, 12>(-0.4542f).quantized(); + awb->ccor_coeff[2][1] = Q<4, 12>(-0.0458f).quantized(); + /* The recommended offsets are (0, 0.5, 0.5) */ + awb->ccor_offs[0] = Q<1, 24>(0.0).quantized(); + awb->ccor_offs[1] = Q<1, 24>(0.5).quantized(); + awb->ccor_offs[2] = Q<1, 24>(0.5).quantized(); + + LOG(RppX1Awb, Debug) + << "window: " << Rectangle(awb->wnd.h_offs, awb->wnd.v_offs, awb->wnd.h_size, awb->wnd.v_size); +} + +void Awb::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) +{ + const auto awb = stats->block(); + if (!awb) + return; + + LOG(RppX1Awb, Debug) << "measured-pixels: " << awb->cnt; + + if (awb->cnt == 0) + return; + + RppX1AwbStats awbStats = calculateRgbMeans(frameContext, *awb); + + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + 0, metadata); +} + +RppX1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, + const rppx1_wbmeas_stats &awb) const +{ + /* Get the YCbCr mean values */ + Vector yuvMeans({ + UQ<0, 24>(awb.mean_y_or_g).value(), + UQ<0, 24>(awb.mean_cb_or_b).value(), + UQ<0, 24>(awb.mean_cr_or_r).value(), + }); + + LOG(RppX1Awb, Debug) + << "yuv-means: (" + << awb.mean_y_or_g << ',' + << awb.mean_cb_or_b << ',' + << awb.mean_cr_or_r << ')' + << " -> " << yuvMeans; + + /* + * The inverse of the coefficients set in `prepare()`. + */ + static const Matrix rec709rgb = { { + 1.0, 0.0, 1.5748, + 1.0, -0.1873, -0.4681, + 1.0, 1.8556, 0.0, + } }; + static const Vector offsets = { { 0.0, 0.5, 0.5 } }; + + auto rgbMeans = rec709rgb * (yuvMeans - offsets); + + /* + * Due to hardware rounding errors in the YCbCr means, the + * calculated RGB means may be negative. This would lead to + * negative gains, messing up calculation. Prevent this by + * clamping the means to positive values. + */ + rgbMeans = rgbMeans.max(0.0); + + /* + * The ISP computes the AWB means after applying the colour gains, + * divide by the gains that were used to get the raw means from the + * sensor. Apply a minimum value to avoid divisions by near-zero. + */ + rgbMeans /= frameContext.awb.gains.max(0.01); + + return RppX1AwbStats(rgbMeans); +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/awb.h b/src/ipa/rppx1/algorithms/awb.h new file mode 100644 index 0000000000..e66c6684be --- /dev/null +++ b/src/ipa/rppx1/algorithms/awb.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 auto white balance algorithm + */ + +#pragma once + +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class RppX1AwbStats; + +class Awb : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + using GainQ = UQ<6, 12>; + + RppX1AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const rppx1_wbmeas_stats &awb) const; + + AwbAlgorithm awbAlgo_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 394ea9510b..4a2c1b52dd 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -2,5 +2,6 @@ rppx1_ipa_algorithms = files([ 'agc.cpp', + 'awb.cpp', 'blc.cpp', ]) diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index f6408e73bb..24fbbfaa28 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -17,6 +17,7 @@ #include #include +#include #include #include @@ -28,12 +29,19 @@ struct IPASessionConfiguration { struct Agc : ipa::agc::Session { rppx1_window measureWindow; } agc; + + struct { + struct rppx1_window measureWindow; + bool enabled; + } awb; }; struct IPAActiveState { struct Agc : ipa::agc::ActiveState { controls::AeMeteringModeEnum meteringMode; } agc; + + ipa::awb::ActiveState awb; }; struct IPAFrameContext : public FrameContext { @@ -46,6 +54,8 @@ struct IPAFrameContext : public FrameContext { controls::AeMeteringModeEnum meteringMode; bool updateMetering; } agc; + + ipa::awb::FrameContext awb; }; struct IPAContext { diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index 45adf43613..478d443a33 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -16,9 +16,11 @@ namespace libcamera { namespace ipa::rppx1 { enum class BlockType : uint16_t { + AwbGPre1, BlsPre1, ExmPre1, HistPost, + WbMeasPost, }; namespace details { @@ -35,9 +37,11 @@ struct block_type { RPPX1_PARAMS_BLOCK_TYPE_##id; \ }; +RPPX1_DEFINE_BLOCK_TYPE(AwbGPre1, awbg, AWBG_PRE1) RPPX1_DEFINE_BLOCK_TYPE(BlsPre1, bls, BLS_PRE1) RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) +RPPX1_DEFINE_BLOCK_TYPE(WbMeasPost, wbmeas, WBMEAS_POST) struct params_traits { using id_type = BlockType; diff --git a/src/ipa/rppx1/stats.h b/src/ipa/rppx1/stats.h index 2925e2bbfc..2c2a6e3778 100644 --- a/src/ipa/rppx1/stats.h +++ b/src/ipa/rppx1/stats.h @@ -18,6 +18,7 @@ namespace ipa::rppx1 { enum class StatsType : uint16_t { ExmPre1, HistPost, + WbMeasPost, }; namespace details { @@ -36,6 +37,7 @@ struct stats_type { RPPX1_DEFINE_STATS_TYPE(ExmPre1, exm, EXM_PRE1) RPPX1_DEFINE_STATS_TYPE(HistPost, hist, HIST_POST) +RPPX1_DEFINE_STATS_TYPE(WbMeasPost, wbmeas, WBMEAS_POST) struct stats_traits { using id_type = StatsType; From patchwork Mon Nov 16 17:49:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28930 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D41FDC3304 for ; Mon, 16 Nov 2026 17:49:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 09C896A895; Mon, 16 Nov 2026 18:49:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Wa4WgG8D"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B9F266A86C for ; Mon, 16 Nov 2026 18:49:32 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4AD70929; Mon, 16 Nov 2026 18:47:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851225; bh=Qi7BCR8n0FqKPanrIyfQBtRXlWDW67krGs9l30y6eQg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wa4WgG8Dxi/uTInnumbvcF+G8HUz86i6wcd9UbhhSs+8L3V+Bm28ew5c6mRkAk11B Y0POgP7la0Ly/LZ7xMJl04oGd7zBblhjKAKUoqeRKdCPjtb9rh2P5RQxbpL1LRdjJh ZIZIZ0dltFH/T/I+QrN5brMrI+gsW/zoI+8u7NZ8= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v5 17/22] ipa: rppx1: Add lux estimation Date: Mon, 16 Nov 2026 18:49:13 +0100 Message-ID: <20261116174919.1056403-18-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/agc.cpp | 1 + src/ipa/rppx1/algorithms/awb.cpp | 2 +- src/ipa/rppx1/algorithms/lux.cpp | 63 ++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/lux.h | 36 ++++++++++++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.h | 5 +++ 6 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rppx1/algorithms/lux.cpp create mode 100644 src/ipa/rppx1/algorithms/lux.h diff --git a/src/ipa/rppx1/algorithms/agc.cpp b/src/ipa/rppx1/algorithms/agc.cpp index 22a3c7467b..38ea4de9ca 100644 --- a/src/ipa/rppx1/algorithms/agc.cpp +++ b/src/ipa/rppx1/algorithms/agc.cpp @@ -314,6 +314,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, }, .exposure = frameContext.sensor.exposure, .gain = frameContext.sensor.gain, + .lux = frameContext.lux.lux, }}, metadata); } else { agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {}, metadata); diff --git a/src/ipa/rppx1/algorithms/awb.cpp b/src/ipa/rppx1/algorithms/awb.cpp index bd59d13ad5..7c748e9a2c 100644 --- a/src/ipa/rppx1/algorithms/awb.cpp +++ b/src/ipa/rppx1/algorithms/awb.cpp @@ -190,7 +190,7 @@ void Awb::process(IPAContext &context, RppX1AwbStats awbStats = calculateRgbMeans(frameContext, *awb); awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, - 0, metadata); + frameContext.lux.lux, metadata); } RppX1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, diff --git a/src/ipa/rppx1/algorithms/lux.cpp b/src/ipa/rppx1/algorithms/lux.cpp new file mode 100644 index 0000000000..56275d1973 --- /dev/null +++ b/src/ipa/rppx1/algorithms/lux.cpp @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 lux estimation algorithm + */ + +#include "lux.h" + +#include + +#include + +#include "libipa/histogram.h" +#include "libipa/lux.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +int Lux::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) +{ + return lux_.init(tuningData); +} + +void Lux::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] RppX1Params *params) +{ + lux_.prepare(context.activeState.lux, frameContext.lux); +} + +void Lux::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) +{ + const auto histPost = stats->block(); + if (!histPost) { + lux_.process(context.activeState.lux, frameContext.lux, metadata, {}); + return; + } + + lux_.process(context.activeState.lux, frameContext.lux, metadata, + { { + .exposureTime = context.configuration.agc.lineDuration * + frameContext.sensor.exposure, + .analogueGain = frameContext.sensor.gain, + .digitalGain = 1, + /* \todo Deduplicate the histogram calculation from AGC */ + .yHist = { + histPost->hist_bins, + [](uint32_t x) { return x >> 4; }, + }, + } }); +} + +REGISTER_IPA_ALGORITHM(Lux, "Lux") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/lux.h b/src/ipa/rppx1/algorithms/lux.h new file mode 100644 index 0000000000..6c94f7507a --- /dev/null +++ b/src/ipa/rppx1/algorithms/lux.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 lux estimation algorithm + */ + +#pragma once + +#include "libipa/lux.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class Lux : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + ipa::LuxAlgorithm lux_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 4a2c1b52dd..19688854a3 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -4,4 +4,5 @@ rppx1_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'lux.cpp', ]) diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index 24fbbfaa28..baa82d5f3b 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -20,6 +20,7 @@ #include #include #include +#include namespace libcamera { @@ -42,6 +43,8 @@ struct IPAActiveState { } agc; ipa::awb::ActiveState awb; + + ipa::lux::ActiveState lux; }; struct IPAFrameContext : public FrameContext { @@ -56,6 +59,8 @@ struct IPAFrameContext : public FrameContext { } agc; ipa::awb::FrameContext awb; + + ipa::lux::FrameContext lux; }; struct IPAContext { From patchwork Mon Nov 16 17:49:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28931 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B126EC3360 for ; Mon, 16 Nov 2026 17:49:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 91E096A86D; Mon, 16 Nov 2026 18:49:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WiQ+N6g2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE50E6A848 for ; Mon, 16 Nov 2026 18:49:32 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E57751D15; Mon, 16 Nov 2026 18:47:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851226; bh=wz+19Yyxn928uPK62JR8PkctjWyA5wzwJM8VuxezGdE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WiQ+N6g2P5f5ASR5568lptWr6wDqXFiWhpCvxxpGoMoxVkPNGRMQqwiDLcNCOa0Xy T76RlJEOSR9MkQ8o6LZrSPbiQsljNKhj9UvCGmPAJW0wuZ/daacXqN2eSjqGtE8xqE Q+mKLufRtRlAlFu7oMAVDhPQzR/pyW4BkgjaKtJE= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham Subject: [PATCH v5 18/22] ipa: rppx1: Add color correction Date: Mon, 16 Nov 2026 18:49:14 +0100 Message-ID: <20261116174919.1056403-19-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/awb.cpp | 6 ++ src/ipa/rppx1/algorithms/ccm.cpp | 91 ++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/ccm.h | 43 +++++++++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.h | 5 ++ src/ipa/rppx1/params.h | 2 + 6 files changed, 148 insertions(+) create mode 100644 src/ipa/rppx1/algorithms/ccm.cpp create mode 100644 src/ipa/rppx1/algorithms/ccm.h diff --git a/src/ipa/rppx1/algorithms/awb.cpp b/src/ipa/rppx1/algorithms/awb.cpp index 7c748e9a2c..ffe8e42a6d 100644 --- a/src/ipa/rppx1/algorithms/awb.cpp +++ b/src/ipa/rppx1/algorithms/awb.cpp @@ -230,6 +230,12 @@ RppX1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, */ rgbMeans = rgbMeans.max(0.0); + /* + * The ISP computes the AWB means after applying the CCM. Apply the + * inverse as we want to get the raw means before the colour gains. + */ + rgbMeans = frameContext.ccm.ccm.inverse() * rgbMeans; + /* * The ISP computes the AWB means after applying the colour gains, * divide by the gains that were used to get the raw means from the diff --git a/src/ipa/rppx1/algorithms/ccm.cpp b/src/ipa/rppx1/algorithms/ccm.cpp new file mode 100644 index 0000000000..1686891c92 --- /dev/null +++ b/src/ipa/rppx1/algorithms/ccm.cpp @@ -0,0 +1,91 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 color correction algorithm + */ + +#include "ccm.h" + +#include + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +LOG_DEFINE_CATEGORY(RppX1Ccm) + +int Ccm::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.colourTemperature); +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* Nothing to do here, the ccm will be calculated in prepare() */ + if (frameContext.awb.autoEnabled) + return; + + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RppX1Params *params) +{ + if (frameContext.awb.autoEnabled) + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, + frame, frameContext.awb.colourTemperature); + + auto config = params->block(); + config.setEnabled(true); + + const Matrix &matrix = frameContext.ccm.ccm; + const Matrix &offsets = frameContext.ccm.offsets; + + /* + * Coefficients are 16-bit Q4.12 signed numbers ranging from + * -8 (0x8000) to +7.9996 (0x7fff). + */ + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + config->coeff[i][j] = Q<4, 12>(matrix[i][j]).quantized(); + } + + /* + * Offsets are 25 bits 2's complement. + * + * \todo: Better investigate how negative offsets are handled in the + * offsets interpolation. + */ + + for (unsigned int i = 0; i < 3; i++) + config->offset[i] = offsets[i][0]; + + LOG(RppX1Ccm, Debug) << "Setting matrix " << matrix; + LOG(RppX1Ccm, Debug) << "Setting offsets " << offsets; +} + +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RppX1Stats *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/ccm.h b/src/ipa/rppx1/algorithms/ccm.h new file mode 100644 index 0000000000..684285a368 --- /dev/null +++ b/src/ipa/rppx1/algorithms/ccm.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 color correction algorithm + */ + +#pragma once + +#include "libipa/ccm.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class Ccm : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 19688854a3..7e0b4ad011 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -4,5 +4,6 @@ rppx1_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'ccm.cpp', 'lux.cpp', ]) diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index baa82d5f3b..1f0b24635a 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -44,6 +45,8 @@ struct IPAActiveState { ipa::awb::ActiveState awb; + ipa::ccm::ActiveState ccm; + ipa::lux::ActiveState lux; }; @@ -60,6 +63,8 @@ struct IPAFrameContext : public FrameContext { ipa::awb::FrameContext awb; + ipa::ccm::FrameContext ccm; + ipa::lux::FrameContext lux; }; diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index 478d443a33..b863b9ba19 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -18,6 +18,7 @@ namespace ipa::rppx1 { enum class BlockType : uint16_t { AwbGPre1, BlsPre1, + CcorPost, ExmPre1, HistPost, WbMeasPost, @@ -39,6 +40,7 @@ struct block_type { RPPX1_DEFINE_BLOCK_TYPE(AwbGPre1, awbg, AWBG_PRE1) RPPX1_DEFINE_BLOCK_TYPE(BlsPre1, bls, BLS_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(CcorPost, ccor, CCOR_POST) RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) RPPX1_DEFINE_BLOCK_TYPE(WbMeasPost, wbmeas, WBMEAS_POST) From patchwork Mon Nov 16 17:49:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28933 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 64772C3363 for ; Mon, 16 Nov 2026 17:49:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BE4526A892; Mon, 16 Nov 2026 18:49:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="a/TRA6JI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AE6F6A86D for ; Mon, 16 Nov 2026 18:49:33 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3EE9C16F9; Mon, 16 Nov 2026 18:47:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851226; bh=IypnRHYlqfsOMq44H6EKZkQaWSGJkHp1Swa7ZBxw3dQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a/TRA6JI83h2FzaEQdxNp+0fi/PalWgJC7EcZIpQYsoJ/LS8VWl50P87mmz7dLNoU sLGhJeEc5+nhtpxEP8RPy2OwqueG5ohULuR+H51aojDPcZZN/2pxuW1ZCtjkRD/9jI hhT8hHX3/wORTGjuKsQ5mSzKPJHEkEMY39GmGljU= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham Subject: [PATCH v5 19/22] ipa: rppx1: Add gamma correction Date: Mon, 16 Nov 2026 18:49:15 +0100 Message-ID: <20261116174919.1056403-20-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/gamma.cpp | 71 ++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/gamma.h | 46 ++++++++++++++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.h | 5 ++ src/ipa/rppx1/params.h | 2 + 5 files changed, 125 insertions(+) create mode 100644 src/ipa/rppx1/algorithms/gamma.cpp create mode 100644 src/ipa/rppx1/algorithms/gamma.h diff --git a/src/ipa/rppx1/algorithms/gamma.cpp b/src/ipa/rppx1/algorithms/gamma.cpp new file mode 100644 index 0000000000..53ccae4c88 --- /dev/null +++ b/src/ipa/rppx1/algorithms/gamma.cpp @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 gamma correction algorithm + */ + +#include "gamma.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +int Gamma::init(IPAContext &context, const ValueNode &tuningData) +{ + /* + * The segments as specified in the reference for + * RPPX1_GA_SEG_MODE_LOGARITHMIC. + */ + static constexpr unsigned int segments[] = { + 64, 64, 64, 64, 128, 128, 128, 128, + 256, 256, 256, 512, 512, 512, 512, 512, + }; + + return gamma_.init(context.ctrlMap, tuningData, segments); +} + +int Gamma::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + gamma_.configure(context.activeState.gamma); + return 0; +} + +void Gamma::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + gamma_.queueRequest(context.activeState.gamma, frame, + frameContext.gamma, controls); +} + +void Gamma::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) +{ + if (!frameContext.gamma.update) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->mode = RPPX1_GA_SEG_MODE_LOGARITHMIC; + gamma_.prepare(frameContext.gamma, std::span(config->gamma_y)); +} + +void Gamma::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RppX1Stats *stats, + ControlList &metadata) +{ + metadata.set(controls::Gamma, frameContext.gamma.gamma); +} + +REGISTER_IPA_ALGORITHM(Gamma, "Gamma") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/gamma.h b/src/ipa/rppx1/algorithms/gamma.h new file mode 100644 index 0000000000..8fcb4429ba --- /dev/null +++ b/src/ipa/rppx1/algorithms/gamma.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 gamma correction algorithm + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class Gamma : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + /* + * for RPP_OUT ("human vision") pipeline with 12-bit gamma values; + * in contrast, RPP_MV ("machine vision") has 24-bit gamma values + */ + GammaAlgorithm> gamma_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 7e0b4ad011..7394b55bca 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -5,5 +5,6 @@ rppx1_ipa_algorithms = files([ 'awb.cpp', 'blc.cpp', 'ccm.cpp', + 'gamma.cpp', 'lux.cpp', ]) diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index 1f0b24635a..d965311ca1 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -21,6 +21,7 @@ #include #include #include +#include #include namespace libcamera { @@ -47,6 +48,8 @@ struct IPAActiveState { ipa::ccm::ActiveState ccm; + ipa::gamma::ActiveState gamma; + ipa::lux::ActiveState lux; }; @@ -65,6 +68,8 @@ struct IPAFrameContext : public FrameContext { ipa::ccm::FrameContext ccm; + ipa::gamma::FrameContext gamma; + ipa::lux::FrameContext lux; }; diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index b863b9ba19..b724fa3162 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -20,6 +20,7 @@ enum class BlockType : uint16_t { BlsPre1, CcorPost, ExmPre1, + GaHv, HistPost, WbMeasPost, }; @@ -42,6 +43,7 @@ RPPX1_DEFINE_BLOCK_TYPE(AwbGPre1, awbg, AWBG_PRE1) RPPX1_DEFINE_BLOCK_TYPE(BlsPre1, bls, BLS_PRE1) RPPX1_DEFINE_BLOCK_TYPE(CcorPost, ccor, CCOR_POST) RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(GaHv, ga, GA_HV) RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) RPPX1_DEFINE_BLOCK_TYPE(WbMeasPost, wbmeas, WBMEAS_POST) From patchwork Mon Nov 16 17:49:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28932 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77DDCC3361 for ; Mon, 16 Nov 2026 17:49:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2DF2B6A87F; Mon, 16 Nov 2026 18:49:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oLcSiMtd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8046B6A85D for ; Mon, 16 Nov 2026 18:49:33 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 84E701E6A; Mon, 16 Nov 2026 18:47:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851226; bh=sjcl8FXciedRkSUt40Tk9tfML8Z/hVjqHAe6reexUpw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oLcSiMtdm2OckWqyX48WqcVNy3Cg6i9YlJZBnAl2H51HsSFrFI1Q0HToJ9LHUfjM7 6cjVm4L8xpsT08kSpAOiX9SQdtpCI2xXsnUI3S28ElsXbqznomWogiTfv5n6X97CTp CFW+KFjrc+sUx1pdK1BDZ1TzPkUCDv8+kyEt+O9w= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v5 20/22] ipa: rppx1: Add input linearization Date: Mon, 16 Nov 2026 18:49:16 +0100 Message-ID: <20261116174919.1056403-21-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze --- src/ipa/rppx1/algorithms/gsl.cpp | 142 +++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/gsl.h | 35 +++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/params.h | 2 + 4 files changed, 180 insertions(+) create mode 100644 src/ipa/rppx1/algorithms/gsl.cpp create mode 100644 src/ipa/rppx1/algorithms/gsl.h diff --git a/src/ipa/rppx1/algorithms/gsl.cpp b/src/ipa/rppx1/algorithms/gsl.cpp new file mode 100644 index 0000000000..a409a3bc14 --- /dev/null +++ b/src/ipa/rppx1/algorithms/gsl.cpp @@ -0,0 +1,142 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 input linearization algorithm + */ + +#include "gsl.h" + +#include +#include +#include +#include +#include + +#include +#include + +#include "libcamera/internal/value_node.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +LOG_DEFINE_CATEGORY(RppX1Gsl) + +/* `LinPre1` only */ +constexpr uint32_t kDxOffset = 8; +constexpr uint32_t kMaxDx = (1u << 4) - 1; +constexpr uint32_t kResolution = 1u << 24; + +namespace { + +[[nodiscard]] +bool parseYLut(const ValueNode &yObject, std::string_view key, uint32_t scale, + std::span res) +{ + auto vals = yObject[key].get>().value_or(utils::defopt); + if (vals.size() != res.size()) { + LOG(RppX1Gsl, Error) + << "Invalid 'y:" << key << "': expected " + << res.size() << " elements, got " << vals.size(); + return false; + } + + for (const auto &[i, y] : utils::enumerate(vals)) { + if (y < 0 || y > 1) { + LOG(RppX1Gsl, Error) + << "Invalid 'y:" << key << "': elements must be in [0; 1]"; + return false; + } + + res[i] = std::lround(y * scale); + } + + return true; +} + +} /* namespace */ + +int GammaSensorLinearization::init([[maybe_unused]] IPAContext &context, + const ValueNode &tuningData) +{ + /* + * \todo `RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2` has different ranges: + * * kDxOffset = 4 + * * kMaxDx = 7 + * * kResolution = 2^12 + * Maybe this parameter could be reworked to be compatible with both. + */ + gammaDx_ = tuningData["x-intervals"].get>().value_or(utils::defopt); + if (gammaDx_.size() != RPPX1_LIN_SAMPLE_POINTS_NUM) { + LOG(RppX1Gsl, Error) + << "Invalid 'x-intervals': expected " + << RPPX1_LIN_SAMPLE_POINTS_NUM << " elements, got " + << gammaDx_.size(); + + return -EINVAL; + } + + uint32_t xSum = 0; + for (const uint8_t x : gammaDx_) { + if (x > kMaxDx) { + LOG(RppX1Gsl, Error) + << "Invalid 'x-intervals': must be at most " << kMaxDx + << ", got " << x; + return -EINVAL; + } + + xSum += 1u << (x + kDxOffset); + } + + /* + * From the documentation: + * + * Typically, the accumulated sum of dx[i] should cover the complete data + * input range of [0; 2^12) or [0; 2^24)." + */ + if (xSum != kResolution) { + LOG(RppX1Gsl, Error) + << "Invalid 'x-intervals': must cover full input range of " << kResolution + << ", got " << xSum; + return -EINVAL; + } + + const ValueNode &yObject = tuningData["y"]; + if (!yObject.isDictionary()) { + LOG(RppX1Gsl, Error) + << "Invalid 'y': must be a dictionary"; + return -EINVAL; + } + + if (!parseYLut(yObject, "red", kResolution - 1, curveYr_) || + !parseYLut(yObject, "green", kResolution - 1, curveYg_) || + !parseYLut(yObject, "blue", kResolution - 1, curveYb_)) + return -EINVAL; + + return 0; +} + +void GammaSensorLinearization::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RppX1Params *params) +{ + if (frame > 0) + return; + + auto config = params->block(); + config.setEnabled(true); + + std::copy(gammaDx_.begin(), gammaDx_.end(), config->dx); + std::copy(curveYr_.begin(), curveYr_.end(), config->curve_r); + std::copy(curveYg_.begin(), curveYg_.end(), config->curve_g); + std::copy(curveYb_.begin(), curveYb_.end(), config->curve_b); +} + +REGISTER_IPA_ALGORITHM(GammaSensorLinearization, "GammaSensorLinearization") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/gsl.h b/src/ipa/rppx1/algorithms/gsl.h new file mode 100644 index 0000000000..0c02ea4d97 --- /dev/null +++ b/src/ipa/rppx1/algorithms/gsl.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 input linearization algorithm + */ + +#pragma once + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class GammaSensorLinearization : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + +private: + std::vector gammaDx_; + std::array curveYr_; + std::array curveYg_; + std::array curveYb_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 7394b55bca..0a73d26ed3 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -6,5 +6,6 @@ rppx1_ipa_algorithms = files([ 'blc.cpp', 'ccm.cpp', 'gamma.cpp', + 'gsl.cpp', 'lux.cpp', ]) diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index b724fa3162..356c6311a9 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -22,6 +22,7 @@ enum class BlockType : uint16_t { ExmPre1, GaHv, HistPost, + LinPre1, WbMeasPost, }; @@ -45,6 +46,7 @@ RPPX1_DEFINE_BLOCK_TYPE(CcorPost, ccor, CCOR_POST) RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) RPPX1_DEFINE_BLOCK_TYPE(GaHv, ga, GA_HV) RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) +RPPX1_DEFINE_BLOCK_TYPE(LinPre1, lin, LIN_PRE1) RPPX1_DEFINE_BLOCK_TYPE(WbMeasPost, wbmeas, WBMEAS_POST) struct params_traits { From patchwork Mon Nov 16 17:49:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28934 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 199A1C3355 for ; Mon, 16 Nov 2026 17:49:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54BB16A899; Mon, 16 Nov 2026 18:49:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TXv96Kki"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE86F6A864 for ; Mon, 16 Nov 2026 18:49:33 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DBCB9929; Mon, 16 Nov 2026 18:47:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851227; bh=ow7pyxZaBIiseq66MvsDOHa2qaBWDVcZzeyaqHfqZoQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TXv96KkiiZVVMUQwTW+WwsXnJpOFLLNyvp8R9s/8qU7ING3ZK15WFgtjJl27fj1Fw QkygJ1YNO28DPW/UvuROaXKpJuYfJ+cSZOUCu2tLMUFJweYEqyUWoBYZq3Wb7DM/ID gyFqIcYHHNORQvP1PuBeYUY+YaRbjSk4cvu1iX1w= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v5 21/22] ipa: rppx1: Add lens shading correction Date: Mon, 16 Nov 2026 18:49:17 +0100 Message-ID: <20261116174919.1056403-22-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add the algorithm to the rppx1 ipa module based on the corresponding algorithm in the rkisp1 ipa module. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/rppx1/algorithms/lsc.cpp | 262 +++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/lsc.h | 58 ++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.h | 5 + src/ipa/rppx1/params.h | 4 + 5 files changed, 330 insertions(+) create mode 100644 src/ipa/rppx1/algorithms/lsc.cpp create mode 100644 src/ipa/rppx1/algorithms/lsc.h diff --git a/src/ipa/rppx1/algorithms/lsc.cpp b/src/ipa/rppx1/algorithms/lsc.cpp new file mode 100644 index 0000000000..c483b503c1 --- /dev/null +++ b/src/ipa/rppx1/algorithms/lsc.cpp @@ -0,0 +1,262 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 lens shading correction algorithm + */ + +#include "lsc.h" + +#include +#include +#include +#include +#include + +#include +#include + +#include "libcamera/internal/value_node.h" + +#include + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +LOG_DEFINE_CATEGORY(RppX1Lsc) + +namespace { + +using GradientQ = Q<12, 0>; + +constexpr int kColourTemperatureQuantization = 10; + +std::vector parseSizes(const ValueNode &tuningData, + const char *prop) +{ + std::vector sizes = + tuningData[prop].get>().value_or(utils::defopt); + if (sizes.size() != RPPX1_LSC_NUM_SECTORS) { + LOG(RppX1Lsc, Error) + << "Invalid '" << prop << "' values: expected " + << RPPX1_LSC_NUM_SECTORS + << " elements, got " << sizes.size(); + return {}; + } + + /* + * The sum of all elements must satisfy hardware constraints. + * Validate it here, allowing a 1% tolerance as rounding errors may + * prevent an exact match (further adjustments will be performed in + * LensShadingCorrection::prepare()). + */ + constexpr double expectedSum = 1.0; + + double sum = std::accumulate(sizes.begin(), sizes.end(), 0.0); + if (std::abs(sum - expectedSum) > 0.01) { + LOG(RppX1Lsc, Error) + << "Invalid '" << prop << "' values: sum of the elements" + << " should be " << expectedSum << ", got " << sum; + return {}; + } + + const auto &[min, max] = std::minmax_element(sizes.begin(), sizes.end()); + if (*min <= 0 || *max > 1) { + LOG(RppX1Lsc, Error) + << "Invalid '" << prop << "' values: elements must be in (0;1]"; + return {}; + } + + return sizes; +} + +std::vector sizesListToPositions(std::span sizes) +{ + std::vector positions(sizes.size() + 1); + + positions[0] = 0; + for (size_t i = 0; i < sizes.size(); i++) + positions[i + 1] = positions[i] + sizes[i]; + + return positions; +} + +unsigned int quantize(unsigned int value, unsigned int step) +{ + return std::lround(value / static_cast(step)) * step; +} + +} /* namespace */ + +int LensShadingCorrection::init([[maybe_unused]] IPAContext &context, + const ValueNode &tuningData) +{ + xSize_ = parseSizes(tuningData, "x-size"); + ySize_ = parseSizes(tuningData, "y-size"); + + if (xSize_.empty() || ySize_.empty()) + return -EINVAL; + + xPos_ = sizesListToPositions(xSize_); + yPos_ = sizesListToPositions(ySize_); + + return lscAlgo_.init(tuningData, context.ctrlMap, { + .keys = { "r", "gr", "gb", "b" }, + .numHSamples = RPPX1_LSC_SAMPLES_MAX, + .numVSamples = RPPX1_LSC_SAMPLES_MAX, + .sensorSize = context.sensorInfo.activeArea.size(), + }); +} + +int LensShadingCorrection::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + const Size &size = configInfo.outputSize; + Size totalSize{}; + + for (unsigned int i = 0; i < RPPX1_LSC_NUM_SECTORS; ++i) { + xSizes_[i] = xSize_[i] * size.width; + ySizes_[i] = ySize_[i] * size.height; + + /* + * From the documentation: + * + * The sum of all 16 {x,y}-size values must exactly match + * image {width,height}. + * + * Enforce it by computing the last table value separately to + * avoid rounding-induced errors. + */ + if (i == RPPX1_LSC_NUM_SECTORS - 1) { + xSizes_[i] = size.width - totalSize.width; + ySizes_[i] = size.height - totalSize.height; + } + + LOG(RppX1Lsc, Debug) + << i << ": " << xSizes_[i] << "=" << size.width << '*' << xSize_[i] << ", " + << ySizes_[i] << "=" << size.height << '*' << ySize_[i]; + + /* + * From the documentation: + * + * The sector size in {x,y}-direction must be greater than 7. + */ + if (xSizes_[i] < 8 || ySizes_[i] < 8) + return -EINVAL; + if (xSizes_[i] > 1023 || ySizes_[i] > 1023) + return -EINVAL; + + totalSize.width += xSizes_[i]; + totalSize.height += ySizes_[i]; + + /* + * From the documentation: + * + * The gradient values are related to the sector sizes and can be + * calculated for both x and y direction using the following equation: + * gradient[i] = int(2^15 / size[i] + 1/2) where i in [0;15] + */ + xGrad_[i] = GradientQ(float(1u << 15) / xSizes_[i]).quantized(); + yGrad_[i] = GradientQ(float(1u << 15) / ySizes_[i]).quantized(); + } + + lastAppliedCt_ = 0; + lastAppliedQuantizedCt_ = 0; + + return lscAlgo_.configure(context.activeState.lsc, configInfo.analogCrop, + xPos_, yPos_); +} + +void LensShadingCorrection::setParameters(rppx1_lsc_params &config) +{ + static_assert(sizeof(config.x_sect_size) == sizeof(xSizes_)); + memcpy(config.x_sect_size, xSizes_, sizeof(xSizes_)); + + static_assert(sizeof(config.y_sect_size) == sizeof(ySizes_)); + memcpy(config.y_sect_size, ySizes_, sizeof(ySizes_)); + + static_assert(sizeof(config.x_grad) == sizeof(xGrad_)); + memcpy(config.x_grad, xGrad_, sizeof(xGrad_)); + + static_assert(sizeof(config.y_grad) == sizeof(yGrad_)); + memcpy(config.y_grad, yGrad_, sizeof(yGrad_)); +} + +void LensShadingCorrection::copyTable(rppx1_lsc_params &config, + const lsc::Components &set) +{ + const auto &r = set.at("r"); + std::copy(r.begin(), r.end(), &config.r_data[0][0]); + const auto &gr = set.at("gr"); + std::copy(gr.begin(), gr.end(), &config.gr_data[0][0]); + const auto &gb = set.at("gb"); + std::copy(gb.begin(), gb.end(), &config.gb_data[0][0]); + const auto &b = set.at("b"); + std::copy(b.begin(), b.end(), &config.b_data[0][0]); +} + +void LensShadingCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, + controls); +} + +void LensShadingCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) +{ + uint32_t ct = frameContext.awb.colourTemperature; + unsigned int quantizedCt = quantize(ct, kColourTemperatureQuantization); + + /* Check if we can skip the update. */ + if (!frameContext.lsc.update) { + if (!frameContext.lsc.enabled) + return; + + /* + * Add a threshold so that oscillations around a quantization + * step don't lead to constant changes. + */ + if (utils::abs_diff(ct, lastAppliedCt_) < kColourTemperatureQuantization / 2) + return; + + if (quantizedCt == lastAppliedQuantizedCt_) + return; + } + + auto config = params->block(); + config.setEnabled(frameContext.lsc.enabled); + + if (!frameContext.lsc.enabled) + return; + + setParameters(*config); + + copyTable(*config, lscAlgo_.interpolateComponents(quantizedCt)); + + lastAppliedCt_ = ct; + lastAppliedQuantizedCt_ = quantizedCt; + + LOG(RppX1Lsc, Debug) + << "ct is " << ct << ", quantized to " << quantizedCt; +} + +void LensShadingCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RppX1Stats *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(LensShadingCorrection, "LensShadingCorrection") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/lsc.h b/src/ipa/rppx1/algorithms/lsc.h new file mode 100644 index 0000000000..d406ebd8ff --- /dev/null +++ b/src/ipa/rppx1/algorithms/lsc.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 lens shading correction algorithm + */ + +#pragma once + +#include + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class LensShadingCorrection : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; +private: + void setParameters(rppx1_lsc_params &config); + void copyTable(rppx1_lsc_params &config, + const lsc::Components &set); + + std::vector xSize_; + std::vector ySize_; + std::vector xPos_; + std::vector yPos_; + uint16_t xGrad_[RPPX1_LSC_NUM_SECTORS]; + uint16_t yGrad_[RPPX1_LSC_NUM_SECTORS]; + uint16_t xSizes_[RPPX1_LSC_NUM_SECTORS]; + uint16_t ySizes_[RPPX1_LSC_NUM_SECTORS]; + + unsigned int lastAppliedCt_ = 0; + unsigned int lastAppliedQuantizedCt_ = 0; + + LscAlgorithm> lscAlgo_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 0a73d26ed3..5a1068cd7f 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -7,5 +7,6 @@ rppx1_ipa_algorithms = files([ 'ccm.cpp', 'gamma.cpp', 'gsl.cpp', + 'lsc.cpp', 'lux.cpp', ]) diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index d965311ca1..538c30814b 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -22,6 +22,7 @@ #include #include #include +#include #include namespace libcamera { @@ -50,6 +51,8 @@ struct IPAActiveState { ipa::gamma::ActiveState gamma; + ipa::lsc::ActiveState lsc; + ipa::lux::ActiveState lux; }; @@ -70,6 +73,8 @@ struct IPAFrameContext : public FrameContext { ipa::gamma::FrameContext gamma; + ipa::lsc::FrameContext lsc; + ipa::lux::FrameContext lux; }; diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h index 356c6311a9..958b0a9b6e 100644 --- a/src/ipa/rppx1/params.h +++ b/src/ipa/rppx1/params.h @@ -22,6 +22,8 @@ enum class BlockType : uint16_t { ExmPre1, GaHv, HistPost, + LscPre1, + LscPre2, LinPre1, WbMeasPost, }; @@ -46,6 +48,8 @@ RPPX1_DEFINE_BLOCK_TYPE(CcorPost, ccor, CCOR_POST) RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) RPPX1_DEFINE_BLOCK_TYPE(GaHv, ga, GA_HV) RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) +RPPX1_DEFINE_BLOCK_TYPE(LscPre1, lsc, LSC_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(LscPre2, lsc, LSC_PRE2) RPPX1_DEFINE_BLOCK_TYPE(LinPre1, lin, LIN_PRE1) RPPX1_DEFINE_BLOCK_TYPE(WbMeasPost, wbmeas, WBMEAS_POST) From patchwork Mon Nov 16 17:49:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28935 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D8EE3C3364 for ; Mon, 16 Nov 2026 17:49:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E5C806A89B; Mon, 16 Nov 2026 18:49:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="s+FZ/It4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3490C6A866 for ; Mon, 16 Nov 2026 18:49:34 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3C1FD1F2B; Mon, 16 Nov 2026 18:47:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794851227; bh=vDZXs0FWKtmC2toQHW/gKapMP1simeCWQhTFRufGG0M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=s+FZ/It4s8zimMx0UDurQA/2jdjilNISdNIrjOxoV8k9CB9vEE+2hqtpfDCS68ZHD 970G4xTihEcwhd/kMVaeRAggabOtXMSxfL8VWgJ8W4LGck/BNh/eDoroH579XxzOuR gCEEgI3Bp4fUgW+Ihf18ywU/gLhKS9p8Ggu0uzb4= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham Subject: [PATCH v5 22/22] ipa: rppx1: Add uncalibrated tuning file Date: Mon, 16 Nov 2026 18:49:18 +0100 Message-ID: <20261116174919.1056403-23-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> References: <20261116174919.1056403-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Enable algorithms that don't need any specific tuning data. Signed-off-by: Barnabás Pőcze Acked-by: Kieran Bingham --- src/ipa/rppx1/data/meson.build | 1 + src/ipa/rppx1/data/uncalibrated.yaml | 11 +++++++++++ 2 files changed, 12 insertions(+) create mode 100644 src/ipa/rppx1/data/uncalibrated.yaml diff --git a/src/ipa/rppx1/data/meson.build b/src/ipa/rppx1/data/meson.build index d1eb321758..b18511441b 100644 --- a/src/ipa/rppx1/data/meson.build +++ b/src/ipa/rppx1/data/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ + 'uncalibrated.yaml', ]) install_data(conf_files, diff --git a/src/ipa/rppx1/data/uncalibrated.yaml b/src/ipa/rppx1/data/uncalibrated.yaml new file mode 100644 index 0000000000..9d6319a941 --- /dev/null +++ b/src/ipa/rppx1/data/uncalibrated.yaml @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - Agc: + - Awb: + - BlackLevelCorrection: + - Ccm: + - Gamma: +...