From patchwork Wed Sep 2 09:07:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28163 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BFA48C3344 for ; Wed, 2 Sep 2026 09:07:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1605F685C4; Wed, 2 Sep 2026 11:07:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LGpMTTiu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BBA926859C for ; Wed, 2 Sep 2026 11:07:34 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 53B0224BD; Wed, 2 Sep 2026 11:06:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339963; bh=G/ZKVxtkM0vGyVqWxm9UpzmxbA7RUcEeTxMnnIFJQyw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=LGpMTTiu5y5giPmR+oacQklP+NhP8gksl0eE9MHmPHvY+hF2LjkDkK2PUCePSQL99 GNni1VcFPIsGRb1dbbkjGhhqg6ZKOgy/DqwSpLm63qKCtcAdJCZocMQgjFeKqa1fcz 9+m9qcahNGRdAu3QgQPuHWC+7nJ1yZYPKhpAvb54= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:19 +0100 Subject: [PATCH v3 01/10] libipa: awb_grey: Correct minimum gain clamping MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-1-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1937; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=G/ZKVxtkM0vGyVqWxm9UpzmxbA7RUcEeTxMnnIFJQyw=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dTX6mQrSVaMQ2AwMGYBshSpPN2jmfM6PqU0 gJ3/xuo9CmJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnUwAKCRDISVd6bEV1 MrjxD/9bDPxFZ9xYIGSp7Gm3rUTYVqM4EqxkfeSk0VzjoYeU4SkJUJ3rV3va16xkiJ5o9Hh3U8g GLG5sOYD8Ll18BjCLM1EbHg4dbrKgLh/s+U1VgppMRKOYAg1Em69xmGCgu9ujuQF0XTj9/v9KR3 DxKZ/5+0sD/s7YIw/OfGBc66dHpUMC+uunMvoLeYRnxS40hvJWW3ReQiCg3r6TokJomo9gNWvN+ IhGGoxMMBMP1FRvqs+Pt/8auP3VRBaT2rYMQFzUXjf+9aQpaYodbSqvpO9gDTn0NpCNdu3drupW hWAxbxMZ905haXpmrzCHEdK23+Kx9SwCg4X6iim40a5I75uoGEORlt2c1UW/EaagmVY/I2HS7PT hykeNjoTGpfDfxOm3xgEE9M9uMK6i9spWv9PLjiHbm6D9vOLyhNQCEPn1bM9zWrw/9NP4GOdbcB YTPOW3sm5ADE3bupXSlw8WSYOnJxuWkPb9N5HBYmxBQOdxFGnGdME8r4x6TR96gM+BqLohgaN/E ocIc/C6PUCM7jyQ3U2yPby91FvFBdy5X+iAcws0+kRSEsoDqpAw8kHQQC08ifDI/swdGQGRaZPq VLjpgT5rCtZxHvPYaWgrZ2EFhAKvXbdNLHRMWF1P6T1Ab7i/zn4U40lJc4UwsOCC4/N2CiQScd+ JlO23xi/walGfgg== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AwbGrey algorithm currently calculates Red and Blue gains by dividing the observed mean green by the observed mean red or blue value, clamped to 1.0. The documentation for AwbStats::rgbMeans() says: Fetch the RGB means from the statistics. The values of each channel are dimensionless and only the ratios are used for further calculations. This is used by the simple grey world model to calculate the gains to apply. \todo Make a requirement to return statistics in the [0, 1] range. This requirement that the means be within the [0, 1] range is at odds with the clamping, as it means that all values will be clamped to 1.0 and so the gains will always all be 1.0. Change the clamp to be to 0.001, which still fulfills the goal of preventing divide-by-zero errors whilst allowing means in the range [0, 1]. Signed-off-by: Daniel Scally --- Changes in v3: - New patch --- src/ipa/libipa/awb_grey.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ipa/libipa/awb_grey.cpp b/src/ipa/libipa/awb_grey.cpp index b1222902388393aa24dbebd6f466cc6d6389fcdc..9ca9b5a587dbce6d02765a07f1da3c782d713555 100644 --- a/src/ipa/libipa/awb_grey.cpp +++ b/src/ipa/libipa/awb_grey.cpp @@ -82,11 +82,11 @@ AwbGrey::calculateAwb(const AwbStats &stats, [[maybe_unused]] unsigned int lux, /* * Estimate the red and blue gains to apply in a grey world. The green * gain is hardcoded to 1.0. Avoid divisions by zero by clamping the - * divisor to a minimum value of 1.0. + * divisor to a minimum value of 0.001. */ - result.gains.r() = means.g() / std::max(means.r(), 1.0); + result.gains.r() = means.g() / std::max(means.r(), 0.001); result.gains.g() = 1.0; - result.gains.b() = means.g() / std::max(means.b(), 1.0); + result.gains.b() = means.g() / std::max(means.b(), 0.001); return result; } From patchwork Wed Sep 2 09:07:20 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28164 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 253D7C3344 for ; Wed, 2 Sep 2026 09:07:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2AD87685B0; Wed, 2 Sep 2026 11:07:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G0KHJxi9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D006E6859D for ; Wed, 2 Sep 2026 11:07:34 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 922EC25CC; Wed, 2 Sep 2026 11:06:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339963; bh=B/mNeOq/BMWFwDja0ce5qaPrrb6CvFMkRZxq/y1M0V4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=G0KHJxi9LFty5VvvFnfO1Gns6PJg4MErWAhmX2mgk7kn/YLZkNZs0yCtSidKhnZsf KqA+g6AJ4xkGmOFV7p+XdQTFVIDraBZuviYXfU90nOn8WJeyukcSgYVdDTPM2WTbTk ZPn2YKaVD+MfWZETCsB9BBUYxOpuZvI2qZPPkFzU= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:20 +0100 Subject: [PATCH v3 02/10] ipa: ipu3: Remove ~Awb() from source and header MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-2-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , =?utf-8?q?Barnab=C3=A1s_?= =?utf-8?b?UMWRY3pl?= , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1432; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=B/mNeOq/BMWFwDja0ce5qaPrrb6CvFMkRZxq/y1M0V4=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dTFLSg33T4g5gPrlCpouowxuDKlWsCNo0+W 7ejT/WgasyJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnUwAKCRDISVd6bEV1 Mq3qD/4kLFXQYDP02cLrpBOszAkis09HU1rWGyXSoFaspanA7wgV1cYoMsSly2Y5LIZKZnZZNKp nQwdAbPMkTFLoDeOmC/yom64vGT14pEbkWaEJtaCFtymrDEQIViDuJxyx3V67ZXU7orijLiPd9u j9dI63POQvBfY1o1N5oldwL01ZukKDdsc7Sb5TxI+ljhI/aKYLZVYsWQwlh0BBYx5TCDhywsgJP /EakG5eWcoFjZZSzu4udBxYOwGLsUUaTWxWPOlaLUUbFsdXWuUaLhETqw3Zmeaoq/0R3BrCHnm9 wDG95zAFZo/caS37i/CXLN5AQHMpkCTN7+E0UvuB6je0LRn6v4nyvTdBPk3mrCKXoEd7+06zFZJ zdED/AoYCNFsyuH7LV71kB2UZJQ6wGiPsQ8DuTjGV3hQbhK263v6ircrG+dmi6Nja89iElqawT0 31E8h/iAIERQcJxONu8QqO2/OccSiDY2yWN5FhdshLl9lwUgI9fkr7JWnDqjX+sC4X55af1eZTj Oe+TgGTRoQ1r5sn+aGh6qmBhCWAocVL2SFQ/pzubvx9z3B9cIj8rBsUFh37OnpmlcNhJsA6h5vE 03N8Zq1Txem0qI9dku9LnGfgui1FpJ4BnRaOcxdERJgjELGjx1p2PX0LWqa3Fjqvf9t9284JJ1i T596uTrPggs8WZA== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove the ~Awb() definition from the source and header files and simply allow the compiler to provide the default implementations. Reviewed-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v3: - None Changes in v2: - None --- src/ipa/ipu3/algorithms/awb.cpp | 2 -- src/ipa/ipu3/algorithms/awb.h | 1 - 2 files changed, 3 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 55de05d9e39f5d2c863b89923964a4c7756f3846..6b4418aa6f3add0a41cae77000c1f5f67897111a 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -195,8 +195,6 @@ Awb::Awb() zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } -Awb::~Awb() = default; - /** * \copydoc libcamera::ipa::Algorithm::configure */ diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index dbf69c9073a1627b666fe63158081e347cf3581e..7ec322318dab54ae7c8a647a67a0cf5815a36eb6 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -38,7 +38,6 @@ class Awb : public Algorithm { public: Awb(); - ~Awb(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, From patchwork Wed Sep 2 09:07:21 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28165 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7427DC3347 for ; Wed, 2 Sep 2026 09:07:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D9035685BD; Wed, 2 Sep 2026 11:07:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BFOW3lB/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF6E56859E for ; Wed, 2 Sep 2026 11:07:34 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DE096265C; Wed, 2 Sep 2026 11:06:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339964; bh=rOyAJHvtnDZWZjtLSuGZg5l3Slr3kiSC6h37DqDnzhw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BFOW3lB/iiUrofLc03Lo57Y5UBru5ExRmlHzLs/BoZssoi7EJaQ9sDA0oXgChze/l GiwgvP0DV5UzQmRX1Dq3G7vc2MeDQF8MgTMQHF6bmRc4gkF3uQb4q04N+7hPa8rkS6 XY6Vr1CuwCVSDeeMCf621lEpuV2oOaARxcWxiUSc= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:21 +0100 Subject: [PATCH v3 03/10] ipu3: awb: Remove CCM settings from Awb algorithm MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-3-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1710; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=rOyAJHvtnDZWZjtLSuGZg5l3Slr3kiSC6h37DqDnzhw=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dThLjk9We5gV9HUVTQ+YxqP/hvud30sA39V PQt595ch7qJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnUwAKCRDISVd6bEV1 Mm91EACuCdkySUOUwuNaQ7sxdywkWrlEIrXtJ+KZ0yU5jK2IkEs4kDOMoxBDlZRbRPxvfp9qToz oJgVTEaN6hQLLOJAKq4rB6bmufVfq05P5zJkBBMQAL1gmtvWJy1U0TwGHekvLBMbfDqtJ0s7ItG Lu/NbL0I0Qsy4aIuefFCUMhmQg/iaEgzxxctKcxvNqrEIPmNPiRDaP8HzRhIbpmxpbHIFIOLdCP uIhMittYTtNUL93vApNhTPCsxOhPM2I5/hkrSXhUQY8yQIaWRGGhe3c1ciZkEujw23yPiPilvcM yJTk8F0Vb0W3wgYgzJKFYfZE5bHWhgYD+NzHqryNZBLKbI6WWYmbdcQgTIFi/ANNYkceARSWhk8 MYr4FHISXLFFWLjeBYAoK76LqVJ7uyfHDNGI01SSKrkJ/RTFzydOZTLCT/T5MUZDISPwLhXrgp6 3mZF3vv0kht4cWAPyPRy5bfu092w9/WLsOGbrrEZEU0J8Xlorz8MyQCovOtQapnklqncMORFFOa gnRWT+twTigVNIWzeX0kY4DV6xYI12kxitymOewA1Bayzy1oa4FI8xwQEWiQvzMmHMQ/XBILzm5 edOYmvpf105MmYhSdIZeocLHRIUcvVgBMicrdYtYhYDtv2y33zIT6gng1wgwbAlAhltICG7/Poa iJ/xy3ZaJZsjrKQ== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 AWB algorithm currently defines a CCM matrix that is just a no-op. Remove the settings entirely; they need really to go in to a dedicated CCM algorithm. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Barnabás Pőcze Signed-off-by: Daniel Scally --- Changes in v3: - None Changes in v2: - None --- src/ipa/ipu3/algorithms/awb.cpp | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 6b4418aa6f3add0a41cae77000c1f5f67897111a..613bdbbf51cd127e03906571813260a6857067dc 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -112,13 +112,6 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { .opt_center_sqr = { 419904, 133956 }, }; -/* Default color correction matrix defined as an identity matrix */ -static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { - 8191, 0, 0, 0, - 0, 8191, 0, 0, - 0, 0, 8191, 0 -}; - /** * \class Awb * \brief A Grey world white balance correction algorithm @@ -293,12 +286,9 @@ void Awb::prepare(IPAContext &context, LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; - /* The CCM matrix may change when color temperature will be used */ - params->acc_param.ccm = imguCssCcmDefault; params->use.acc_awb = 1; params->use.acc_bnr = 1; - params->use.acc_ccm = 1; } /* Generate an RGB vector with the average values for each zone */ From patchwork Wed Sep 2 09:07:22 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28166 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4333C3344 for ; Wed, 2 Sep 2026 09:07:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D8F1A685BC; Wed, 2 Sep 2026 11:07:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tzwyVslS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 459E06859F for ; Wed, 2 Sep 2026 11:07:35 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3D42E27F6; Wed, 2 Sep 2026 11:06:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339964; bh=jn6whBE1Vyo3imlQKwoHwKJRD6QBjwBYrSFNwA/1lcU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=tzwyVslSoF0H03TXFJYZvdxJw4twTjoPog/u0KckuE1h1ete4uRYqRkfsb3KH8Ujr vmLrofRoziS5R8eMfeexm3wNyFQpYQR4r82dA1UsyU3ISymSTaImUztklYyitbI66h Ec8WphY8Ay5Lu5pPVc7hpRsXq5MGA0FMEN5zePWY= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:22 +0100 Subject: [PATCH v3 04/10] ipa: ipu3: awb: Port to the new libipa AwbAlgorithm MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-4-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=18510; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=jn6whBE1Vyo3imlQKwoHwKJRD6QBjwBYrSFNwA/1lcU=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dTkWRvs47Q9a0XNb0xieHEiVhoGxc9IhgpL j3EzThfJAqJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnUwAKCRDISVd6bEV1 MmFID/sFaV+Hoh5D8T+AkXDptqx51/gRl+H4UzkhLgD+KeqOIsl/+7GxXb8tLrzuXH13JizGx09 GdC6u31Muw+QHY8eky88MTsuIyJlljrEmrJXlFfulGEiYFbt17QlOb3EB6u/VXkGh7R13iTW7nF ltwKcCo+4c4fAvqFCl59SKkHst7qzJgimZZHYH2ysgYbk9KLGLGAuj7PSc3unGE92CbHylUXO+g y1AP45bvCmeCdcZhMvlX44RzqIiZAhq3E3yw1+iReLG/yzVerj3Co/7RM6k0YVZrqqQzsojRZ7o cycnoSuMF7641fzuyhIGqoEeynMcGuFGkR57M/FA5faWFuDUyL2fwnVdomMr48D1gWl6hv6KVfs 4Bv3Cp4c3evXH9X0yMC8sWkO7vA05ljGyecdDniaJTODMTlxSQZcW14DRlfBRWqymbqIubU8XxP wQDJWkPGZN9MlfKiUn8s7Ujljc0MdLvY8AxjDirnhgNi0MmRNG86ZnRe1QN3oE22VLjS8YGZIwD eFOiHteI9HTjNjNR3GZv3Ebn2jx3PTFOpd1SyHNpjnlQN0vITwNG8xZPoip3YEOveS4hps7NQTt NcW/RU2culyltrbkJQoKxg+WfP5bpg83KhEYyaKGpi0YXc1XeOialDj6teMP8cpBXUAiUdVMsow RJz8zQdiyo+JKXA== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Port the IPU3 Awb algorithm to use the new libipa implementation of AwbAlgorithm. In this implementation the awbAlgo_ class member is initialised as UQ<3, 13> following the IPU3 params format documentation. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v3: - Revised to account for the changes to the libipa AWB implementation Changes in v2: - Used the new RGB(gains_) member of the class - Fixed includes - Commented minColourValue --- src/ipa/ipu3/algorithms/agc.cpp | 16 ++- src/ipa/ipu3/algorithms/awb.cpp | 233 +++++++++++++++++++++------------------- src/ipa/ipu3/algorithms/awb.h | 21 ++-- src/ipa/ipu3/ipa_context.cpp | 22 ++-- src/ipa/ipu3/ipa_context.h | 13 +-- src/ipa/libipa/awb.cpp | 5 - 6 files changed, 150 insertions(+), 160 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index b0f535b9902fa54d0ec3d689467d75613473cbcd..b30c40d93eb0f2aae6aec04a0f7fefa92978e8af 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -217,17 +217,15 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, ControlList &metadata) { Histogram hist = parseStatistics(stats, context.configuration.grid.bdsGrid); + ipa::awb::Context awbContext; + + if (context.activeState.awb.autoEnabled) + awbContext = context.activeState.awb.automatic; + else + awbContext = context.activeState.awb.manual; agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {{ - .traits = AgcTraits{ - rgbTriples_, - {{ - context.activeState.awb.gains.red, - context.activeState.awb.gains.blue, - context.activeState.awb.gains.green, - }}, - bdsGrid_, - }, + .traits = AgcTraits{rgbTriples_, awbContext.gains, bdsGrid_}, .yHist = hist, .exposure = frameContext.sensor.exposure, .gain = frameContext.sensor.gain, diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 613bdbbf51cd127e03906571813260a6857067dc..84d9f62e1c5836380776df785339ef847354583a 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -13,8 +13,6 @@ #include -#include "libipa/colours.h" - /** * \file awb.h */ @@ -26,11 +24,77 @@ namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Awb) /* - * When zones are used for the grey world algorithm, they are only considered if - * their average green value is at least 16/255 (after black level subtraction) - * to exclude zones that are too dark and don't provide relevant colour - * information (on the opposite side of the spectrum, saturated regions are - * excluded by the ImgU statistics engine). + * \todo IPU3 doesn't support the Lux algorithm. + */ +static constexpr unsigned int kDefaultLux = 500; + +/** + * \brief The IPU3 specific implementation of AwbStats + */ +class Ipu3AwbStats final : public AwbStats +{ +public: + Ipu3AwbStats() = default; + + /** + * \brief Constructs an Ipu3AwbStats object with the given RGB means + * + * \param rgbMeans A vector of doubles representing the RGB mean values + */ + Ipu3AwbStats(const RGB &rgbMeans) + : rgbMeans_(rgbMeans) + { + rg_ = rgbMeans_.r() / rgbMeans_.g(); + bg_ = rgbMeans_.b() / rgbMeans_.g(); + } + + double computeColourError(const RGB &gains) const override; + bool valid() const override; + RGB rgbMeans() const override; + +private: + RGB rgbMeans_; + double rg_; + double bg_; +}; + +double Ipu3AwbStats::computeColourError(const RGB &gains) const +{ + /* + * Compute the sum of the squared colour error (non-greyness) as + * it appears in the log likelihood equation. + */ + double deltaR = gains.r() * rg_ - 1.0; + double deltaB = gains.b() * bg_ - 1.0; + double delta2 = deltaR * deltaR + deltaB * deltaB; + + return delta2; +} + +bool Ipu3AwbStats::valid() const +{ + /* + * This validity assessment is designed to allow libipa to decide + * whether there's enough information in the statistics for a frame to + * be useful. The IPU3 implementation already drops any statistics zone + * with an average value below a threshold though so we don't need to do + * it in libipa. Report all stats as valid. + */ + + return true; +} + +RGB Ipu3AwbStats::rgbMeans() const +{ + return rgbMeans_; +} + +/* + * Zones are only considered if their average green value is at least + * kMinGreenLevelInZone/255 (after black level subtraction) to exclude zones + * that are too dark and don't provide relevant colour information (on the + * opposite side of the spectrum, saturated regions are excluded by the ImgU + * statistics engine). */ static constexpr uint32_t kMinGreenLevelInZone = 16; @@ -74,26 +138,6 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; * \brief Sum of the average blue values of each unsaturated cell in the zone */ -/** - * \struct Awb::AwbStatus - * \brief AWB parameters calculated - * - * The AwbStatus structure is intended to store the AWB - * parameters calculated by the algorithm - * - * \var AwbStatus::temperatureK - * \brief Color temperature calculated - * - * \var AwbStatus::redGain - * \brief Gain calculated for the red channel - * - * \var AwbStatus::greenGain - * \brief Gain calculated for the green channel - * - * \var AwbStatus::blueGain - * \brief Gain calculated for the blue channel - */ - /* Default settings for Bayer noise reduction replicated from the Kernel */ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { .wb_gains = { 16, 16, 16, 16 }, @@ -114,11 +158,7 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { /** * \class Awb - * \brief A Grey world white balance correction algorithm - * - * The Grey World algorithm assumes that the scene, in average, is neutral grey. - * Reference: Lam, Edmund & Fung, George. (2008). Automatic White Balancing in - * Digital Photography. 10.1201/9781420054538.ch10. + * \brief The IPU3 white balance correction algorithm implementation * * The IPU3 generates statistics from the Bayer Down Scaler output into a grid * defined in the ipu3_uapi_awb_config_s structure. @@ -168,26 +208,26 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { * cells are ignored. The grid configuration is computed by * IPAIPU3::calculateBdsGrid(). * - * Before calculating the gains, the algorithm aggregates the cell averages for - * each zone in generateAwbStats(). Cells that have a too high ratio of - * saturated pixels are ignored, and only zones that contain enough - * non-saturated cells are then used by the algorithm. - * - * The Grey World algorithm will then estimate the red and blue gains to apply, and - * store the results in the metadata. The green gain is always set to 1. + * Before running the AWB algorithm, we aggregate the cell averages for each + * zone in generateAwbStats(). Cells that have a too high ratio of saturated + * pixels are ignored, and only zones that contain enough non-saturated cells + * are then used by the algorithm. */ Awb::Awb() : Algorithm() { - asyncResults_.blueGain = 1.0; - asyncResults_.greenGain = 1.0; - asyncResults_.redGain = 1.0; - asyncResults_.temperatureK = 4500; - zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ @@ -197,12 +237,14 @@ int Awb::configure(IPAContext &context, const ipu3_uapi_grid_config &grid = context.configuration.grid.bdsGrid; stride_ = context.configuration.grid.stride; + awbAlgo_.configure(context.activeState.awb); + cellsPerZoneX_ = std::round(grid.width / static_cast(kAwbStatsSizeX)); cellsPerZoneY_ = std::round(grid.height / static_cast(kAwbStatsSizeY)); /* * Configure the minimum proportion of cells counted within a zone - * for it to be relevant for the grey world algorithm. + * for it to be used. * \todo This proportion could be configured. */ cellsPerZoneThreshold_ = cellsPerZoneX_ * cellsPerZoneY_ * kMaxCellSaturationRatio; @@ -211,6 +253,17 @@ int Awb::configure(IPAContext &context, return 0; } +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + constexpr uint16_t Awb::threshold(float value) { /* AWB thresholds are in the range [0, 8191] */ @@ -237,11 +290,12 @@ constexpr uint16_t Awb::gainValue(double gain) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - ipu3_uapi_params *params) +void Awb::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] ipu3_uapi_params *params) { + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + /* * Green saturation thresholds are reduced because we are using the * green channel only in the exposure computation. @@ -279,13 +333,11 @@ void Awb::prepare(IPAContext &context, params->acc_param.bnr.opt_center_sqr.y_sqr_reset = params->acc_param.bnr.opt_center.y_reset * params->acc_param.bnr.opt_center.y_reset; - params->acc_param.bnr.wb_gains.gr = gainValue(context.activeState.awb.gains.green); - params->acc_param.bnr.wb_gains.r = gainValue(context.activeState.awb.gains.red); - params->acc_param.bnr.wb_gains.b = gainValue(context.activeState.awb.gains.blue); - params->acc_param.bnr.wb_gains.gb = gainValue(context.activeState.awb.gains.green); - - LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; + params->acc_param.bnr.wb_gains.gr = gainValue(frameContext.awb.gains.g()); + params->acc_param.bnr.wb_gains.r = gainValue(frameContext.awb.gains.r()); + params->acc_param.bnr.wb_gains.b = gainValue(frameContext.awb.gains.b()); + params->acc_param.bnr.wb_gains.gb = gainValue(frameContext.awb.gains.g()); params->use.acc_awb = 1; params->use.acc_bnr = 1; @@ -366,9 +418,17 @@ void Awb::clearAwbStats() } } -void Awb::awbGreyWorld() +Ipu3AwbStats Awb::calculateRgbMeans(const ipu3_uapi_stats_3a *stats) { - LOG(IPU3Awb, Debug) << "Grey world AWB"; + ASSERT(stats->stats_3a_status.awb_en); + + clearAwbStats(); + generateAwbStats(stats); + generateZones(); + + if (zones_.size() <= 10) + return {}; + /* * Make a separate list of the derivatives for each of red and blue, so * that we can sort them to exclude the extreme gains. We could @@ -399,66 +459,21 @@ void Awb::awbGreyWorld() double redGain = sumRed.g() / (sumRed.r() + 1), blueGain = sumBlue.g() / (sumBlue.b() + 1); - /* Color temperature is not relevant in Grey world but still useful to estimate it :-) */ - asyncResults_.temperatureK = estimateCCT({{ sumRed.r(), sumRed.g(), sumBlue.b() }}); - - /* - * Gain values are unsigned integer value ranging [0, 8) with 13 bit - * fractional part. - */ - redGain = std::clamp(redGain, 0.0, 65535.0 / 8192); - blueGain = std::clamp(blueGain, 0.0, 65535.0 / 8192); - - asyncResults_.redGain = redGain; - /* Hardcode the green gain to 1.0. */ - asyncResults_.greenGain = 1.0; - asyncResults_.blueGain = blueGain; -} - -void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) -{ - ASSERT(stats->stats_3a_status.awb_en); - - clearAwbStats(); - generateAwbStats(stats); - generateZones(); - - LOG(IPU3Awb, Debug) << "Valid zones: " << zones_.size(); - - if (zones_.size() > 10) { - awbGreyWorld(); - LOG(IPU3Awb, Debug) << "Gain found for red: " << asyncResults_.redGain - << " and for blue: " << asyncResults_.blueGain; - } + return Ipu3AwbStats({ { 1.0 / redGain, 1.0, 1.0 / blueGain } }); } /** * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - const ipu3_uapi_stats_3a *stats, - [[maybe_unused]] ControlList &metadata) + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) { - calculateWBGains(stats); + Ipu3AwbStats awbStats = calculateRgbMeans(stats); - /* - * Gains are only recalculated if enough zones were detected. - * The results are cached, so if no results were calculated, we set the - * cached values from asyncResults_ here. - */ - context.activeState.awb.gains.blue = asyncResults_.blueGain; - context.activeState.awb.gains.green = asyncResults_.greenGain; - context.activeState.awb.gains.red = asyncResults_.redGain; - context.activeState.awb.temperatureK = asyncResults_.temperatureK; - - metadata.set(controls::AwbEnable, true); - metadata.set(controls::ColourGains, { - static_cast(context.activeState.awb.gains.red), - static_cast(context.activeState.awb.gains.blue) - }); - metadata.set(controls::ColourTemperature, - context.activeState.awb.temperatureK); + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + kDefaultLux, metadata); } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 7ec322318dab54ae7c8a647a67a0cf5815a36eb6..e1a7b2139e5dafa748b908230aa3189f33b09ced 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -13,7 +13,8 @@ #include -#include "libcamera/internal/vector.h" +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" #include "algorithm.h" @@ -21,6 +22,8 @@ namespace libcamera { namespace ipa::ipu3::algorithms { +class Ipu3AwbStats; + /* Region size for the statistics generation algorithm */ static constexpr uint32_t kAwbStatsSizeX = 16; static constexpr uint32_t kAwbStatsSizeY = 12; @@ -39,7 +42,11 @@ class Awb : public Algorithm public: Awb(); + int init(IPAContext &context, const ValueNode &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; @@ -49,15 +56,7 @@ public: ControlList &metadata) override; private: - struct AwbStatus { - double temperatureK; - double redGain; - double greenGain; - double blueGain; - }; - -private: - void calculateWBGains(const ipu3_uapi_stats_3a *stats); + Ipu3AwbStats calculateRgbMeans(const ipu3_uapi_stats_3a *stats); void generateZones(); void generateAwbStats(const ipu3_uapi_stats_3a *stats); void clearAwbStats(); @@ -67,7 +66,7 @@ private: std::vector> zones_; Accumulator awbStats_[kAwbStatsSizeX * kAwbStatsSizeY]; - AwbStatus asyncResults_; + AwbAlgorithm> awbAlgo_; uint32_t stride_; uint32_t cellsPerZoneX_; diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index c358b099810477c5f8264d3024f99b5bf2467f06..5cd8fedf27870474eb35745520206de885006b4a 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -113,22 +113,7 @@ namespace libcamera::ipa::ipu3 { /** * \var IPAActiveState::awb - * \brief Context for the Automatic White Balance algorithm - * - * \var IPAActiveState::awb.gains - * \brief White balance gains - * - * \var IPAActiveState::awb.gains.red - * \brief White balance gain for R channel - * - * \var IPAActiveState::awb.gains.green - * \brief White balance gain for G channel - * - * \var IPAActiveState::awb.gains.blue - * \brief White balance gain for B channel - * - * \var IPAActiveState::awb.temperatureK - * \brief Estimated color temperature + * \brief Active auto-white balance parameters for the IPA */ /** @@ -162,4 +147,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame state for the AGC algorithm */ +/** + * \var IPAFrameContext::awb + * \brief Per-frame auto-white balance parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index e8f27bc4299bbc22dc987f68cff8388ad80d9322..d2673b42bebb36dde06a795b6c1fbed6e9052535 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -18,6 +18,7 @@ #include #include +#include #include #include @@ -47,16 +48,7 @@ struct IPAActiveState { } af; agc::ActiveState agc; - - struct { - struct { - double red; - double green; - double blue; - } gains; - - double temperatureK; - } awb; + ipa::awb::ActiveState awb; struct { double gamma; @@ -71,6 +63,7 @@ struct IPAFrameContext : public FrameContext { } sensor; agc::FrameContext agc; + ipa::awb::FrameContext awb; }; struct IPAContext { diff --git a/src/ipa/libipa/awb.cpp b/src/ipa/libipa/awb.cpp index da835bec018d55b13200008bc24ac11db25b7bff..0e312690b8d2becccbb1405d93cac9c04b2d61a4 100644 --- a/src/ipa/libipa/awb.cpp +++ b/src/ipa/libipa/awb.cpp @@ -75,11 +75,6 @@ namespace awb { * hardware-specific statistics data. */ -/** - * AwbStat::~AwbStat - * \brief Virtual class destructor - */ - /** * \fn AwbStats::computeColourError() * \brief Compute an error value for when the given gains would be applied From patchwork Wed Sep 2 09:07:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28167 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F0CDEC3347 for ; Wed, 2 Sep 2026 09:07:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 92F84685D7; Wed, 2 Sep 2026 11:07:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cYoDyaX/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 92AAA685A1 for ; Wed, 2 Sep 2026 11:07:35 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 922DC24BD; Wed, 2 Sep 2026 11:06:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339964; bh=JM6wZqPlBag87uB7YzcmfNKMellRr5XzgmuieVOA3/g=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cYoDyaX/iaBMLyGN3gLoB2UFLHLvrxCZ35U+LuIGeRC0GKVk2p9Y6/avNQ5JzPq73 RUTZ3S80J6BXNkarOrd6ibFY7kYuFYJszN25mNU1KZZEX3/IvKCjGWrmQUfX8sdrqg cT3iqaGQiTiqHIEO3o1hUGXAONApwdkbdlE0Nlzs= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:23 +0100 Subject: [PATCH v3 05/10] ipa: ipu3: ccm: Add a Colour Correction Matrix algorithm MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-5-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham , Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8212; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=JM6wZqPlBag87uB7YzcmfNKMellRr5XzgmuieVOA3/g=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dUYnq4uqSSb0wuV3mj9329eDde/VHLG1tZu pi2+shHHZ2JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnVAAKCRDISVd6bEV1 MmMqEACFqSE2YMLBp/CJv8zghMxa4QvUyp/pYfo4QNq0JkmR1JnTwIE5UGwnPgYKx0+3oDppI5P I/aOLXN+xlA6HhF/mz96uiB0RVUHBmB/AxVtLW/QumlET7dMKj7dCi6a/EI8WUtmoHJ1+RJioOz 4B4igFK3pbFKcaj4uR2k4pE2kWQlO3vB1LwtEtZP8TaJe1mZ70R4PZSEAPMVtPYHjnKOWmHzQZC MtlAPxdc22Hcy0kIdnnzW8wIWH2HRAeYrELFNcvYABJTIqaJ40Yd+cIm6X3Z8LFmWdSBbqN3+Vl yTZMXe35v+eVnUFxzdSckla5akYiJbCaSyqW3LLptsjZCGBOsuHQx4iwsEWBtFOHz21iO3Jm6Aq VwPwqf5aFz7fCGpjoX3fhuyuaqgPXXiJTUNcecyDRklQFr0Ks2PSC525stiZiMUxMqgq1Er3wUi nQn0s42mj2L2a31VCKBWH8/WvZoAmHjE4jKzJBxt4/kEGrAff10gp6ORDgZmwiI4jU42OcH1Utl XmlGuJPVI8m84J25/lJYNRu60Jkv1qRCTsv+ukGuvslJhzjh2JSQN0rKYulPSUXvEoiheRNQHJ/ FYTGqLA3IiFdIEIFWOAkg3UV7gixdvxFWpmwhnCFo9i4+GdN9r8iIrZc0HGK1FQnpWt/vMn0nAD xU6TceGszYZWxYw== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a Colour Correction Matrix algorithm that uses the new libipa implementation. The module isn't well documented in the kernel but the default values from the driver suggest a Q3.13 format. Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v3: - None Changes in v2: - Fixed includes - Documentation comment rework - Some styling changes --- src/ipa/ipu3/algorithms/ccm.cpp | 116 ++++++++++++++++++++++++++++++++++++ src/ipa/ipu3/algorithms/ccm.h | 48 +++++++++++++++ src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipa_context.cpp | 10 ++++ src/ipa/ipu3/ipa_context.h | 3 + 5 files changed, 178 insertions(+) diff --git a/src/ipa/ipu3/algorithms/ccm.cpp b/src/ipa/ipu3/algorithms/ccm.cpp new file mode 100644 index 0000000000000000000000000000000000000000..105d815b03c45b4138b069a2f094d5ae4966c299 --- /dev/null +++ b/src/ipa/ipu3/algorithms/ccm.cpp @@ -0,0 +1,116 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * IPU3 Colour correction matrix algorithm + */ + +#include "ccm.h" + +/** + * \file ccm.h + */ + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +/** + * \class Ccm + * \brief The IPU3 color correction matrix algorithm + */ + +LOG_DEFINE_CATEGORY(IPU3Ccm) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init(IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.colourTemperature); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Ccm::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* Nothing to do here, the ccm will be calculated in prepare() */ + if (frameContext.awb.autoEnabled) + return; + + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(ipu3_uapi_params *params, IPAFrameContext &context) +{ + const Matrix &matrix = context.ccm.ccm; + const Matrix &offsets = context.ccm.offsets; + + params->use.acc_ccm = 1; + + params->acc_param.ccm.coeff_m11 = Q<3, 13>(matrix[0][0]).quantized(); + params->acc_param.ccm.coeff_m12 = Q<3, 13>(matrix[0][1]).quantized(); + params->acc_param.ccm.coeff_m13 = Q<3, 13>(matrix[0][2]).quantized(); + params->acc_param.ccm.coeff_o_r = offsets[0][0]; + + params->acc_param.ccm.coeff_m21 = Q<3, 13>(matrix[1][0]).quantized(); + params->acc_param.ccm.coeff_m22 = Q<3, 13>(matrix[1][1]).quantized(); + params->acc_param.ccm.coeff_m23 = Q<3, 13>(matrix[1][2]).quantized(); + params->acc_param.ccm.coeff_o_g = offsets[1][0]; + + params->acc_param.ccm.coeff_m31 = Q<3, 13>(matrix[2][0]).quantized(); + params->acc_param.ccm.coeff_m32 = Q<3, 13>(matrix[2][1]).quantized(); + params->acc_param.ccm.coeff_m33 = Q<3, 13>(matrix[2][2]).quantized(); + params->acc_param.ccm.coeff_o_b = offsets[2][0]; + + LOG(IPU3Ccm, Debug) << "Setting matrix " << matrix; + LOG(IPU3Ccm, Debug) << "Setting offsets " << offsets; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, ipu3_uapi_params *params) +{ + if (!frameContext.awb.autoEnabled) { + setParameters(params, frameContext); + return; + } + + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, frame, + frameContext.awb.colourTemperature); + + setParameters(params, frameContext); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/ccm.h b/src/ipa/ipu3/algorithms/ccm.h new file mode 100644 index 0000000000000000000000000000000000000000..9103de54b32b795d0efd22fad15ca6defb194057 --- /dev/null +++ b/src/ipa/ipu3/algorithms/ccm.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * IPU3 Colour correction matrix algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include +#include + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +class Ccm : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ipu3_uapi_stats_3a *stats, + ControlList &metadata) override; + +private: + void setParameters(ipu3_uapi_params *params, IPAFrameContext &context); + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::ipu3::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index b70a551cacf6cbdedf3ab41771a34ddcdae9fd8b..3dafd2fda9897942cf87d9640665c4fcff383859 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -5,5 +5,6 @@ ipu3_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'ccm.cpp', 'tone_mapping.cpp', ]) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 5cd8fedf27870474eb35745520206de885006b4a..5f5ac92955869898f072f5228524ca7ec8a02839 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -116,6 +116,11 @@ namespace libcamera::ipa::ipu3 { * \brief Active auto-white balance parameters for the IPA */ +/** + * \var IPAActiveState::ccm + * \brief Active colour Correction Matrix parameters for the IPA + */ + /** * \var IPAActiveState::toneMapping * \brief Context for ToneMapping and Gamma control @@ -152,4 +157,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame auto-white balance parameters for the IPA */ +/** + * \var IPAFrameContext::ccm + * \brief Per-frame colour Correction Matrix parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index d2673b42bebb36dde06a795b6c1fbed6e9052535..5476a8ad0256dbb04d33fcd591b72d4b56a52897 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -20,6 +20,7 @@ #include #include #include +#include #include namespace libcamera { @@ -49,6 +50,7 @@ struct IPAActiveState { agc::ActiveState agc; ipa::awb::ActiveState awb; + ipa::ccm::ActiveState ccm; struct { double gamma; @@ -64,6 +66,7 @@ struct IPAFrameContext : public FrameContext { agc::FrameContext agc; ipa::awb::FrameContext awb; + ipa::ccm::FrameContext ccm; }; struct IPAContext { From patchwork Wed Sep 2 09:07:24 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28168 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B98FFC3348 for ; Wed, 2 Sep 2026 09:07:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DFEEF685CC; Wed, 2 Sep 2026 11:07:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="arWr27o2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DCA19685A2 for ; Wed, 2 Sep 2026 11:07:35 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DCE312803; Wed, 2 Sep 2026 11:06:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339965; bh=499CfkMF/aTeKT2CH8GZ2aIYBtBh4VcFCEJxEJKHD1Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=arWr27o2GuO6eKHhWJRm+EB320MQ9m0rut0vcdWVOuCoRC2pjqiDUcBHeJixmOzGX PdkcL5d168h4PbkJSXR3KXXc6THmUMfX/6LjE9jy8BWc1dMfXagT7QSSxGINHf7/M6 Q26OgCg7vZzQTIaGCqodDIs4xYbgvJQIApSL+T6s= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:24 +0100 Subject: [PATCH v3 06/10] ipa: libipa: Add GammaAlgorithm class MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-6-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham , Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12179; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=499CfkMF/aTeKT2CH8GZ2aIYBtBh4VcFCEJxEJKHD1Y=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dUILr0e/wlQQwrJ966gUtDi0BHWIn5iobo7 ToUDf7VsEyJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnVAAKCRDISVd6bEV1 Mh7YEACfiZ+GCncK5+UDmH0DFbtYcOEy1Luzsz4xRZ7VMrCo07N4/z0E0/wYYuCWe3l1nWYcGxl Qu6Ka2OpohrSWgja6aaFj/M7GkEeKkbRRPdnPIyNHfkkHvxTNSbmlMU/ZJDShfY4Alxtrt/PJ3+ VySDXflVNQejs7G64EBWlKODsAU/vffG2kyhFWnjz0xKJP65NtOuQzrZmSEa6NiSc0HE8E4WaP1 vOcCvCgvOcJusR5zD7V1PO7Jdm9q3wvbyCxVcPrMOG19I5t0j9wcHHYW6JYNOwBZ/8jehKuvO5H oBiWS1csH1M85tDvbabzOo2QAkGhUhMaALVK3tn9AZziVmgzHDakEDGggNFX6CZy6If5zCnOXE+ JGi9ZjaKIHxFZMPOirWDYJ/3OfQRyw2uu/bgrwm0wrTWTnOwra1vOuqmKQKfxz7xIc4rrtE/dh6 JgRMT0SsF27i2ly8m+Akyne9sSzxA23xOXz3H8wJvRQmtPDZF+f2T7xeVfCZPtH0i+WfJARzRFy kXRXRHlho8MTQ6VtwPWCF4wJlE9FJXZre4g58lFx+t4XPH16n2B+80c18Aofl8gUJxDd3I4rtZh ixYS84siWPUCGLb3G+bLEwP/ftEO2p1tPGAdhYRDUarW3kG1zQ6iskHQrb/nOYucifzPwxtjeA6 jlV0h03Er5xnmjA== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a base GammaAlgorithm class that can be used by IPA specific gamma algorithms to reduce the amount of work that they need to implement. Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v3: - Require a size template parameter for the Span passed to prepare() Changes in v2: - Updated documentation comments, and minor styling changes --- src/ipa/libipa/gamma.cpp | 257 +++++++++++++++++++++++++++++++++++++++++++++ src/ipa/libipa/gamma.h | 94 +++++++++++++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 353 insertions(+) diff --git a/src/ipa/libipa/gamma.cpp b/src/ipa/libipa/gamma.cpp new file mode 100644 index 0000000000000000000000000000000000000000..ed4fcefb81df51ee13bef6a91150c4b1487ab863 --- /dev/null +++ b/src/ipa/libipa/gamma.cpp @@ -0,0 +1,257 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * + * libIPA Gamma correction algorithm + */ + +#include "gamma.h" + +#include + +#include + +#include "libcamera/internal/value_node.h" + +/** + * \file gamma.h + * \brief libipa implementation of a gamma curve correction algorithm + */ + +namespace libcamera { + +namespace ipa { + +LOG_DEFINE_CATEGORY(Gamma) + +namespace gamma { + +/** + * \struct ActiveState + * \brief Active gamma correction algorithm state + * + * \var ActiveState::gamma + * \brief The gamma correction value + */ + +/** + * \struct FrameContext + * \brief Per-frame gamma correction settings + * + * \var FrameContext::gamma + * \brief The gamma correction value applied for this frame + * + * \var FrameContext::update + * \brief A flag instructing the algorithm to push an update to the hardware + */ + +} /* namespace gamma */ + +/** + * \brief The default gamma correction value + */ +const float kDefaultGamma = 2.2f; + +/** + * \class GammaAlgorithmBase + * \brief Base class for GammaAlgorithm to implement non-templated functions + * + * This base class for GammeaAlgorithm allows us to implement non templated + * functions. IPA specific implementations shall derive from GammaAlgorithm and + * not this class. + */ + +/** + * \fn GammaAlgorithmBase::GammaAlgorithmBase + * \brief Construct an instance of the class + * \param[in] nLutNodes Set the number of function knee-points expected by the + * IPA algorithm + */ + +/** + * \brief Initialise the algorithm with the given tuning data + * \param[out] controls The ControlList into which this algorithm's supported + * controls will be emplaced. + * \param[in] tuningData The tuning data to use with the algorithm + * \param[in] segments A vector of segment spacings to define a custom + * X coordinate system for the curve + * + * Parse \a tuningData and \a segments to initialize the gamma correction curve. + * The tuning data may contain a default gamma value to use; otherwise the value + * of \a kDefaultGamma will be taken as the default. The piecewise linear + * function will be applied on a number of knots whose position is described by + * the optional \a segments argument, which describes each segment's relative + * length. + * + * For example, if the gamma correction has to be applied on 16 equally spaced + * sampling points, a \a segments array like: + * + * [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + * + * would result in evenly spaced knee-points along the X-axis. + * + * Hardware may expect the knee-points to be spaced more densely towards the + * start of the curve and more sparsely towards the end, in which case an + * alternative array might be: + * + * [1, 1, 1, 1, 2, 2, 2, 2, 4, 4, 4, 8, 8, 8, 8, 8] + * + * As the values in \a segments represent the distance between two knee-points + * relative to the total distance between the first and last point, the length + * of \a segments should be equal to the number of knee-points minus one. + *; + * If an IPA implementation doesnt't provide \a segments, the GammaAlgorithm + * class consturcts an evenly-spaced default. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::init() + * + * @return 0 on success, a negative error code otherwise + */ +int GammaAlgorithmBase::init(ControlInfoMap::Map &controls, const ValueNode &tuningData, + Span segments) +{ + /* + * If the caller doesn't pass in a segment list we simply construct one + * with equally spaced segments. We need one less segment than we have + * LUT nodes. + */ + unsigned int expectedNSegments = nLutNodes_ - 1; + + if (segments.empty()) { + for (unsigned int i = 0; i < expectedNSegments; i++) + segments_.push_back(1); + } else { + segments_.assign(segments.begin(), segments.end()); + } + + if (segments_.size() != expectedNSegments) + return -EINVAL; + + segmentsSum_ = std::accumulate(segments_.begin(), segments_.end(), 0.0f); + + defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); + controls[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + + return 0; +} + +/** + * \brief Configure the gamma correction algorithm + * \param[out] state The gamma correction algorithm's active state + * + * Reset to the default gamma correction value. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::configure() + */ +void GammaAlgorithmBase::configure(gamma::ActiveState &state) +{ + state.gamma = defaultGamma_; +} + +/** + * \brief Queue a request to the gamma correction algorithm + * \param[in] state The algorithm's active state + * \param[in] frame The current frame number + * \param[in] context The algorithm's frame context + * \param[in] controls The ControlList that was queued with the request + * + * Queue a new request to the gamma correction algorithm and handle any relevant + * controls that were queued. The only control currently handled is: + * + * - controls::Gamma + * + * If a control with that ID is queued the value is stored in \a state and + * \a context. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::queueRequest() + */ +void GammaAlgorithmBase::queueRequest(gamma::ActiveState &state, + const uint32_t frame, + gamma::FrameContext &context, + const ControlList &controls) +{ + if (frame == 0) + context.update = true; + + const auto &gamma = controls.get(controls::Gamma); + if (gamma) { + state.gamma = *gamma; + context.update = true; + LOG(Gamma, Info) << "Set gamma to " << *gamma; + } + + context.gamma = state.gamma; +} + +/** + * \brief Populate metadata with the gamma correction values for a frame + * \param[in] context The frame context + * \param[out] metadata The ControlList of metadata for a frame + * + * Report the gamma value used to calculate the correction curve that was + * applied to a frame. + */ +void GammaAlgorithmBase::process(gamma::FrameContext &context, ControlList &metadata) +{ + metadata.set(controls::Gamma, context.gamma); +} + +/** + * \var GammaAlgorithmBase::nLutNodes_ + * \brief The number of gamma LUT sampling points + */ + +/** + * \var GammaAlgorithmBase::defaultGamma_ + * \brief The default gamma parameter + */ + +/** + * \var GammaAlgorithmBase::segments_ + * \brief The vector of segment sizes describing the space between knee-points + */ + +/** + * \var GammaAlgorithmBase::segmentsSum_ + * \brief The sum of \a GammaAlgorithmBase::segments_ + */ + +/** + * \class GammaAlgorithm + * \brief The libipa gamma correction algorithm + * \tparam NLutNodes The number of gamma LUT sampling points + * \tparam UQ The fixedpoint representation of the gamma correction values + * + * Gamma correction adjusts for the differences in the way light is perceived + * by a camera and the human eye by applying a function to the input values. + * The GammaAlgorithm class facilitates this by building a piecewise linear + * function from a gamma parameter and supplying it in the hardware-specific + * formats defined by the IPA algorithms. + * + * IPA modules are expected to store an instance of GammaAlgorithm as a class + * member, templated with the format and number of knee-points in the PWL + * expected by their hardware and then call its functions in their overload of + * the Algorithm class's function. + * + * When an application queues a new value for the gamma parameter with a + * Request, the GammaAlgorithm will recalculate and populate the new LUT to be + * sent to the ISP. + * + * Useful links: + * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm + * - https://en.wikipedia.org/wiki/SRGB + */ + +/** + * \fn GammaAlgorithm::prepare() + * \tparam T The type of data expected by the hardware's look-up table + * \param[in] context The frame context + * \param[out] lut The Span into which to place the calculated look-up table + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/gamma.h b/src/ipa/libipa/gamma.h new file mode 100644 index 0000000000000000000000000000000000000000..5d3556ac291abfb05b6c1aa89e35116f829466f3 --- /dev/null +++ b/src/ipa/libipa/gamma.h @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * + * libIPA Gamma correction algorithm + */ + +#pragma once + +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "fixedpoint.h" + +namespace libcamera { + +namespace ipa { + +LOG_DECLARE_CATEGORY(Gamma) + +namespace gamma { + +struct ActiveState { + double gamma; +}; + +struct FrameContext { + double gamma; + bool update; +}; + +} /* namespace gamma */ + +class GammaAlgorithmBase +{ +public: + GammaAlgorithmBase(unsigned int nLutNodes) + : nLutNodes_(nLutNodes) + { + } + + int init(ControlInfoMap::Map &controls, const ValueNode &tuningData, + Span segments = {}); + + void configure(gamma::ActiveState &state); + void queueRequest(gamma::ActiveState &state, const uint32_t frame, + gamma::FrameContext &context, const ControlList &controls); + void process(gamma::FrameContext &context, ControlList &metadata); + +protected: + unsigned int nLutNodes_; + float defaultGamma_; + std::vector segments_; + unsigned int segmentsSum_; +}; + +template +class GammaAlgorithm : public GammaAlgorithmBase +{ +public: + GammaAlgorithm() + : GammaAlgorithmBase(NLutNodes) + { + } + + template + void prepare(gamma::FrameContext &context, Span lut) + { + float x = 0; + + for (unsigned int i = 0; i < nLutNodes_; i++) { + float gamma = std::pow(x / segmentsSum_, + 1.0 / context.gamma); + lut[i] = UQ(gamma).quantized(); + + LOG(Gamma, Debug) << "LUT[" << i << "]=" << gamma + << "(" << lut[i] << ")"; + + if (i < segments_.size()) + x += segments_[i]; + } + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 0fb65a23931bdb3430f76843cd3a8d18fb5083d6..a38d5d528aac964d25c9ec2a06e10f6c4ed74494 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -14,6 +14,7 @@ libipa_headers = files([ 'exposure_mode_helper.h', 'fc_queue.h', 'fixedpoint.h', + 'gamma.h', 'histogram.h', 'interpolator.h', 'lsc.h', @@ -42,6 +43,7 @@ libipa_sources = files([ 'exposure_mode_helper.cpp', 'fc_queue.cpp', 'fixedpoint.cpp', + 'gamma.cpp', 'histogram.cpp', 'interpolator.cpp', 'lsc.cpp', From patchwork Wed Sep 2 09:07:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28169 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52435C3344 for ; Wed, 2 Sep 2026 09:07:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8843F685C4; Wed, 2 Sep 2026 11:07:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EAWkJhyh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F048685A4 for ; Wed, 2 Sep 2026 11:07:36 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 31F912810; Wed, 2 Sep 2026 11:06:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339965; bh=Wv9q4ZF76yWqnt++oMvOuYIBUnnWfgPYHyDRjvdtYNQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=EAWkJhyh8SCv9PKYgXASc5S7TQ0r3ZG4sdMSvNl1cN2NOYo/B3ccc49xo7e3ptrDC gCSXVyil7TZ3nITm/HPuSunFelbeAAejNb9iDMUxFSevArB0NFrscuu3OuMaWOu9/6 iZtDUDvNTdStL7jiFGFObDls9vwH+92T1CeOY8w4= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:25 +0100 Subject: [PATCH v3 07/10] ipa: ipu3: ToneMapping: Convert to use GammaAlgorithm MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-7-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8333; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=Wv9q4ZF76yWqnt++oMvOuYIBUnnWfgPYHyDRjvdtYNQ=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dUiZx8zXR4wVHNps/80wkWdbn6jZAzJCJvX sFVdaEcUy+JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnVAAKCRDISVd6bEV1 Mt7rD/9HBXbRqimA6w387HYyoYmmBFViE0t8y1Z7SCJEb3HyDMEm/HV7B1CGeIo/4VgYWWf10Ir oy3fDiON2r34Ba6ZaKrcm2zJr73ZvKytRTKZtFqTetjQmHdwD+MrsQikZIacwThTMMcpmJoypx/ Hit0iFi6BHMyHupcClngF3FdWsHiW0ybzIQ2HJk5oK/e6cqLh23vio0zxk/UZyQkBZUXXDHNGB2 AVk2kCczlJmd7RAHIkJ77Z/BnGxKHhMqHthU8lNGAwF7oDquaE6SSxwzOZpwyWdpitnQnMNPjpA RGbTFsmb7uROu8auE8QJu5XIU8/klcHK+CBb5BTUylONXPXWfoFAoHOvxtHZybweqPwP9fqb3Rf NINY9zfbO+aSsHJOiTopqKq1qRSWLTG6fuwyXd31iBY9HhXvHQSL9LLKJGGXG1TnNrdpUPFukS3 CHp8/6Y+0BjSD2UmaYFwm8PjdzjpRqVP/72RCL/MqcUi6Ctx8Hqi6IKlV4A9rc1vYyQbGSRg9IE WOIKVV+d0EMfpSj830Gb0TEoAlRwY/QtqmkLa8gRoxyLnWQ3HCuoOGfQ0Pag3zSxVkgKSd6kxk9 4997vqTY83o8cHmrjPzvGJ4fV5ZnlYkXd3o345+QLiZTvQ5tRkwl3zOU0ffTU4DRODeWYVzIkaL 6TcA9I03AMka/+w== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Convert the IPU3 ToneMapping algorithm to use the new GammaAlgorithm base class. This gives us configurable gamma via the tuning files and at runtime using the Gamma control. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v3: - Construct Span with size template param Changes in v2: - Fixed includes - Minor styling changes --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 73 +++++++++++++++++--------------- src/ipa/ipu3/algorithms/tone_mapping.h | 12 +++++- src/ipa/ipu3/ipa_context.cpp | 18 +++----- src/ipa/ipu3/ipa_context.h | 8 ++-- 4 files changed, 60 insertions(+), 51 deletions(-) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 160338c139448cc9a0bc1fe2400c335a96f68f73..faa4853a27a23b911acccf8c41b8fdcc1524dbcd 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -10,6 +10,8 @@ #include #include +#include + /** * \file tone_mapping.h */ @@ -27,10 +29,17 @@ namespace ipa::ipu3::algorithms { */ ToneMapping::ToneMapping() - : gamma_(1.0) { } +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int ToneMapping::init(IPAContext &context, const ValueNode &tuningData) +{ + return gammaAlgo_.init(context.ctrlMap, tuningData); +} + /** * \brief Configure the tone mapping given a configInfo * \param[in] context The shared IPA context @@ -41,12 +50,21 @@ ToneMapping::ToneMapping() int ToneMapping::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { - /* Initialise tone mapping gamma value. */ - context.activeState.toneMapping.gamma = 0.0; - + gammaAlgo_.configure(context.activeState.gamma); return 0; } +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void ToneMapping::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + gammaAlgo_.queueRequest(context.activeState.gamma, frame, + frameContext.gamma, controls); +} + /** * \brief Fill in the parameter structure, and enable gamma control * \param[in] context The shared IPA context @@ -59,14 +77,21 @@ int ToneMapping::configure(IPAContext &context, */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, ipu3_uapi_params *params) { - /* Copy the calculated LUT into the parameters buffer. */ - memcpy(params->acc_param.gamma.gc_lut.lut, - context.activeState.toneMapping.gammaCorrection.lut, - IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES * - sizeof(params->acc_param.gamma.gc_lut.lut[0])); + if (!frameContext.gamma.update) + return; + + /* + * Unfortunately necessary given the IPU3's gamma uAPI struct has the + * __packed attribute. + */ + uint16_t *lutData = reinterpret_cast( + __builtin_assume_aligned(params->acc_param.gamma.gc_lut.lut, 16)); + Span lut{ lutData, kNumLutNodes }; + + gammaAlgo_.prepare(frameContext.gamma, lut); /* Enable the custom gamma table. */ params->use.acc_gamma = 1; @@ -84,33 +109,13 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, +void ToneMapping::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats, [[maybe_unused]] ControlList &metadata) { - /* - * Hardcode gamma to 1.1 as a default for now. - * - * \todo Expose gamma control setting through the libcamera control API - */ - gamma_ = 1.1; - - if (context.activeState.toneMapping.gamma == gamma_) - return; - - struct ipu3_uapi_gamma_corr_lut &lut = - context.activeState.toneMapping.gammaCorrection; - - for (uint32_t i = 0; i < std::size(lut.lut); i++) { - double j = static_cast(i) / (std::size(lut.lut) - 1); - double gamma = std::pow(j, 1.0 / gamma_); - - /* The output value is expressed on 13 bits. */ - lut.lut[i] = gamma * 8191; - } - - context.activeState.toneMapping.gamma = gamma_; + gammaAlgo_.process(frameContext.gamma, metadata); } REGISTER_IPA_ALGORITHM(ToneMapping, "ToneMapping") diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index b2b380108e014b3d5ee7b93bcdd948dea4d2302d..3437b7fcbbb70c9bbc895b514b354d26c1d707c6 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -7,6 +7,11 @@ #pragma once +#include + +#include +#include + #include "algorithm.h" namespace libcamera { @@ -18,7 +23,11 @@ class ToneMapping : public Algorithm public: ToneMapping(); + int init(IPAContext &context, const ValueNode &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, const uint32_t frame, @@ -27,7 +36,8 @@ public: ControlList &metadata) override; private: - double gamma_; + static constexpr unsigned int kNumLutNodes = IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES; + GammaAlgorithm> gammaAlgo_; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 5f5ac92955869898f072f5228524ca7ec8a02839..02e5f79fa5def888fe3776e5cb002771a497ee41 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -122,17 +122,8 @@ namespace libcamera::ipa::ipu3 { */ /** - * \var IPAActiveState::toneMapping - * \brief Context for ToneMapping and Gamma control - * - * \var IPAActiveState::toneMapping.gamma - * \brief Gamma value for the LUT - * - * \var IPAActiveState::toneMapping.gammaCorrection - * \brief Per-pixel tone mapping implemented as a LUT - * - * The LUT structure is defined by the IPU3 kernel interface. See - * struct ipu3_uapi_gamma_corr_lut for further details. + * \var IPAActiveState::gamma + * \brief Active gamma correction parameters for the IPA */ /** @@ -162,4 +153,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame colour Correction Matrix parameters for the IPA */ +/** + * \var IPAFrameContext::gamma + * \brief Per-frame gamma correction parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 5476a8ad0256dbb04d33fcd591b72d4b56a52897..1755fc62a89152470bd059c6eb6b6a366e451ba6 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -22,6 +22,7 @@ #include #include #include +#include namespace libcamera { @@ -51,11 +52,7 @@ struct IPAActiveState { agc::ActiveState agc; ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; - - struct { - double gamma; - struct ipu3_uapi_gamma_corr_lut gammaCorrection; - } toneMapping; + ipa::gamma::ActiveState gamma; }; struct IPAFrameContext : public FrameContext { @@ -67,6 +64,7 @@ struct IPAFrameContext : public FrameContext { agc::FrameContext agc; ipa::awb::FrameContext awb; ipa::ccm::FrameContext ccm; + ipa::gamma::FrameContext gamma; }; struct IPAContext { From patchwork Wed Sep 2 09:07:26 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28170 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C804EC3349 for ; Wed, 2 Sep 2026 09:07:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 467C9685C1; Wed, 2 Sep 2026 11:07:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oY4GHqhd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7FF5C685AC for ; Wed, 2 Sep 2026 11:07:36 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 79B9A1CF3; Wed, 2 Sep 2026 11:06:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339965; bh=G1YCe4uB23UyRQXSnXc/2fn0AhCSVjx2NICcX0/4uKo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=oY4GHqhdGXA6ih5OabZIY/zhraAcz/2fgPUuHZw//4qGwf73IAyCnH44Kl4sxRqSK h6Qt5teca+bunKxEfAwK/R5txdgHz3dHEb16Lzuy89UtcnXzru/viij/t9y/YARx8A M04AOIIL8j4BLyKvu9xmld7ieCkNvY6A6DSWImx4= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:26 +0100 Subject: [PATCH v3 08/10] ipa: ipu3: Add Ccm to uncalibrated.yaml MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-8-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=867; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=G1YCe4uB23UyRQXSnXc/2fn0AhCSVjx2NICcX0/4uKo=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dUw3IjfDkQSKtfUiRLHaxN2SxAKb454y1u6 AFB+cjVVS6JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnVAAKCRDISVd6bEV1 Mj1ND/9wlAh2d9TomW8XBpnz73N/GIJE2CqCgkZaI2aNxUBaMNlRUE61EKwpRC3b8VRSfpDKdVk A7goH/4FB+ERCbQ29aXUrRSY3btoMxaXs+4TOxypYuiQY0Rl5Dx9x4FRKmCjwH/FI7Ix1fPQmWL 7u1i4DrIS7EzLkBBv/cmdaXIPSQunztePRXcyg2G/bti83MwuDQjUgvTdeE/rH6RzHp8MdizkHJ TTFRFCeYSJgLltZFx9wh5CIgSnB00YM6f1DdfuIMBqXZOzq/b6SNPFIk8Q3ZljUISBrBfWPQyTC 9dpS5R5wWRQowRvg4xyv0GOiqkvxa/laQdW5IdSSR9m6UaGv/KTDG53Mmk63ds3SvHpuYwpAoiY iQactlDKuRC/X9jZkeJ51CoBHaJ6KvtLj4i8ohCsGRoQHR2XRymttxcIt+HVBNrlr6R2ITjP4fH CROftrl+MOK0Z6y+96QDW9xx1fCytD+e9PreJAWdtru/FkO4yQGq0v27EpvLbSyJOETaETzPtY0 edkfefN4TAFwlOUNySBNov85IlmNRgnW59c43NvrprQMbSpbKqPGNylTrwIcEo/OOmaD6XdiXFl lmvcVqrnuZL+jXBfuQzXImM7Jew7FJesYoY387tmKE3xaRELgpgps94jQrEoEW0HlqRYV8J1/Ps OMXcnfE8CTBB9aQ== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the Ccm algorithm to uncalibrated.yaml. Even without calibrated values we should still expose the controls to the application. Reviewed-by: Jacopo Mondi Acked-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v3: - None Changes in v2: - None --- src/ipa/ipu3/data/uncalibrated.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ipa/ipu3/data/uncalibrated.yaml b/src/ipa/ipu3/data/uncalibrated.yaml index 794ab3edbb778bbe14c2dc65e5540413ee7fd536..415ca4578fab57ea731493c978a04e740946197f 100644 --- a/src/ipa/ipu3/data/uncalibrated.yaml +++ b/src/ipa/ipu3/data/uncalibrated.yaml @@ -7,5 +7,6 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: + - Ccm: - ToneMapping: ... From patchwork Wed Sep 2 09:07:27 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28171 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3B218C334A for ; Wed, 2 Sep 2026 09:07:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3AA88685B0; Wed, 2 Sep 2026 11:07:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nXNKMqHz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C60B6685A8 for ; Wed, 2 Sep 2026 11:07:36 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C4E8E2821; Wed, 2 Sep 2026 11:06:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339965; bh=uJ/h3KM1E6DFxQFTUTEN1y8pPZHfIAQybtcZ3ek7JzI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nXNKMqHzNvTmBkY34jtyeWZpGKt7SoS90jjhJ4ZiM6iHDNsiL1oH9NXckhZ/hEg4X DsYjFW10PVaRQXhxUBK92Ghd4LNMtTJ/3SuZSYncXf7vTIuo00adSXrmuc9dCI4nV7 Ir/FShLVK7GE8NblT2EOQTBAL35YRtx4NEiLIZJI= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:27 +0100 Subject: [PATCH v3 09/10] ipa: rkisp1: goc: Re-work to use GammaAlgorithm class MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-9-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6828; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=uJ/h3KM1E6DFxQFTUTEN1y8pPZHfIAQybtcZ3ek7JzI=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dV6MSmXfU8uQC2dtmi1ILJaRXxkOS507BMJ QtvjYRC0oaJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnVQAKCRDISVd6bEV1 MhQKEACBwk5rmo2Gs3K0sOetw5RYQQZr6a2D9u73kauEwOiuprHDTNzp1E5x2Ixcf1Ma0ra1cmp v2TD7xz4BbqG4k0Odbc5FW/Czrx3ZuhWbyeZt1Of6u2kPbOUMNeMUaPHXKn2rKefegI5v/bdb2H OxSd/kvgEfSTX5qs8VleLV5IQeVMDWq5E6v+comcuGOzGsqfJC4bZ9MoYGyqUcve5KOoJq12Tjo MJgajzSvvRuqKUv0+ICooQegAPCrH+OfBAwODGDE9c3cuL/0uA/mtsXwJ6Ry8Y0SuigQkTOoCHP pafesOlI27obDuusxTgv/0TNaRf25/+uZi0/KkJXGfH71Nl26j1AxVDxt02PejAxQ7tL98ERaeM yWCOQDwLLN5ye0YFKyZkE2FWwPIf7nvnATAKi4omFWijoVEalYDZne/M+idBsI6Xu0DNvk7D6fp hPG1RmvlrWanfdB8j+aikUoP4Nr8zzzRubchQ2lLoFq17OtFpTpGlOz0PxtRw683sDr9uHr1GXB zpv8jAGbDF2P0TfnFFvCUC/eTpX6JAozz5fOMaCLMjE66MwSNBXuf+nLKuIq95iLDhF5PxxoD3u ovrfH9nzbi3lyYMIp+71mUJ8dDRvh66t6xuTcPVQL8XlOKN1DoXW4uz9MgOIumLOcoAXol9d9/u Bmg3/Tb5pQhfaPg== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Re-work the RkISP1 Gamma Out Correction algorithm to use the new GammaAlgorithm class from libipa, which allows us to share an implementation with the other IPAs. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v3: - None Changes in v2: - Fixed includes - Dropped the comment explaining how gamma correction is implemented - it's on libipa gamma now - Dropped unused variable kDefaultGamma --- src/ipa/rkisp1/algorithms/goc.cpp | 65 ++++++++++++--------------------------- src/ipa/rkisp1/algorithms/goc.h | 7 ++++- src/ipa/rkisp1/ipa_context.h | 10 ++---- 3 files changed, 28 insertions(+), 54 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/goc.cpp b/src/ipa/rkisp1/algorithms/goc.cpp index e8f64bf3d5e028c2013d37b7ef9fe90b737fad5f..dc08b6ba972e3f3390bdae862b4935396313c4a7 100644 --- a/src/ipa/rkisp1/algorithms/goc.cpp +++ b/src/ipa/rkisp1/algorithms/goc.cpp @@ -6,10 +6,11 @@ */ #include "goc.h" +#include #include #include -#include +#include #include @@ -29,21 +30,13 @@ namespace ipa::rkisp1::algorithms { * \class GammaOutCorrection * \brief RkISP1 Gamma out correction * - * This algorithm implements the gamma out curve for the RkISP1. It defaults to - * a gamma value of 2.2. - * - * As gamma is internally represented as a piecewise linear function with only - * 17 knots, the difference between gamma=2.2 and sRGB gamma is minimal. - * Therefore sRGB gamma was not implemented as special case. - * - * Useful links: - * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm - * - https://en.wikipedia.org/wiki/SRGB + * This algorithm implements the gamma out curve for the RkISP1 using the + * libipa GammaAlgorithm class. */ LOG_DEFINE_CATEGORY(RkISP1Gamma) -const float kDefaultGamma = 2.2f; +static constexpr unsigned int kNumLutSegments = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10 - 1; /** * \copydoc libcamera::ipa::Algorithm::init @@ -57,10 +50,12 @@ int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) return -EINVAL; } - defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); - context.ctrlMap[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + std::array segments = { + 64, 64, 64, 64, 128, 128, 128, 128, + 256, 256, 256, 512, 512, 512, 512, 512 + }; - return 0; + return gammaAlgo_.init(context.ctrlMap, tuningData, segments); } /** @@ -69,7 +64,7 @@ int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) int GammaOutCorrection::configure(IPAContext &context, [[maybe_unused]] const IPACameraSensorInfo &configInfo) { - context.activeState.goc.gamma = defaultGamma_; + gammaAlgo_.configure(context.activeState.gamma); return 0; } @@ -80,17 +75,8 @@ void GammaOutCorrection::queueRequest(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const ControlList &controls) { - if (frame == 0) - frameContext.goc.update = true; - - const auto &gamma = controls.get(controls::Gamma); - if (gamma) { - context.activeState.goc.gamma = *gamma; - frameContext.goc.update = true; - LOG(RkISP1Gamma, Debug) << "Set gamma to " << *gamma; - } - - frameContext.goc.gamma = context.activeState.goc.gamma; + gammaAlgo_.queueRequest(context.activeState.gamma, frame, + frameContext.gamma, controls); } /** @@ -104,29 +90,16 @@ void GammaOutCorrection::prepare(IPAContext &context, ASSERT(context.hw.numGammaOutSamples == RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10); - if (!frameContext.goc.update) + if (!frameContext.gamma.update) return; - /* - * The logarithmic segments as specified in the reference. - * Plus an additional 0 to make the loop easier - */ - static constexpr std::array segments = { - 64, 64, 64, 64, 128, 128, 128, 128, 256, - 256, 256, 512, 512, 512, 512, 512, 0 - }; - auto config = params->block(); config.setEnabled(true); - __u16 *gamma_y = config->gamma_y; - - unsigned x = 0; - for (const auto [i, size] : utils::enumerate(segments)) { - gamma_y[i] = std::pow(x / 4096.0, 1.0 / frameContext.goc.gamma) * 1023.0; - x += size; - } - + Span lut{ + config->gamma_y, RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10 + }; + gammaAlgo_.prepare(frameContext.gamma, lut); config->mode = RKISP1_CIF_ISP_GOC_MODE_LOGARITHMIC; } @@ -139,7 +112,7 @@ void GammaOutCorrection::process([[maybe_unused]] IPAContext &context, [[maybe_unused]] const rkisp1_stat_buffer *stats, ControlList &metadata) { - metadata.set(controls::Gamma, frameContext.goc.gamma); + gammaAlgo_.process(frameContext.gamma, metadata); } REGISTER_IPA_ALGORITHM(GammaOutCorrection, "GammaOutCorrection") diff --git a/src/ipa/rkisp1/algorithms/goc.h b/src/ipa/rkisp1/algorithms/goc.h index bd79fe19cc86b8aefa2603e98e9d7130b44105d9..767d707b2758a89db286ab62d8f20d7a01912a2a 100644 --- a/src/ipa/rkisp1/algorithms/goc.h +++ b/src/ipa/rkisp1/algorithms/goc.h @@ -9,6 +9,11 @@ #include "algorithm.h" +#include + +#include +#include + namespace libcamera { namespace ipa::rkisp1::algorithms { @@ -35,7 +40,7 @@ public: ControlList &metadata) override; private: - float defaultGamma_; + GammaAlgorithm> gammaAlgo_; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index cc07bb9462d7a9d2c5e3f4dd1e89165b664116a4..a6db004c16a5d067110786d84b26a6b86b3b0639 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -30,6 +30,7 @@ #include "libipa/ccm.h" #include "libipa/fc_queue.h" #include "libipa/fixedpoint.h" +#include "libipa/gamma.h" #include "libipa/lsc.h" namespace libcamera { @@ -102,9 +103,7 @@ struct IPAActiveState { uint8_t sharpness; } filter; - struct { - double gamma; - } goc; + ipa::gamma::ActiveState gamma; struct { double lux; @@ -154,10 +153,7 @@ struct IPAFrameContext : public FrameContext { bool update; } filter; - struct { - double gamma; - bool update; - } goc; + ipa::gamma::FrameContext gamma; struct { uint32_t exposure; From patchwork Wed Sep 2 09:07:28 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 28172 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C2786C3347 for ; Wed, 2 Sep 2026 09:07:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 86491685D5; Wed, 2 Sep 2026 11:07:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iwmAamKA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 38342685AA for ; Wed, 2 Sep 2026 11:07:37 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 191C22826; Wed, 2 Sep 2026 11:06:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1788339966; bh=sj2eLPG0Mn4qhE1OJX5QTxSNdnIdJNCc0bBUQL9vtgg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=iwmAamKAMiAq6lbCE/8lekhegMEWMmsZgVgdX8NLMvKLcmdPak4fkwApV1n6dBOMv XB5Q7uLC3TMn++r8K+vV8EPGG7OsuPmfKuB8DGbs/Amh0zH3LOtOYVLayvLLNuALw2 BohNcX4zrss8Fr5yX9tkXSLvQiF5UCJOEBLqcCDM= From: Daniel Scally Date: Wed, 02 Sep 2026 10:07:28 +0100 Subject: [PATCH v3 10/10] ipa: ipu3: Add Lens Shading Correction algorithm MIME-Version: 1.0 Message-Id: <20260902-ipu3-libipa-rework-v3-10-b7da9b062326@ideasonboard.com> References: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> In-Reply-To: <20260902-ipu3-libipa-rework-v3-0-b7da9b062326@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham , Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=15305; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=sj2eLPG0Mn4qhE1OJX5QTxSNdnIdJNCc0bBUQL9vtgg=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBql+dVCFsaSD7Ch32kSFYm9QBxNDrybH/52OSoV 14C+/byjwSJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCapfnVQAKCRDISVd6bEV1 MscCD/4540K0KS5lxIW36VzUds9ZCYjsInmzlgfBJdcHA78lEGVk6mfCC1DIxUXDqVSkV4qk75D YV4i7ec2OpwS9FrrbE84A1lzkKwH20chmWBAPVJrkMqW8ePgA5vAXzRrRCBTIKJ83350VACqzKF z0ETA4bVgwK9tj2EjUPAVWTi3lnPIBakCQMGGVgFyVG53SSxRX4jgco4eCaBdnoOumbklcrdSNJ yV7es64+6Z1ckcIiwQdo0E7X13c9HanbF2LERxdc+e9sONDrTAp47oaNY1OQ8NwI0MVMzMYjGA+ 3ovhNUG9z903b+t/9JWmxE2Xi8fTrqTUmMyVrgV+HeqgqeK2RnTb7aOHtqNOyrBnRExZRA7Ewkz koW8/dK+X4CNRpB9udTGbVceube4PsAZTy3a8jNncm+fmBHqpm9ZbkJEKVghVdDEXlFSyEEdpL7 nyHr+xnYVoWmzsPba8CwpBWgf2q8o3zQp/TrNbWnT9cl/th/CQcBkopZDijhY9OzZC3KT0JUOVY mqAqeUzo/JrnZXNtKi0FLpp7y0nlrlTvlUlZ71H1k7lsT6vYzqtJBfC07y0+9n/t5Gm4CVaIgDL WFW27JudBiVuiK06SMZT0juPHUsPN0yy7R6K2JUdvzS48z909w1jEQIxvb/P7dsIzSiJiC6CkPu wzjD3ZHro1YZVrw== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a lens shading correction algorithm for the IPU3 IPA, using the recently implemented libipa base algorithm class. Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v3: - None Changes in v2: - Fixed includes - Updated some documentation comments - Check tuning file for "type" key, not "polynomial" --- src/ipa/ipu3/algorithms/lsc.cpp | 295 ++++++++++++++++++++++++++++++++++++ src/ipa/ipu3/algorithms/lsc.h | 60 ++++++++ src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipa_context.cpp | 10 ++ src/ipa/ipu3/ipa_context.h | 3 + 5 files changed, 369 insertions(+) diff --git a/src/ipa/ipu3/algorithms/lsc.cpp b/src/ipa/ipu3/algorithms/lsc.cpp new file mode 100644 index 0000000000000000000000000000000000000000..26a44a3ec3db22697c1302b10263713c47ef590d --- /dev/null +++ b/src/ipa/ipu3/algorithms/lsc.cpp @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board, Oy + * + * IPU3 Lens Shading Correction algorithm + */ + +#include "lsc.h" + +#include +#include + +#include + +/** + * \file lsc.h + * \brief IPU3 Lens Shading Correction + */ + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +/** + * \class Lsc + * \brief IPU3 Lens Shading Correction algorithm + */ + +LOG_DEFINE_CATEGORY(IPU3Lsc) + +static constexpr unsigned int kMaxNumHCells = 73; +static constexpr unsigned int kMaxNumVCells = 56; +static constexpr int kColourTemperatureQuantization = 10; + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Lsc::init(IPAContext &context, const ValueNode &tuningData) +{ + /* + * The IPU3 lens shading block expects a table of data that isn't of a + * fixed size, but rather is configurable based on 4 parameters: + * + * block_width_log2: The log2 of the horizontal pixel count per cell + * block_height_log2: The log2 of the vertical pixel count per cell + * width: The number of horizontal cells + * height: The number of vertical cells + * + * The constructed grid should be capable of covering the image, but + * ideally won't extend past the edges of the image. Fixing either set + * of parameters for the algorithm as a whole is likely to result in + * suboptimal situations for some sensors, so let's determine them + * programmatically instead. + * + * What we want is the densest possible grid that ideally doesn't extend + * past the edges of the image at all. The maximum grid size is 73x56, + * which gives us the lower bounds on cell size. Unfortunately we can + * only specify sizes in powers of two, which can have the effect of + * making the grid much more coarse. For example for a 2592x1944 input + * image, 2592 / 73 = 35.5...which means we need to set blockWidthLog2 + * to 6 (I.E. 64) and have just 40.5 (or rather 41) cells horizontally. + */ + sensorWidth_ = context.sensorInfo.activeAreaSize.width; + sensorHeight_ = context.sensorInfo.activeAreaSize.height; + + unsigned int cellWidth = (sensorWidth_ + kMaxNumHCells - 1) / kMaxNumHCells; + unsigned int cellHeight = (sensorHeight_ + kMaxNumVCells - 1) / kMaxNumVCells; + + unsigned int minCellWidth = std::bit_ceil(cellWidth); + unsigned int minCellHeight = std::bit_ceil(cellHeight); + + numHCells_ = (sensorWidth_ + minCellWidth - 1) / minCellWidth; + numVCells_ = (sensorHeight_ + minCellHeight - 1) / minCellHeight; + + blockWidthLog2_ = std::bit_width(minCellWidth) - 1; + blockHeightLog2_ = std::bit_width(minCellHeight) - 1; + + LOG(IPU3Lsc, Debug) << "Calculated Grid configuration: " + << numHCells_ << "x" << numVCells_ << " cells of " + << minCellWidth << "x" << minCellHeight << " pixels"; + + /* + * We need to know if we're running the polynomial algorithm or not as + * things will behave slightly differently. + */ + polynomial_ = tuningData["type"].get(false); + + return lscAlgo_.init(tuningData, context.ctrlMap, + { .keys = { "r", "gr", "gb", "b" }, + .numHSamples = numHCells_, + .numVSamples = numVCells_, + .sensorSize = context.sensorInfo.activeAreaSize }); +} + +std::vector Lsc::calculatePositions(unsigned int dimension) +{ + std::vector positions(dimension); + for (double i = 0.0; i < dimension; i++) + positions[i] = i / (dimension - 1); + + return positions; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Lsc::configure(IPAContext &context, const IPAConfigInfo &configInfo) +{ + cropWidth_ = configInfo.sensorInfo.analogCrop.width; + cropHeight_ = configInfo.sensorInfo.analogCrop.height; + std::vector xPos = calculatePositions(numHCells_); + std::vector yPos = calculatePositions(numVCells_); + + return lscAlgo_.configure(context.activeState.lsc, + configInfo.sensorInfo.analogCrop, xPos, yPos); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Lsc::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* + * The base algorithm defines the LensShadingCorrectionEnable control + * with a default value of true, but actually the IPU3 driver defaults + * it to off. If this is the first frame, check for the control, but if + * there isn't one, force it on to fulfil the advertised default. + */ + if (frame == 0) { + const auto &lscEnable = controls.get(controls::LensShadingCorrectionEnable); + if (!lscEnable) { + frameContext.lsc.enabled = true; + frameContext.lsc.update = true; + } + } + + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, controls); +} + +static unsigned int quantize(unsigned int value, unsigned int step) +{ + return std::lround(value / static_cast(step)) * step; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Lsc::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, ipu3_uapi_params *params) +{ + uint32_t ct = frameContext.awb.colourTemperature; + unsigned int quantizedCt = quantize(ct, kColourTemperatureQuantization); + + if (!frameContext.lsc.update) { + if (!frameContext.lsc.enabled) + return; + + /* + * Add a threshold so that oscillations around a quantization + * step don't lead to constant changes. + */ + if (utils::abs_diff(ct, lastAppliedCt_) < kColourTemperatureQuantization / 2) + return; + + if (quantizedCt == lastAppliedQuantizedCt_) + return; + } + + /* + * This flag tells the kernel driver that it should read the LSC params + * passed from userspace instead of using its cached copy. + */ + params->use.acc_shd = 1; + + /* + * Pass the enabled flag. If we're not enabled, we can then just bail + * out. + */ + ipu3_uapi_shd_config_static *config = ¶ms->acc_param.shd.shd; + config->general.shd_enable = frameContext.lsc.enabled; + + if (!frameContext.lsc.enabled) + return; + + config->grid.width = numHCells_; + config->grid.height = numVCells_; + config->grid.block_width_log2 = blockWidthLog2_; + config->grid.block_height_log2 = blockHeightLog2_; + config->grid.grid_height_per_slice = IPU3_UAPI_SHD_MAX_CELLS_PER_SET / numHCells_; + + /* + * The IPU3's documentation describes the x_start and y_start members + * as follows: + * + * "[X/Y] value of top left corner of sensor relative to ROI + * s13, [-4096, 0], default 0, only negative values." + * + * I interpret that as allowing us to configure the cropped rectangle + * relative to the full grid. That's useful if we're running the tabular + * algorithm, which would otherwise apply the full grid inappropriately. + * If we're running the polynomial one though the calculated grid is + * probably more appropriate than a coarse application of the full grid + * so let's tell the hardware not to bother correcting in that case. + */ + if (polynomial_) { + config->grid.x_start = 0; + config->grid.y_start = 0; + } else { + config->grid.x_start = (cropWidth_ - sensorWidth_) / 2; + config->grid.y_start = (cropHeight_ - sensorHeight_) / 2; + } + + /* No idea what this is, but the docs say it should be set as so */ + config->general.init_set_vrt_offst_ul = + config->grid.y_start >> (config->grid.block_height_log2 % + config->grid.grid_height_per_slice); + + /* + * Values in the LUT cease taking effect at 4096, and a value of 0.0 is + * "no correction" rather than black. The gain factor is described by + * the documentation like so: + * + * "Shift calculated anti shading value. Precision u2. 0x0 - gain factor + * [1, 5], means no shift interpolated value. 0x1 - gain factor [1, 9], + * means shift interpolated by 1. 0x2 - gain factor [1, 17], means shift + * interpolated by 2." + * + * The simplest interpretation for those pieces of information is I + * think that the LUT stores 12-bit Q numbers who's represented values + * depend on the gain_factor setting like so: + * + * 0: UQ<2, 10> representing values in range [0, 4) + * 1: UQ<3, 9> representing values in range [0, 8) + * 2: UQ<4, 8> representing values in range [0, 16) + * + * And that a base gain of 1.0 is added to those configured values. As a + * gain of more than 5.0 is fairly unlikely, let's fix gain_factor to 0 + * for now and revisit if needed. + */ + config->general.gain_factor = 0; + + /* + * Disable the black level settings here - we do that through another + * parameters block. + */ + config->black_level.bl_r = 0; + config->black_level.bl_gr = 0; + config->black_level.bl_gb = 0; + config->black_level.bl_b = 0; + + ipu3_uapi_shd_lut *lut = ¶ms->acc_param.shd.shd_lut; + + const auto &set = lscAlgo_.interpolateComponents(quantizedCt); + + unsigned int totalCells = numHCells_ * numVCells_; + unsigned int cellsPerSet = numHCells_ * config->grid.grid_height_per_slice; + unsigned int numSets = (numHCells_ + config->grid.grid_height_per_slice - 1) / + config->grid.grid_height_per_slice; + unsigned int i = 0; + + for (unsigned int s = 0; s < numSets; s++) { + for (unsigned int c = 0; c < cellsPerSet && i < totalCells; c++, i++) { + lut->sets[s].r_and_gr[c].r = set.at("r")[i]; + lut->sets[s].r_and_gr[c].gr = set.at("gr")[i]; + lut->sets[s].gb_and_b[c].gb = set.at("gb")[i]; + lut->sets[s].gb_and_b[c].b = set.at("b")[i]; + } + } + + lastAppliedCt_ = ct; + lastAppliedQuantizedCt_ = quantizedCt; + LOG(IPU3Lsc, Debug) + << "ct is " << ct << ", quantized to " + << quantizedCt; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Lsc::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(Lsc, "Lsc") + +} // namespace ipa::ipu3::algorithms + +} // namespace libcamera diff --git a/src/ipa/ipu3/algorithms/lsc.h b/src/ipa/ipu3/algorithms/lsc.h new file mode 100644 index 0000000000000000000000000000000000000000..aba3db6ce14c23947392a542d243e80042230162 --- /dev/null +++ b/src/ipa/ipu3/algorithms/lsc.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board, Oy + * + * IPU3 Lens Shading Correction algorithm + */ + +#pragma once + +#include + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +class Lsc : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ipu3_uapi_stats_3a *stats, + ControlList &metadata) override; + +private: + std::vector calculatePositions(unsigned int dimension); + + unsigned int numHCells_; + unsigned int numVCells_; + unsigned int blockWidthLog2_; + unsigned int blockHeightLog2_; + + unsigned int lastAppliedCt_; + unsigned int lastAppliedQuantizedCt_; + + unsigned int sensorWidth_; + unsigned int sensorHeight_; + unsigned int cropWidth_; + unsigned int cropHeight_; + + bool polynomial_; + + LscAlgorithm> lscAlgo_; +}; + +} // namespace ipa::ipu3::algorithms + +} // namespace libcamera diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 3dafd2fda9897942cf87d9640665c4fcff383859..70177f50415259e0c2bd207205c385dde07d6624 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -6,5 +6,6 @@ ipu3_ipa_algorithms = files([ 'awb.cpp', 'blc.cpp', 'ccm.cpp', + 'lsc.cpp', 'tone_mapping.cpp', ]) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 02e5f79fa5def888fe3776e5cb002771a497ee41..6632c6f45cd3393f8841454a12c2c0da472a61e1 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -126,6 +126,11 @@ namespace libcamera::ipa::ipu3 { * \brief Active gamma correction parameters for the IPA */ +/** + * \var IPAActiveState::lsc + * \brief Active lens shading correction parameters for the IPA + */ + /** * \struct IPAFrameContext * \brief IPU3-specific FrameContext @@ -158,4 +163,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame gamma correction parameters for the IPA */ +/** + * \var IPAFrameContext::lsc + * \brief Per-frame lens shading correction parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 1755fc62a89152470bd059c6eb6b6a366e451ba6..fde628c228ea73fcf968db573cb60db648d68904 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -23,6 +23,7 @@ #include #include #include +#include namespace libcamera { @@ -53,6 +54,7 @@ struct IPAActiveState { ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; ipa::gamma::ActiveState gamma; + ipa::lsc::ActiveState lsc; }; struct IPAFrameContext : public FrameContext { @@ -65,6 +67,7 @@ struct IPAFrameContext : public FrameContext { ipa::awb::FrameContext awb; ipa::ccm::FrameContext ccm; ipa::gamma::FrameContext gamma; + ipa::lsc::FrameContext lsc; }; struct IPAContext {