From patchwork Fri Jun 26 13:05:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27061 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 62A85C3301 for ; Fri, 26 Jun 2026 13:06:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E8F865EFA; Fri, 26 Jun 2026 15:06:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HrErYQgn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F3423658FA for ; Fri, 26 Jun 2026 15:06:04 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DF13FE2C; Fri, 26 Jun 2026 15:05:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479124; bh=doQKSK44QFyMvkC8EuIN8TSx1AHGSeggFu6MvvflL20=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=HrErYQgnQCk8aCR/Yrs1VcWf9HFwMYVJ5BbzL9Dzd73icaOvZUDktCZ0z2XR9iDhe vkgVWn+P6zNhRafntVi7oyC/Dw6oFyul5VawJL/CEPZ5gTbc5ZyoKGhMb2zJqymvEW 7LhvdAPmpVuvSDMsIVJZdGe0nQtu+vOJMu76mryk= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:48 +0100 Subject: [PATCH v2 01/12] ipa: ipu3: Remove ~Awb() from source and header MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-1-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , =?utf-8?q?Barnab=C3=A1s_?= =?utf-8?b?UMWRY3pl?= , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1403; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=doQKSK44QFyMvkC8EuIN8TSx1AHGSeggFu6MvvflL20=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk5CiUTqb17OcRJiaaNPY5acNs8we4Wd04uK EOs5U4rg76JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OQAKCRDISVd6bEV1 Mh0uD/oC4/b6XTwuPn+lLK3qDp548EmUzgPHkgrqK6Gky+6akW9QGYzQ12jcCUSH9OwGk2LW0lD dCzC8tTMM16M8QVJKwWCRljb8FXC2qv6Z15H05wbg0K1F1qHlaep4cHXD4gnH5jbgycgwLRAgEJ gavwFFl7WbuPVi1ITsfxEH+meBAUuvZlctOAFm8FX0QVHOYrjSPeAwDjN4Qqxm+e8hM6q52/Xnb LPbcv9uc5wySDhbxmdu9KXLlp7B63CVOvp0bjTAVseRe1PTXl+e70cZbzE95sUi0j1RjK6TAET9 7MQYyXHbJoKkm6bzkP2CXm4PBivhexYcCI+Q2n8xgc5gtwI2cvtjZVf1eam4vGpJjsktjEW0sP2 tiC684YtbosQll5yJ+O5t7Br9UonBdDD7rFiK1BGavPhr5egI52jmkyjv6lkyDnWxCb52V5qPJf bpRWqARqnJmP5PbAj46e/AAqMpbx+3GS/DePemH8agHF3kUZOaCiqjFbs8le5Q7DusxNGkzbjYL 2AXWQqDWs+LkPTAUxldEh6nVrvK8VMYVnj13jsPBRV102reckRSCNqC4ij5VoSf3d5A9A8iDbCT 3c5LjE+x+emPC54gfmLF6Svu06BNJmS3I0TYYUYW46y4p37/yIbPW+6UfxBt2poUrhs/c0RTOMv r8evmJU2inVVAtA== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove the ~Awb() definition from the source and header files and simply allow the compiler to provide the default implementations. Reviewed-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v2: - None --- src/ipa/ipu3/algorithms/awb.cpp | 2 -- src/ipa/ipu3/algorithms/awb.h | 1 - 2 files changed, 3 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 55de05d9e39f5d2c863b89923964a4c7756f3846..6b4418aa6f3add0a41cae77000c1f5f67897111a 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -195,8 +195,6 @@ Awb::Awb() zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } -Awb::~Awb() = default; - /** * \copydoc libcamera::ipa::Algorithm::configure */ diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index dbf69c9073a1627b666fe63158081e347cf3581e..7ec322318dab54ae7c8a647a67a0cf5815a36eb6 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -38,7 +38,6 @@ class Awb : public Algorithm { public: Awb(); - ~Awb(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, From patchwork Fri Jun 26 13:05:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27062 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7EF85C3302 for ; Fri, 26 Jun 2026 13:06:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C5BE65EFC; Fri, 26 Jun 2026 15:06:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AI91Hf46"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 30D8A65878 for ; Fri, 26 Jun 2026 15:06:05 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 388631438; Fri, 26 Jun 2026 15:05:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479124; bh=Pd82DKCym4GNm+qhZLWfJ/wUXOexPxja3Fmk9clyA+8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=AI91Hf46dXe2hvokniyM6iNhtA5nDNem6JUh65p/vp7O1EcW+3zkGX6NsOkSqcDQO ecSBZDJXRRgUyJNpKQBkaEAwlahbIJ+gbV9dJaT7tYoq3mrM59eaZWcHUVto9luoNR jOVAY9zGe5yHyYv9ReRFP5QUU3tBdEKLlcEtXKwU= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:49 +0100 Subject: [PATCH v2 02/12] ipu3: awb: Remove CCM settings from Awb algorithm MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-2-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , =?utf-8?q?Barnab=C3=A1s_?= =?utf-8?b?UMWRY3pl?= X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1552; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=Pd82DKCym4GNm+qhZLWfJ/wUXOexPxja3Fmk9clyA+8=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk5VN/fZ4OasjtS5p87+TQnoCcSuiY+PSYLj tI7A51lVv6JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OQAKCRDISVd6bEV1 MtWuD/4+Zy8cgnPlJ3aI61KtapPFQTaGF4v9Beo5OOWTXdGvKqUnY58WW2Ps35uEjaTNlF/LSjR ub60J7eVXF1HT/J1Kh2sNd1b0V+NQtWSovCHYZ0PdkoIUcTenHYV7WcyJrbcSGN0Lw5Rexa1m5Z 5/O8xtv4IXtIfM52pGRgoNTgMwBNceC48Ooow3XzokBZKuE1YQz9wiNtM97HqwKRU6HO7wkRreT FxGRL3I5+uIn38d1lHJe1gv+/x4idrFe7+Onn5lMEaBIjmb1bYwJJYrVyzzR56cMQhnWdRzSjib bqdII025rmNvRzkBsWIMmGBUZQGmga1Pd+EP+etHPwoZkptEN4/bDXa5Icf1DY+XduZP9Au8kXu BXNUtmxoQyrTgHil/ba08iH10f33bAOCAG255+hpTZ+yUzUN3hZhtPOxFXO7kOBH/1Q/wJP0yhz 2Z0e8TPYveMbn1tlStTPvj6Hd+/hUhlWFmLF/dj0HZque21yuODmk6IsUpVbWAqNpO7Y94aMznh +iVELMkXb3T3fx+G3Kl+8fl5kMUoiahooIlF5ycBOxpyvmO/5so0rUhIQQjrVqyUkj+plqOTdNc U08C3v3h/3O+hu4Ywo4SV0Q+sqnH397wxsXHtDklHrvJ+sYhFjMRumPBprHoTYZ3Krks7GZ8YDv C4TEdGGgirYVSQQ== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPU3 AWB algorithm currently defines a CCM matrix that is just a no-op. Remove the settings entirely; they need really to go in to a dedicated CCM algorithm. Reviewed-by: Barnabás Pőcze Signed-off-by: Daniel Scally --- Changes in v2: - None --- src/ipa/ipu3/algorithms/awb.cpp | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 6b4418aa6f3add0a41cae77000c1f5f67897111a..613bdbbf51cd127e03906571813260a6857067dc 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -112,13 +112,6 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { .opt_center_sqr = { 419904, 133956 }, }; -/* Default color correction matrix defined as an identity matrix */ -static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = { - 8191, 0, 0, 0, - 0, 8191, 0, 0, - 0, 0, 8191, 0 -}; - /** * \class Awb * \brief A Grey world white balance correction algorithm @@ -293,12 +286,9 @@ void Awb::prepare(IPAContext &context, LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; - /* The CCM matrix may change when color temperature will be used */ - params->acc_param.ccm = imguCssCcmDefault; params->use.acc_awb = 1; params->use.acc_bnr = 1; - params->use.acc_ccm = 1; } /* Generate an RGB vector with the average values for each zone */ From patchwork Fri Jun 26 13:05:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27063 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4E6CFBF415 for ; Fri, 26 Jun 2026 13:06:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 03D1565F07; Fri, 26 Jun 2026 15:06:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vnuDMNtq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E814658FA for ; Fri, 26 Jun 2026 15:06:05 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7BF5E19EC; Fri, 26 Jun 2026 15:05:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479124; bh=cPbSJkOCWsDRjI/shTMr9ESWFrPH86HAjyeuXWQiXmU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=vnuDMNtqkCvUrOle/fJ9YzX/hiakUELx2suIaXUxBt73bQwpTUcfT6Q0q1tBAcK54 dIYCB7OSvL3QuxR1Bg8PyGV1YZ3zVs5eooSSDbcKz70L1ZW6zu68Oj0Zi9aRia8om6 HdTrGFPdxPOdONJPpnNq7dC7PA3zgQUdc7PVzF+I= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:50 +0100 Subject: [PATCH v2 03/12] ipa: ipu3: Convert from separate gains to RGB in AGC MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-3-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2356; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=cPbSJkOCWsDRjI/shTMr9ESWFrPH86HAjyeuXWQiXmU=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk5+jfijVknwt+44YYtRE0Zq6LUf3vDItggJ YZp1gRMuwyJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OQAKCRDISVd6bEV1 Ms9/D/0dMZ9ZMkVIPNZ6Z0MSwf8po67A0RbVLKjoVFpAKEZhtobajSYRf+FMolFPYb/FeM3+nEr fF3czmusZn9Jq0Sh7A4zpDlyynkkPf6pIMMBsFWJwFswpizi8Bufj91uc9ulrMomXLBUMRmRx03 TEwgLJwT8FoCKPul5EZ3QEnMCuJhk5WzIva5XTM43ZSFOj+nOq4mOod6HhZgP5WPZkFjnQqVfKI 1YUDk2MOmf1P/lTSVyXrx36jXK1UBk1dRMblihLd1GKoncWexn352HBgMe74lENwVSmAOJljxri T+v+iC1FNHcuY1TTHbxhFRBWErSvkWXye5cKiaMcyuEX67rWmGZ4U/Xy7Y5EB5e/0+bol/xC714 iwUs9SDMViPqGc+4/uHwEVE8AJqcj44U43hkqMkMw6jGkJiSWGUmSY7Y/gqycCANmyTxMZWBSfb iXRZBcpApRCKL3h6ueEm4P44Gqc682xQnuHWWYfAqXvr0m/qpQIpcf2UodrlnXFE1SPlzt5JOdO k48tjKWS/RrmfB2ztbIzZKxnk48HiAoAA9daQ5nQQG5oCROmxxNSkH+qnb7ASTSuhTUtMnt2bmI ITppoZwbctAghP3xRG0JO01bmGWUU53m+n63aV209SwopsQ2oxU7oNi+IBWLSbkmhNfvYXBDBh5 0SeBmx8j4awUvww== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC algorithm stores red, blue and green gains as type double class members. Switch to using a single member of type RGB instead to simplify the code. Signed-off-by: Daniel Scally --- Changes in v2: - New patch --- src/ipa/ipu3/algorithms/agc.cpp | 9 ++++----- src/ipa/ipu3/algorithms/agc.h | 6 +++--- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index d6a7036c6504acb106f5c773b529ad80b8349f85..b8b880b357a4770efd6810a3bdf616dd25ce93e4 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -186,8 +186,7 @@ double Agc::estimateLuminance(double gain) const sum.b() += std::min(std::get<2>(rgbTriples_[i]) * gain, 255.0); } - RGB gains{{ rGain_, gGain_, bGain_ }}; - double ySum = rec601LuminanceFromRGB(sum * gains); + double ySum = rec601LuminanceFromRGB(sum * gains_); return ySum / (bdsGrid_.height * bdsGrid_.width) / 255; } @@ -208,9 +207,9 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, ControlList &metadata) { Histogram hist = parseStatistics(stats, context.configuration.grid.bdsGrid); - rGain_ = context.activeState.awb.gains.red; - gGain_ = context.activeState.awb.gains.blue; - bGain_ = context.activeState.awb.gains.green; + gains_ = RGB({ context.activeState.awb.gains.red, + context.activeState.awb.gains.blue, + context.activeState.awb.gains.green }); /* * The Agc algorithm needs to know the effective exposure value that was diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index d274a2350485f8f9a3870b24ceb29927cb9c2bec..12ddb173ff89b7de68dd544a194292f4d14efdbb 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -13,6 +13,8 @@ #include +#include "libcamera/internal/vector.h" + #include "libipa/agc_mean_luminance.h" #include "libipa/histogram.h" @@ -49,9 +51,7 @@ private: double maxAnalogueGain_; uint32_t stride_; - double rGain_; - double gGain_; - double bGain_; + RGB gains_; ipu3_uapi_grid_config bdsGrid_; std::vector> rgbTriples_; }; From patchwork Fri Jun 26 13:05:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27064 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B02CBC3303 for ; Fri, 26 Jun 2026 13:06:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8546B65EF2; Fri, 26 Jun 2026 15:06:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aNtk72J9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B94BB65EE7 for ; Fri, 26 Jun 2026 15:06:05 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B95001ADF; Fri, 26 Jun 2026 15:05:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479125; bh=YYbJaObXm/0PNCPcdffs9U7nMV4aoqxvPj5rhy71Le4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=aNtk72J9jNDNOiYAATuARdgjcZ8XCWNOJzuRkEd0+fzvsvJktsZ1C+95KNCrZ9ihT EaSTZuxdz33T7si3OSSwx39Tcl1z8QLBLLzNze0mmgIQs++b0Q5nKQjhmw0wUbMlpf nWLRzJEqQE9ulJUj3/aBLhM/lzSaCgM4JdX01aVk= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:51 +0100 Subject: [PATCH v2 04/12] ipa: ipu3: awb: Port to the new libipa AwbAlgorithm MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-4-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=17392; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=YYbJaObXm/0PNCPcdffs9U7nMV4aoqxvPj5rhy71Le4=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk6TMNgLOolQWCQYoR5zC5STIC3Ndp0Z/AJY 8lFI+U2RV+JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OgAKCRDISVd6bEV1 MkLrD/43B4yYPAFTdjq9wLMR3AqeJLE+GzniPBw15HWoQAsOjIRAaa3MzQ+bRJLAJtsoLBoR1qp fEnq1eaZw93A8aC2zPfxHIhQrFNBVQWYx9n3D5VW4POVsHYuD2ppc6MU80B/K1wqqjp74aUYAV7 K5KVhWVoWnVsMYGVKj75HUF3D9pe3XxZeq7Oh+vv+zc+tEhdzdV8GzMHB8QgaiW0MIoM2RChqX6 ECa2v6DP07RTvYxGNMqrkEfkw1ZWAjSqhnhV4KCwvmejHagsbGeDtGiT9EHcLjS8NUe57VupkLC ZGI5fKjWyY44Cs66ExfeyBFLJHmxPb3MMGhVGhPk/ElmowXoJSBk4N/Li/Jy/tj0tSIKgM8ukam I2cR+ZkChN15lbeUA6S5foKp4R5qttBKW/fx7oDmOz+0DAMhNjfTplpCcER41hfXrCvA4mHfeVx yysqFxjv52n2J0/TItR2G3Mbt9oWT7/Rf9NAeCoW7EKQfxQy/KQGxHBGsMD3Mc2bCb2+0Xa7vBl TqTOfWRapyVn1ZkwW3+fWDygUXmFZbwQt81mEC+pqJhBKWQZZgjtvIdPJKislCFL1ojq0V7Iwbg bFpAnTgtUewRws9EAcZECAHLHcwx51Sm563r2LiizavhHgQrU/bnnXasORJu4tIW4LQvJVRwet0 Tein9Q5Kao/trOQ== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Port the IPU3 Awb algorithm to use the new libipa implementation of AwbAlgorithm. In this implementation the awbAlgo_ class member is initialised as UQ<3, 13> following the IPU3 params format documentation. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v2: - Used the new RGB(gains_) member of the class - Fixed includes - Commented minColourValue --- src/ipa/ipu3/algorithms/agc.cpp | 8 +- src/ipa/ipu3/algorithms/awb.cpp | 204 +++++++++++++++++++--------------------- src/ipa/ipu3/algorithms/awb.h | 21 ++--- src/ipa/ipu3/ipa_context.cpp | 30 ++---- src/ipa/ipu3/ipa_context.h | 13 +-- 5 files changed, 124 insertions(+), 152 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index b8b880b357a4770efd6810a3bdf616dd25ce93e4..de26e01ec901c2af109dbabd8fb84505a3107e82 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -207,9 +207,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, ControlList &metadata) { Histogram hist = parseStatistics(stats, context.configuration.grid.bdsGrid); - gains_ = RGB({ context.activeState.awb.gains.red, - context.activeState.awb.gains.blue, - context.activeState.awb.gains.green }); + + if (context.activeState.awb.autoEnabled) + gains_ = context.activeState.awb.automatic.gains; + else + gains_ = context.activeState.awb.manual.gains; /* * The Agc algorithm needs to know the effective exposure value that was diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 613bdbbf51cd127e03906571813260a6857067dc..be493659ea5780b6971fe3c4863c5a44b2114d26 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -13,8 +13,6 @@ #include -#include "libipa/colours.h" - /** * \file awb.h */ @@ -26,11 +24,48 @@ namespace ipa::ipu3::algorithms { LOG_DEFINE_CATEGORY(IPU3Awb) /* - * When zones are used for the grey world algorithm, they are only considered if - * their average green value is at least 16/255 (after black level subtraction) - * to exclude zones that are too dark and don't provide relevant colour - * information (on the opposite side of the spectrum, saturated regions are - * excluded by the ImgU statistics engine). + * \todo IPU3 doesn't support the Lux algorithm. + */ +static constexpr unsigned int kDefaultLux = 500; + +/** + * \brief The IPU3 implementation of AwbStats + */ +class Ipu3AwbStats final : public AwbStats +{ +public: + Ipu3AwbStats(){}; + Ipu3AwbStats(const RGB means) + : AwbStats(means) + { + } + + /* + * The minimum colour value is designed to allow libipa to make an + * assessment as to whether there's enough information in the statistics + * for a frame to be useful. The IPU3 implementation already drops any + * statistics zone with an average value below a threshold though so we + * don't need to do it in libipa. Set the threshold to zero so that all + * frames' statistics are counted as valid. + */ + double minColourValue() const override + { + return 0.0; + } +}; + +/** + * \fn Ipu3AwbStats::Ipu3AwbStats(const RGB means) + * \brief Construct an instance of the class with RGB means + * \param[in] means The mean R, G and B values from the statistics + */ + +/* + * Zones are only considered if their average green value is at least + * kMinGreenLevelInZone/255 (after black level subtraction) to exclude zones + * that are too dark and don't provide relevant colour information (on the + * opposite side of the spectrum, saturated regions are excluded by the ImgU + * statistics engine). */ static constexpr uint32_t kMinGreenLevelInZone = 16; @@ -74,26 +109,6 @@ static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 90 / 100; * \brief Sum of the average blue values of each unsaturated cell in the zone */ -/** - * \struct Awb::AwbStatus - * \brief AWB parameters calculated - * - * The AwbStatus structure is intended to store the AWB - * parameters calculated by the algorithm - * - * \var AwbStatus::temperatureK - * \brief Color temperature calculated - * - * \var AwbStatus::redGain - * \brief Gain calculated for the red channel - * - * \var AwbStatus::greenGain - * \brief Gain calculated for the green channel - * - * \var AwbStatus::blueGain - * \brief Gain calculated for the blue channel - */ - /* Default settings for Bayer noise reduction replicated from the Kernel */ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { .wb_gains = { 16, 16, 16, 16 }, @@ -114,11 +129,7 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { /** * \class Awb - * \brief A Grey world white balance correction algorithm - * - * The Grey World algorithm assumes that the scene, in average, is neutral grey. - * Reference: Lam, Edmund & Fung, George. (2008). Automatic White Balancing in - * Digital Photography. 10.1201/9781420054538.ch10. + * \brief The IPU3 white balance correction algorithm implementation * * The IPU3 generates statistics from the Bayer Down Scaler output into a grid * defined in the ipu3_uapi_awb_config_s structure. @@ -168,26 +179,26 @@ static const struct ipu3_uapi_bnr_static_config imguCssBnrDefaults = { * cells are ignored. The grid configuration is computed by * IPAIPU3::calculateBdsGrid(). * - * Before calculating the gains, the algorithm aggregates the cell averages for - * each zone in generateAwbStats(). Cells that have a too high ratio of - * saturated pixels are ignored, and only zones that contain enough - * non-saturated cells are then used by the algorithm. - * - * The Grey World algorithm will then estimate the red and blue gains to apply, and - * store the results in the metadata. The green gain is always set to 1. + * Before running the AWB algorithm, we aggregate the cell averages for each + * zone in generateAwbStats(). Cells that have a too high ratio of saturated + * pixels are ignored, and only zones that contain enough non-saturated cells + * are then used by the algorithm. */ Awb::Awb() : Algorithm() { - asyncResults_.blueGain = 1.0; - asyncResults_.greenGain = 1.0; - asyncResults_.redGain = 1.0; - asyncResults_.temperatureK = 4500; - zones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY); } +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ @@ -197,12 +208,14 @@ int Awb::configure(IPAContext &context, const ipu3_uapi_grid_config &grid = context.configuration.grid.bdsGrid; stride_ = context.configuration.grid.stride; + awbAlgo_.configure(context.activeState.awb); + cellsPerZoneX_ = std::round(grid.width / static_cast(kAwbStatsSizeX)); cellsPerZoneY_ = std::round(grid.height / static_cast(kAwbStatsSizeY)); /* * Configure the minimum proportion of cells counted within a zone - * for it to be relevant for the grey world algorithm. + * for it to be used. * \todo This proportion could be configured. */ cellsPerZoneThreshold_ = cellsPerZoneX_ * cellsPerZoneY_ * kMaxCellSaturationRatio; @@ -211,6 +224,17 @@ int Awb::configure(IPAContext &context, return 0; } +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + constexpr uint16_t Awb::threshold(float value) { /* AWB thresholds are in the range [0, 8191] */ @@ -237,11 +261,12 @@ constexpr uint16_t Awb::gainValue(double gain) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - ipu3_uapi_params *params) +void Awb::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] ipu3_uapi_params *params) { + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + /* * Green saturation thresholds are reduced because we are using the * green channel only in the exposure computation. @@ -279,13 +304,11 @@ void Awb::prepare(IPAContext &context, params->acc_param.bnr.opt_center_sqr.y_sqr_reset = params->acc_param.bnr.opt_center.y_reset * params->acc_param.bnr.opt_center.y_reset; - params->acc_param.bnr.wb_gains.gr = gainValue(context.activeState.awb.gains.green); - params->acc_param.bnr.wb_gains.r = gainValue(context.activeState.awb.gains.red); - params->acc_param.bnr.wb_gains.b = gainValue(context.activeState.awb.gains.blue); - params->acc_param.bnr.wb_gains.gb = gainValue(context.activeState.awb.gains.green); - - LOG(IPU3Awb, Debug) << "Color temperature estimated: " << asyncResults_.temperatureK; + params->acc_param.bnr.wb_gains.gr = gainValue(frameContext.awb.gains.g()); + params->acc_param.bnr.wb_gains.r = gainValue(frameContext.awb.gains.r()); + params->acc_param.bnr.wb_gains.b = gainValue(frameContext.awb.gains.b()); + params->acc_param.bnr.wb_gains.gb = gainValue(frameContext.awb.gains.g()); params->use.acc_awb = 1; params->use.acc_bnr = 1; @@ -366,9 +389,17 @@ void Awb::clearAwbStats() } } -void Awb::awbGreyWorld() +Ipu3AwbStats Awb::calculateRgbMeans(const ipu3_uapi_stats_3a *stats) { - LOG(IPU3Awb, Debug) << "Grey world AWB"; + ASSERT(stats->stats_3a_status.awb_en); + + clearAwbStats(); + generateAwbStats(stats); + generateZones(); + + if (zones_.size() <= 10) + return {}; + /* * Make a separate list of the derivatives for each of red and blue, so * that we can sort them to exclude the extreme gains. We could @@ -399,66 +430,21 @@ void Awb::awbGreyWorld() double redGain = sumRed.g() / (sumRed.r() + 1), blueGain = sumBlue.g() / (sumBlue.b() + 1); - /* Color temperature is not relevant in Grey world but still useful to estimate it :-) */ - asyncResults_.temperatureK = estimateCCT({{ sumRed.r(), sumRed.g(), sumBlue.b() }}); - - /* - * Gain values are unsigned integer value ranging [0, 8) with 13 bit - * fractional part. - */ - redGain = std::clamp(redGain, 0.0, 65535.0 / 8192); - blueGain = std::clamp(blueGain, 0.0, 65535.0 / 8192); - - asyncResults_.redGain = redGain; - /* Hardcode the green gain to 1.0. */ - asyncResults_.greenGain = 1.0; - asyncResults_.blueGain = blueGain; -} - -void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) -{ - ASSERT(stats->stats_3a_status.awb_en); - - clearAwbStats(); - generateAwbStats(stats); - generateZones(); - - LOG(IPU3Awb, Debug) << "Valid zones: " << zones_.size(); - - if (zones_.size() > 10) { - awbGreyWorld(); - LOG(IPU3Awb, Debug) << "Gain found for red: " << asyncResults_.redGain - << " and for blue: " << asyncResults_.blueGain; - } + return Ipu3AwbStats({ { 1.0 / redGain, 1.0, 1.0 / blueGain } }); } /** * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - const ipu3_uapi_stats_3a *stats, - [[maybe_unused]] ControlList &metadata) + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) { - calculateWBGains(stats); + Ipu3AwbStats awbStats = calculateRgbMeans(stats); - /* - * Gains are only recalculated if enough zones were detected. - * The results are cached, so if no results were calculated, we set the - * cached values from asyncResults_ here. - */ - context.activeState.awb.gains.blue = asyncResults_.blueGain; - context.activeState.awb.gains.green = asyncResults_.greenGain; - context.activeState.awb.gains.red = asyncResults_.redGain; - context.activeState.awb.temperatureK = asyncResults_.temperatureK; - - metadata.set(controls::AwbEnable, true); - metadata.set(controls::ColourGains, { - static_cast(context.activeState.awb.gains.red), - static_cast(context.activeState.awb.gains.blue) - }); - metadata.set(controls::ColourTemperature, - context.activeState.awb.temperatureK); + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + kDefaultLux, metadata); } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 7ec322318dab54ae7c8a647a67a0cf5815a36eb6..e1a7b2139e5dafa748b908230aa3189f33b09ced 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -13,7 +13,8 @@ #include -#include "libcamera/internal/vector.h" +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" #include "algorithm.h" @@ -21,6 +22,8 @@ namespace libcamera { namespace ipa::ipu3::algorithms { +class Ipu3AwbStats; + /* Region size for the statistics generation algorithm */ static constexpr uint32_t kAwbStatsSizeX = 16; static constexpr uint32_t kAwbStatsSizeY = 12; @@ -39,7 +42,11 @@ class Awb : public Algorithm public: Awb(); + int init(IPAContext &context, const ValueNode &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; @@ -49,15 +56,7 @@ public: ControlList &metadata) override; private: - struct AwbStatus { - double temperatureK; - double redGain; - double greenGain; - double blueGain; - }; - -private: - void calculateWBGains(const ipu3_uapi_stats_3a *stats); + Ipu3AwbStats calculateRgbMeans(const ipu3_uapi_stats_3a *stats); void generateZones(); void generateAwbStats(const ipu3_uapi_stats_3a *stats); void clearAwbStats(); @@ -67,7 +66,7 @@ private: std::vector> zones_; Accumulator awbStats_[kAwbStatsSizeX * kAwbStatsSizeY]; - AwbStatus asyncResults_; + AwbAlgorithm> awbAlgo_; uint32_t stride_; uint32_t cellsPerZoneX_; diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 3b22f7917650d9e400d5368c2f890d6b2dc846a0..e33c925f42b87887467a32dd46a42248a8723415 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -109,6 +109,11 @@ namespace libcamera::ipa::ipu3 { * \brief Maximum analogue gain supported with the configured sensor */ +/** + * \var IPAActiveState::awb + * \brief Active auto-white balance parameters for the IPA + */ + /** * \var IPASessionConfiguration::sensor * \brief Sensor-specific configuration of the IPA @@ -139,26 +144,6 @@ namespace libcamera::ipa::ipu3 { * The gain should be adapted to the sensor specific gain code before applying. */ -/** - * \var IPAActiveState::awb - * \brief Context for the Automatic White Balance algorithm - * - * \var IPAActiveState::awb.gains - * \brief White balance gains - * - * \var IPAActiveState::awb.gains.red - * \brief White balance gain for R channel - * - * \var IPAActiveState::awb.gains.green - * \brief White balance gain for G channel - * - * \var IPAActiveState::awb.gains.blue - * \brief White balance gain for B channel - * - * \var IPAActiveState::awb.temperatureK - * \brief Estimated color temperature - */ - /** * \var IPAActiveState::toneMapping * \brief Context for ToneMapping and Gamma control @@ -187,4 +172,9 @@ namespace libcamera::ipa::ipu3 { * \brief Analogue gain multiplier */ +/** + * \var IPAFrameContext::awb + * \brief Per-frame auto-white balance parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 97fcf06cd4ac9ac6d64c4933fcea80ace0e572df..564cedde51d9e20263c4cc2767853d2a5f02d2e1 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -15,6 +15,7 @@ #include #include +#include #include namespace libcamera { @@ -60,15 +61,7 @@ struct IPAActiveState { uint32_t exposureMode; } agc; - struct { - struct { - double red; - double green; - double blue; - } gains; - - double temperatureK; - } awb; + ipa::awb::ActiveState awb; struct { double gamma; @@ -81,6 +74,8 @@ struct IPAFrameContext : public FrameContext { uint32_t exposure; double gain; } sensor; + + ipa::awb::FrameContext awb; }; struct IPAContext { From patchwork Fri Jun 26 13:05:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27065 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DE9DCC3304 for ; Fri, 26 Jun 2026 13:06:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F1B2E65EEE; Fri, 26 Jun 2026 15:06:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MwR/yKVT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1A68865EE7 for ; Fri, 26 Jun 2026 15:06:06 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 24FEA1438; Fri, 26 Jun 2026 15:05:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479125; bh=uGLdvDgcqMhdrFOKVJf1d7WCwiHYzz3bBtYelD5Zv+c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=MwR/yKVTXqk+IkPxQKlMher243Qddb7N4zJXwFcObehSlM27yaR0a7fp7MzZAqsed en6EYYovMAYUbr5yBAIu7kZ/YDPNExGnBfZNN21ZR2Csh68gkc1owDeSEsA0/3YCCF Sv3jEKbDKdRv2kJMhDkLmAH3Kq1peYIf+rUdSXo4= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:52 +0100 Subject: [PATCH v2 05/12] ipa: ipu3: ccm: Add a Colour Correction Matrix algorithm MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-5-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham , Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8109; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=uGLdvDgcqMhdrFOKVJf1d7WCwiHYzz3bBtYelD5Zv+c=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk6SYGLrrf3djBtl7i2+GVME6/EhTXETEv78 3eh9SYfziWJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OgAKCRDISVd6bEV1 MkptEACE0/y1HFM+SGua8GEWE5dBHG6P/7bzNVP8duL+SRBwezXVA8krvPYgI+33Dp2mL1PHop6 zuf93E46lxQQT6g91YnZsm9eUXzSKjXVZS/u0nLrWQOBohh1H5KxhdHO3RFgUvYVEnvzmPri0JG +502bI2ugFR7LX39JuEkGHte+Mr9K7841+/rHfmaRQOnCcKsscoSpNgJ2i1w6JR5fUARjb880B4 FcSDqD9usw53C5hsnMpE3pnd7ZV87Z7f6ZYDHnYQ3iip2n4jLTNe8YELtk9kbmq4iZyewpTYJke wF5iIza/BZhk7ZuWLbYH2JmOIz7KHYTBKGgvyPS/Xm/KAVtA1L8GSsFhajBQ0Exy2p6A0hLTUuJ oCtY/ht3CVLh9OeHjKdjYnu85HvUicxk3P6KcAgvb55pzA95aCQQdernGt+BLQSsUyn8flm066S GtLup54g7c9B+n4kjt7Dw4sP4Vz7mDuFLweX8qGcZPWVH6WOYcIQ/AKmTMm3sWqOCZBigvjZhLR YTFfY9v0+sD+eYQtqy1ND+Kk6865DdYz5otoIYIc87MHX2gvoBwylXieoSYDPi36/ARY1vKHpZC fqwvNipDzfo4G3BJTAhTXL1Qzhlb5cNL/+fFhFouzBZYxgurOACt6ohzF1fahLZhtGbPC3ofkac 251Ue/fNVjO2U0w== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a Colour Correction Matrix algorithm that uses the new libipa implementation. The module isn't well documented in the kernel but the default values from the driver suggest a Q3.13 format. Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v2: - Fixed includes - Documentation comment rework - Some styling changes --- src/ipa/ipu3/algorithms/ccm.cpp | 116 ++++++++++++++++++++++++++++++++++++ src/ipa/ipu3/algorithms/ccm.h | 48 +++++++++++++++ src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipa_context.cpp | 10 ++++ src/ipa/ipu3/ipa_context.h | 3 + 5 files changed, 178 insertions(+) diff --git a/src/ipa/ipu3/algorithms/ccm.cpp b/src/ipa/ipu3/algorithms/ccm.cpp new file mode 100644 index 0000000000000000000000000000000000000000..44b8c107b079c544c5afd1fce3cbe355f6dacd76 --- /dev/null +++ b/src/ipa/ipu3/algorithms/ccm.cpp @@ -0,0 +1,116 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * IPU3 Colour correction matrix algorithm + */ + +#include "ccm.h" + +/** + * \file ccm.h + */ + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +/** + * \class Ccm + * \brief The IPU3 color correction matrix algorithm + */ + +LOG_DEFINE_CATEGORY(IPU3Ccm) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init(IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.temperatureK); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Ccm::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* Nothing to do here, the ccm will be calculated in prepare() */ + if (frameContext.awb.autoEnabled) + return; + + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(ipu3_uapi_params *params, IPAFrameContext &context) +{ + const Matrix &matrix = context.ccm.ccm; + const Matrix &offsets = context.ccm.offsets; + + params->use.acc_ccm = 1; + + params->acc_param.ccm.coeff_m11 = Q<3, 13>(matrix[0][0]).quantized(); + params->acc_param.ccm.coeff_m12 = Q<3, 13>(matrix[0][1]).quantized(); + params->acc_param.ccm.coeff_m13 = Q<3, 13>(matrix[0][2]).quantized(); + params->acc_param.ccm.coeff_o_r = offsets[0][0]; + + params->acc_param.ccm.coeff_m21 = Q<3, 13>(matrix[1][0]).quantized(); + params->acc_param.ccm.coeff_m22 = Q<3, 13>(matrix[1][1]).quantized(); + params->acc_param.ccm.coeff_m23 = Q<3, 13>(matrix[1][2]).quantized(); + params->acc_param.ccm.coeff_o_g = offsets[1][0]; + + params->acc_param.ccm.coeff_m31 = Q<3, 13>(matrix[2][0]).quantized(); + params->acc_param.ccm.coeff_m32 = Q<3, 13>(matrix[2][1]).quantized(); + params->acc_param.ccm.coeff_m33 = Q<3, 13>(matrix[2][2]).quantized(); + params->acc_param.ccm.coeff_o_b = offsets[2][0]; + + LOG(IPU3Ccm, Debug) << "Setting matrix " << matrix; + LOG(IPU3Ccm, Debug) << "Setting offsets " << offsets; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, ipu3_uapi_params *params) +{ + if (!frameContext.awb.autoEnabled) { + setParameters(params, frameContext); + return; + } + + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, frame, + frameContext.awb.temperatureK); + + setParameters(params, frameContext); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/ccm.h b/src/ipa/ipu3/algorithms/ccm.h new file mode 100644 index 0000000000000000000000000000000000000000..9103de54b32b795d0efd22fad15ca6defb194057 --- /dev/null +++ b/src/ipa/ipu3/algorithms/ccm.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * IPU3 Colour correction matrix algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include +#include + +#include "algorithm.h" +#include "ipa_context.h" + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +class Ccm : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ipu3_uapi_stats_3a *stats, + ControlList &metadata) override; + +private: + void setParameters(ipu3_uapi_params *params, IPAFrameContext &context); + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::ipu3::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index b70a551cacf6cbdedf3ab41771a34ddcdae9fd8b..3dafd2fda9897942cf87d9640665c4fcff383859 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -5,5 +5,6 @@ ipu3_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'ccm.cpp', 'tone_mapping.cpp', ]) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index e33c925f42b87887467a32dd46a42248a8723415..469905a43c4300bfa7a7fee5777f4a51837cf4c6 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -114,6 +114,11 @@ namespace libcamera::ipa::ipu3 { * \brief Active auto-white balance parameters for the IPA */ +/** + * \var IPAActiveState::ccm + * \brief Active colour Correction Matrix parameters for the IPA + */ + /** * \var IPASessionConfiguration::sensor * \brief Sensor-specific configuration of the IPA @@ -177,4 +182,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame auto-white balance parameters for the IPA */ +/** + * \var IPAFrameContext::ccm + * \brief Per-frame colour Correction Matrix parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 564cedde51d9e20263c4cc2767853d2a5f02d2e1..6445db2603645d804b6caa40cde65099c4e57511 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -16,6 +16,7 @@ #include #include +#include #include namespace libcamera { @@ -62,6 +63,7 @@ struct IPAActiveState { } agc; ipa::awb::ActiveState awb; + ipa::ccm::ActiveState ccm; struct { double gamma; @@ -76,6 +78,7 @@ struct IPAFrameContext : public FrameContext { } sensor; ipa::awb::FrameContext awb; + ipa::ccm::FrameContext ccm; }; struct IPAContext { From patchwork Fri Jun 26 13:05:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27066 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1DAC9C3305 for ; Fri, 26 Jun 2026 13:06:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3818565EE1; Fri, 26 Jun 2026 15:06:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fTDSjas1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5F0CB65EE8 for ; Fri, 26 Jun 2026 15:06:06 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7A76519EC; Fri, 26 Jun 2026 15:05:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479125; bh=3GaMPMabKUaQu1vwLcblL1vRtBOXJ1YreK2a2eNpgww=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=fTDSjas1W2FWYmrURHuNjES8eYijbr/R3p8EJaO50drGhmF59lPbgGhzxW2NnJV7y xtYrFK4IA8S/fYQkO205ETwcQHBxsV3THXnl+OoXkaHFRZwva63mLwxtYJwR7NLA+P CfRXjXnBTFiLeuKVaS4RAV1qgpWgEPIosKg5QGw0= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:53 +0100 Subject: [PATCH v2 06/12] ipa: libipa: Add GammaAlgorithm class MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-6-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12001; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=3GaMPMabKUaQu1vwLcblL1vRtBOXJ1YreK2a2eNpgww=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk6AhIs9ULOwxe7qtF6jSqrqc7qmGLsOsKPy Pw4JQz6qr6JAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OgAKCRDISVd6bEV1 MtZxD/9F/pQ1pxLysCUeRn3vcBGL/dyNXV2tk/N8+nUp9f7R67GPzmK2DSKYlGKfhq8R1HDPcuz S4KOOsRw+fGCUsBY3fgZ8+RJ0x5QG0pI6NkMzYdKKOvwZVveCq/drIOYyff0WIt4cVPZssrKru3 OoODqSoMTco1nS+3cti2mGFQvG78EmQ+X3t+Jm49M9XjT0qJ46hN9V2+cxG80Y1LA5qWJuh2fA1 fpZ0bmucEJJR6+bfYvv0gSOXAPDhsDqofO69Hy60znsq+liNJNTmUlnPuS7bZ6YTQd41wdXlU3a GB1T9k7MfkITXpI9UUTWXwgH7Vxj4IFLE+lEtKs7dmylw+l6mWicOLmq6tcpTXiTFbaKxTv7qiY p+yoYUPzxZBQTQF8EDeKspLjk/22o8tisBsnm+rgR1MCexcXhcGO2R5JRJkR4zYKBqya76ftaCf MpcaRNmxujf+OlCZBfsux22HaadMG46wYubXwVSpzPyTPJ55uM2UnFnh41AFYJceyIX3wqH7iFr ygtQLDUS+ioccWqCOi9b6QWxv8qnBvyRtF7xG9q6fFDDQBGX1Zbm0tLRwgEuYAFXjLQKrY/w0zn M1qw+mI8VNVaZPdm85FytABuyV1WRzNwlPUziNGL5MgIebTbPPRUeastSky392KL7V03H8hG9gV csBvNPwRB+jxTgg== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a base GammaAlgorithm class that can be used by IPA specific gamma algorithms to reduce the amount of work that they need to implement. Reviewed-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v2: - Updated documentation comments, and minor styling changes --- src/ipa/libipa/gamma.cpp | 257 +++++++++++++++++++++++++++++++++++++++++++++ src/ipa/libipa/gamma.h | 94 +++++++++++++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 353 insertions(+) diff --git a/src/ipa/libipa/gamma.cpp b/src/ipa/libipa/gamma.cpp new file mode 100644 index 0000000000000000000000000000000000000000..5fec5329a50f0b3dd0e39504dfbb63484ca4fcd9 --- /dev/null +++ b/src/ipa/libipa/gamma.cpp @@ -0,0 +1,257 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * + * libIPA Gamma correction algorithm + */ + +#include "gamma.h" + +#include + +#include + +#include "libcamera/internal/value_node.h" + +/** + * \file gamma.h + * \brief libipa implementation of a gamma curve correction algorithm + */ + +namespace libcamera { + +namespace ipa { + +LOG_DEFINE_CATEGORY(Gamma) + +namespace gamma { + +/** + * \struct ActiveState + * \brief Active gamma correction algorithm state + * + * \var ActiveState::gamma + * \brief The gamma correction value + */ + +/** + * \struct FrameContext + * \brief Per-frame gamma correction settings + * + * \var FrameContext::gamma + * \brief The gamma correction value applied for this frame + * + * \var FrameContext::update + * \brief A flag instructing the algorithm to push an update to the hardware + */ + +} /* namespace gamma */ + +/** + * \brief The default gamma correction value + */ +const float kDefaultGamma = 2.2f; + +/** + * \class GammaAlgorithmBase + * \brief Base class for GammaAlgorithm to implement non-templated functions + * + * This base class for GammeaAlgorithm allows us to implement non templated + * functions. IPA specific implementations shall derive from GammaAlgorithm and + * not this class. + */ + +/** + * \fn GammaAlgorithmBase::GammaAlgorithmBase + * \brief Construct an instance of the class + * \param[in] nLutNodes Set the number of function knee-points expected by the + * IPA algorithm + */ + +/** + * \brief Initialise the algorithm with the given tuning data + * \param[out] controls The ControlList into which this algorithm's supported + * controls will be emplaced. + * \param[in] tuningData The tuning data to use with the algorithm + * \param[in] segments A vector of segment spacings to define a custom + * X coordinate system for the curve + * + * Parse \a tuningData and \a segments to initialize the gamma correction curve. + * The tuning data may contain a default gamma value to use; otherwise the value + * of \a kDefaultGamma will be taken as the default. The piecewise linear + * function will be applied on a number of knots whose position is described by + * the optional \a segments argument, which describes each segment's relative + * length. + * + * For example, if the gamma correction has to be applied on 16 equally spaced + * sampling points, a \segments array like: + * + * [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + * + * would result in evenly spaced knee-points along the X-axis. + * + * Hardware may expect the knee-points to be spaced more densely towards the + * start of the curve and more sparsely towards the end, in which case an + * alternative array might be: + * + * [1, 1, 1, 1, 2, 2, 2, 2, 4, 4, 4, 8, 8, 8, 8, 8] + * + * As the values in \a segments represent the distance between two knee-points + * relative to the total distance between the first and last point, the length + * of \a segments should be equal to the number of knee-points minus one. + *; + * If an IPA implementation doesnt't provide \a segments, the GammaAlgorithm + * class consturcts an evenly-spaced default. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::init() + * + * @return 0 on success, a negative error code otherwise + */ +int GammaAlgorithmBase::init(ControlInfoMap::Map &controls, const ValueNode &tuningData, + Span segments) +{ + /* + * If the caller doesn't pass in a segment list we simply construct one + * with equally spaced segments. We need one less segment than we have + * LUT nodes. + */ + unsigned int expectedNSegments = nLutNodes_ - 1; + + if (segments.empty()) { + for (unsigned int i = 0; i < expectedNSegments; i++) + segments_.push_back(1); + } else { + segments_.assign(segments.begin(), segments.end()); + } + + if (segments_.size() != expectedNSegments) + return -EINVAL; + + segmentsSum_ = std::accumulate(segments_.begin(), segments_.end(), 0.0f); + + defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); + controls[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + + return 0; +} + +/** + * \brief Configure the gamma correction algorithm + * \param[out] state The gamma correction algorithm's active state + * + * Reset to the default gamma correction value. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::configure() + */ +void GammaAlgorithmBase::configure(gamma::ActiveState &state) +{ + state.gamma = defaultGamma_; +} + +/** + * \brief Queue a request to the gamma correction algorithm + * \param[in] state The algorithm's active state + * \param[in] frame The current frame number + * \param[in] context The algorithm's frame context + * \param[in] controls The ControlList that was queued with the request + * + * Queue a new request to the gamma correction algorithm and handle any relevant + * controls that were queued. The only control currently handled is: + * + * - controls::Gamma + * + * If a control with that ID is queued the value is stored in \a state and + * \a context. + * + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::queueRequest() + */ +void GammaAlgorithmBase::queueRequest(gamma::ActiveState &state, + const uint32_t frame, + gamma::FrameContext &context, + const ControlList &controls) +{ + if (frame == 0) + context.update = true; + + const auto &gamma = controls.get(controls::Gamma); + if (gamma) { + state.gamma = *gamma; + context.update = true; + LOG(Gamma, Info) << "Set gamma to " << *gamma; + } + + context.gamma = state.gamma; +} + +/** + * \brief Populate metadata with the gamma correction values for a frame + * \param[in] context The frame context + * \param[out] metadata The ControlList of metadata for a frame + * + * Report the gamma value used to calculate the correction curve that was + * applied to a frame. + */ +void GammaAlgorithmBase::process(gamma::FrameContext &context, ControlList &metadata) +{ + metadata.set(controls::Gamma, context.gamma); +} + +/** + * \var GammaAlgorithmBase::nLutNodes_ + * \brief The number of gamma LUT sampling points + */ + +/** + * \var GammaAlgorithmBase::defaultGamma_ + * \brief The default gamma parameter + */ + +/** + * \var GammaAlgorithmBase::segments_ + * \brief The vector of segment sizes describing the space between knee-points + */ + +/** + * \var GammaAlgorithmBase::segmentsSum_ + * \brief The sum of \a GammaAlgorithmBase::segments_ + */ + +/** + * \class GammaAlgorithm + * \brief The libipa gamma correction algorithm + * \tparam nLutNodes The number of gamma LUT sampling points + * \tparam UQ The fixedpoint representation of the gamma correction values + * + * Gamma correction adjusts for the differences in the way light is perceived + * by a camera and the human eye by applying a function to the input values. + * The GammaAlgorithm class facilitates this by building a piecewise linear + * function from a gamma parameter and supplying it in the hardware-specific + * formats defined by the IPA algorithms. + * + * IPA modules are expected to store an instance of GammaAlgorithm as a class + * member, templated with the format and number of knee-points in the PWL + * expected by their hardware and then call its functions in their overload of + * the Algorithm class's function. + * + * When an application queues a new value for the gamma parameter with a + * Request, the GammaAlgorithm will recalculate and populate the new LUT to be + * sent to the ISP. + * + * Useful links: + * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm + * - https://en.wikipedia.org/wiki/SRGB + */ + +/** + * \fn GammaAlgorithm::prepare() + * \tparam T The type of data expected by the hardware's look-up table + * \param[in] context The frame context + * \param[out] lut The Span into which to place the calculated look-up table + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/gamma.h b/src/ipa/libipa/gamma.h new file mode 100644 index 0000000000000000000000000000000000000000..94be8c7a2328e7b5a3618b0d65989ee511cece09 --- /dev/null +++ b/src/ipa/libipa/gamma.h @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * + * libIPA Gamma correction algorithm + */ + +#pragma once + +#include +#include + +#include +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "fixedpoint.h" + +namespace libcamera { + +namespace ipa { + +LOG_DECLARE_CATEGORY(Gamma) + +namespace gamma { + +struct ActiveState { + double gamma; +}; + +struct FrameContext { + double gamma; + bool update; +}; + +} /* namespace gamma */ + +class GammaAlgorithmBase +{ +public: + GammaAlgorithmBase(unsigned int nLutNodes) + : nLutNodes_(nLutNodes) + { + } + + int init(ControlInfoMap::Map &controls, const ValueNode &tuningData, + Span segments = {}); + + void configure(gamma::ActiveState &state); + void queueRequest(gamma::ActiveState &state, const uint32_t frame, + gamma::FrameContext &context, const ControlList &controls); + void process(gamma::FrameContext &context, ControlList &metadata); + +protected: + unsigned int nLutNodes_; + float defaultGamma_; + std::vector segments_; + unsigned int segmentsSum_; +}; + +template +class GammaAlgorithm : public GammaAlgorithmBase +{ +public: + GammaAlgorithm() + : GammaAlgorithmBase(nLutNodes) + { + } + + template + void prepare(gamma::FrameContext &context, Span lut) + { + float x = 0; + + for (unsigned int i = 0; i < nLutNodes_; i++) { + float gamma = std::pow(x / segmentsSum_, + 1.0 / context.gamma); + lut[i] = UQ(gamma).quantized(); + + LOG(Gamma, Debug) << "LUT[" << i << "]=" << gamma + << "(" << lut[i] << ")"; + + if (i < segments_.size()) + x += segments_[i]; + } + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index da6ea0c5e13000f78b2196c7334610c350f1ad13..565da9be9059f2167d107061d643e58202e655ef 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -12,6 +12,7 @@ libipa_headers = files([ 'exposure_mode_helper.h', 'fc_queue.h', 'fixedpoint.h', + 'gamma.h', 'histogram.h', 'interpolator.h', 'lsc.h', @@ -37,6 +38,7 @@ libipa_sources = files([ 'exposure_mode_helper.cpp', 'fc_queue.cpp', 'fixedpoint.cpp', + 'gamma.cpp', 'histogram.cpp', 'interpolator.cpp', 'lsc.cpp', From patchwork Fri Jun 26 13:05:54 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27067 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2683FC3306 for ; Fri, 26 Jun 2026 13:06:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E9DD665F13; Fri, 26 Jun 2026 15:06:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EMMo9yl2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BDEF765EE7 for ; Fri, 26 Jun 2026 15:06:06 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C13AE1AE5; Fri, 26 Jun 2026 15:05:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479125; bh=MV1OudgVxwqKcw+PZWyvAFv5rAPA2Ey0KE5JXKGr7Os=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=EMMo9yl2H4szF7oOGBEbJAWatIOJgI5OoNjD6ji9Tu2Ot1mgM24az32wbF6p9R8eH oRV7ehQyROv7wZrT7velWu9ptDbSR9OlfBcVOQW66ObC541b9V+S9h3W7fMJZdKZ2O PO4g6q3GNPe64tFvm+tvopO8UEFOKL4PRFIdgR2Q= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:54 +0100 Subject: [PATCH v2 07/12] ipa: ipu3: ToneMapping: Convert to use GammaAlgorithm MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-7-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8593; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=MV1OudgVxwqKcw+PZWyvAFv5rAPA2Ey0KE5JXKGr7Os=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk62xG6GoNzv874hDnaI3WUNoRnQZZpoPcc5 X+d3o6QHAyJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OgAKCRDISVd6bEV1 MkBqD/441lIw/ISxUkps4lFW9fNmRwQ8pOUvU0rMLoFVn10kpKlA9hQ3CZTqPRYGGyvQqQUxy9J KdbSLZQWkHpi023zK/rv09oNgW16r7KGDBBLOGUOaRErhRS1zRhlZhhZgDCodeJ+FVbcQ8m085W cQ/B+C1VRNUUK0ZBVSfWKRwgkTyk/sPFWPOY3syoA9Qq3dKZPsizrLRlF15pWB9e4RHl+/MWVhq acac8/zhcj+4+KOh4LVbte/8Ud0zsOW70lGSoAjoGBlkMSvC/T2Co4PCt1i2ZPnjDDvKdZ3Smpj peugQaoPPKTHN+zLu/Pl85DHA0ShZyrJ10omHgTNvjEa2CJtx81gMkYKbfF2BNvF7MtFr0MpHJP nsAjdVvodALZMJ5ebctPFHQ42eIvOd4ILsFoQeGOXpfT+wINV0mk/Nd7Xa/tsZk6ADrYEi5rgFO ttcqEbQGLI5EVl/QwlPZeuCpzgOioA24WYUz8jVK55fYeerxbvZnQVYrZE9T974hMKug6MzoDGA 2CgTWcHQ8qu7iggg3eGlZbAUgXMZS5oWgJRFCC/Fx6Z4e2zNp/YybzDz50TjFW2rb0+v6qaOo/y 092cY24IvD2bI6jDAN5wcJ9bxAvS6Qbh+dJsQMFlZlyNey5BCbhxHwwdjGosl5yyE+EKd+cdClk vuBEYB44BQKHzUA== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Convert the IPU3 ToneMapping algorithm to use the new GammaAlgorithm base class. This gives us configurable gamma via the tuning files and at runtime using the Gamma control. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v2: - Fixed includes - Minor styling changes --- src/ipa/ipu3/algorithms/tone_mapping.cpp | 73 +++++++++++++++++--------------- src/ipa/ipu3/algorithms/tone_mapping.h | 12 +++++- src/ipa/ipu3/ipa_context.cpp | 24 +++++------ src/ipa/ipu3/ipa_context.h | 8 ++-- 4 files changed, 63 insertions(+), 54 deletions(-) diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 160338c139448cc9a0bc1fe2400c335a96f68f73..ea17abff8167238dfc5d65f6bb64754e8a7834ef 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -10,6 +10,8 @@ #include #include +#include + /** * \file tone_mapping.h */ @@ -27,10 +29,17 @@ namespace ipa::ipu3::algorithms { */ ToneMapping::ToneMapping() - : gamma_(1.0) { } +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int ToneMapping::init(IPAContext &context, const ValueNode &tuningData) +{ + return gammaAlgo_.init(context.ctrlMap, tuningData); +} + /** * \brief Configure the tone mapping given a configInfo * \param[in] context The shared IPA context @@ -41,12 +50,21 @@ ToneMapping::ToneMapping() int ToneMapping::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { - /* Initialise tone mapping gamma value. */ - context.activeState.toneMapping.gamma = 0.0; - + gammaAlgo_.configure(context.activeState.gamma); return 0; } +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void ToneMapping::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + gammaAlgo_.queueRequest(context.activeState.gamma, frame, + frameContext.gamma, controls); +} + /** * \brief Fill in the parameter structure, and enable gamma control * \param[in] context The shared IPA context @@ -59,14 +77,21 @@ int ToneMapping::configure(IPAContext &context, */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, ipu3_uapi_params *params) { - /* Copy the calculated LUT into the parameters buffer. */ - memcpy(params->acc_param.gamma.gc_lut.lut, - context.activeState.toneMapping.gammaCorrection.lut, - IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES * - sizeof(params->acc_param.gamma.gc_lut.lut[0])); + if (!frameContext.gamma.update) + return; + + /* + * Unfortunately necessary given the IPU3's gamma uAPI struct has the + * __packed attribute. + */ + uint16_t *lutData = reinterpret_cast( + __builtin_assume_aligned(params->acc_param.gamma.gc_lut.lut, 16)); + Span lut{ lutData, kNumLutNodes }; + + gammaAlgo_.prepare(frameContext.gamma, lut); /* Enable the custom gamma table. */ params->use.acc_gamma = 1; @@ -84,33 +109,13 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, +void ToneMapping::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats, [[maybe_unused]] ControlList &metadata) { - /* - * Hardcode gamma to 1.1 as a default for now. - * - * \todo Expose gamma control setting through the libcamera control API - */ - gamma_ = 1.1; - - if (context.activeState.toneMapping.gamma == gamma_) - return; - - struct ipu3_uapi_gamma_corr_lut &lut = - context.activeState.toneMapping.gammaCorrection; - - for (uint32_t i = 0; i < std::size(lut.lut); i++) { - double j = static_cast(i) / (std::size(lut.lut) - 1); - double gamma = std::pow(j, 1.0 / gamma_); - - /* The output value is expressed on 13 bits. */ - lut.lut[i] = gamma * 8191; - } - - context.activeState.toneMapping.gamma = gamma_; + gammaAlgo_.process(frameContext.gamma, metadata); } REGISTER_IPA_ALGORITHM(ToneMapping, "ToneMapping") diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index b2b380108e014b3d5ee7b93bcdd948dea4d2302d..3437b7fcbbb70c9bbc895b514b354d26c1d707c6 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -7,6 +7,11 @@ #pragma once +#include + +#include +#include + #include "algorithm.h" namespace libcamera { @@ -18,7 +23,11 @@ class ToneMapping : public Algorithm public: ToneMapping(); + int init(IPAContext &context, const ValueNode &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, const uint32_t frame, @@ -27,7 +36,8 @@ public: ControlList &metadata) override; private: - double gamma_; + static constexpr unsigned int kNumLutNodes = IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES; + GammaAlgorithm> gammaAlgo_; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 469905a43c4300bfa7a7fee5777f4a51837cf4c6..d3bcd7a81fe010d17d1437bf7f6cedce83a9be5c 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -119,6 +119,11 @@ namespace libcamera::ipa::ipu3 { * \brief Active colour Correction Matrix parameters for the IPA */ +/** + * \var IPAActiveState::gamma + * \brief Active gamma correction parameters for the IPA + */ + /** * \var IPASessionConfiguration::sensor * \brief Sensor-specific configuration of the IPA @@ -149,20 +154,6 @@ namespace libcamera::ipa::ipu3 { * The gain should be adapted to the sensor specific gain code before applying. */ -/** - * \var IPAActiveState::toneMapping - * \brief Context for ToneMapping and Gamma control - * - * \var IPAActiveState::toneMapping.gamma - * \brief Gamma value for the LUT - * - * \var IPAActiveState::toneMapping.gammaCorrection - * \brief Per-pixel tone mapping implemented as a LUT - * - * The LUT structure is defined by the IPU3 kernel interface. See - * struct ipu3_uapi_gamma_corr_lut for further details. - */ - /** * \struct IPAFrameContext * \brief IPU3-specific FrameContext @@ -187,4 +178,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame colour Correction Matrix parameters for the IPA */ +/** + * \var IPAFrameContext::gamma + * \brief Per-frame gamma correction parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 6445db2603645d804b6caa40cde65099c4e57511..ce496ed477f4cbe2653aedada3381258396d92c8 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -18,6 +18,7 @@ #include #include #include +#include namespace libcamera { @@ -64,11 +65,7 @@ struct IPAActiveState { ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; - - struct { - double gamma; - struct ipu3_uapi_gamma_corr_lut gammaCorrection; - } toneMapping; + ipa::gamma::ActiveState gamma; }; struct IPAFrameContext : public FrameContext { @@ -79,6 +76,7 @@ struct IPAFrameContext : public FrameContext { ipa::awb::FrameContext awb; ipa::ccm::FrameContext ccm; + ipa::gamma::FrameContext gamma; }; struct IPAContext { From patchwork Fri Jun 26 13:05:55 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27068 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1F17DC3307 for ; Fri, 26 Jun 2026 13:06:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 87FA065EF9; Fri, 26 Jun 2026 15:06:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XVgBgq68"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D2D565EF5 for ; Fri, 26 Jun 2026 15:06:07 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1D3891AED; Fri, 26 Jun 2026 15:05:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479126; bh=qsDto6FcSGcabbsNsDlmV91IEE5AdG7vMLLPU7cxzJ0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XVgBgq68ZrUEqCjjorAiWeDoE0j6L4VdT3DNZkApvKrPMS5gzjVUlP1Y7BPMFTKnT ygqcPDQjsWtcwBeIJe4ZvAjS2bwysLIBEBuqUT/Am5dXHcy1L68pqkMzuaW5mT5wN5 Ak+tuqcvEyLrND+V4i/NVZ8a1VR/ZdWZ/enN0Euk= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:55 +0100 Subject: [PATCH v2 08/12] ipa: ipu3: Add Ccm to uncalibrated.yaml MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-8-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=838; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=qsDto6FcSGcabbsNsDlmV91IEE5AdG7vMLLPU7cxzJ0=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk64yWz4Ks+D8gf58zKFuRSTJgZ5Sr8t3MIm 3Dx/r5FvNOJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OgAKCRDISVd6bEV1 Mk5sD/0dYlC5TdYPHqahRQ4PaUwSvrdTt6OFcFrUz96OueQ9v9ijZQgjgdkDa8g9UUJDraya8rX PfnZIlYoWQouyzyaCYffcJY8DO8hXi86uPbI3+vfkPpvaSjhiGBRlTuK43uMKps6yZzrppE0HE/ 50m7uYYv6BcFj6gjjbr0vDDXVFwQOpOYnvfxev1idj3Ea2wL9UJygAlou67K2GQPvEJbLnbqGId cUVdKar3/uJ9SZMwx4qQXcQKCytJEXWHi4keqNFY52Xr1mJor7rwYl+7e+zP2IsngkoMgTBevSP qsQpdEemfksKPaN9BNH94CQtN3anws4gbA8xyMI4odJRg4FdpKz9YBFbqwoEAUmz94Gy+/G8tre T4VAxwTUiywpJ24qPTWyMgGZXV9k3S1Jw6vlKo/cTngvOOl08DFRxmDJ46+PpgOwyAJGKILFuE7 7eb7Zvoge36Nbs1yQy2sdc1Yn3QEhx9xzCuyEfJPaOFD0UCWNu01q+uVcd1CTqB+qmEJev6+qJ7 h8hgQYn64+nzW4Bssk+TW5WdawpmCCdkTgQQlfYLIqlTsOBzk9w6GqOHteog8WNRmuZSEHudV3a fX43XpgjP+LOy4/MBP65StmW3YIgeJiFW/Q0Bu7tTNBanvIlkG4Wq4JseLsHO5A2HTlMhjoeo3H QWSagh4cOBHu1cQ== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the Ccm algorithm to uncalibrated.yaml. Even without calibrated values we should still expose the controls to the application. Reviewed-by: Jacopo Mondi Acked-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v2: - None --- src/ipa/ipu3/data/uncalibrated.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ipa/ipu3/data/uncalibrated.yaml b/src/ipa/ipu3/data/uncalibrated.yaml index 794ab3edbb778bbe14c2dc65e5540413ee7fd536..415ca4578fab57ea731493c978a04e740946197f 100644 --- a/src/ipa/ipu3/data/uncalibrated.yaml +++ b/src/ipa/ipu3/data/uncalibrated.yaml @@ -7,5 +7,6 @@ algorithms: - Agc: - Awb: - BlackLevelCorrection: + - Ccm: - ToneMapping: ... From patchwork Fri Jun 26 13:05:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27069 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CAC5C330A for ; Fri, 26 Jun 2026 13:06:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B842D65F11; Fri, 26 Jun 2026 15:06:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ama2Y2HT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5F7ED65EF6 for ; Fri, 26 Jun 2026 15:06:07 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6AA2219EC; Fri, 26 Jun 2026 15:05:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479126; bh=msbC9/WWRfcIej2bHzjxQUvbu1GUuXBibfEkb53E2ao=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ama2Y2HTYSEmuZMLJtnipxwOu+c/Oo+HVFqJtuxUwLzoeK8U3JQPn8CHjoTWTPvuE IP4Pza9IQV/KQUPWN+YvfqTnSvNvvUktfPjhdwRLTlL9I+qgaAlrREaYjSQaQaCFGL XjekcFaiS+EpIIaBAXwjhoRpgThR36cDVUr+2kvo= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:56 +0100 Subject: [PATCH v2 09/12] ipa: rkisp1: goc: Re-work to use GammaAlgorithm class MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-9-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6650; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=msbC9/WWRfcIej2bHzjxQUvbu1GUuXBibfEkb53E2ao=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk7nRGYoCySX05TI85PcWA/fffVeyeTyXbQ8 7RkqUyuz3yJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OwAKCRDISVd6bEV1 MgDFEACMXlfBgcu++uMkLg4FfvjVSUw1wtNmQcXrSVKcbfYp802NFUq5kk8X6XL/pxY55/mtjU9 PXikAsoirw7vU26g9+PUhSCUCz8/ZD/wp/BJ1LpxMCNoB4ackQyMCWoQypmQ/R7TRID4Pihv/9I bYcZjSOKkm76UMNlPrTWO0RkFSnEUXf7Y4WofvR4cfwfdCwxvAqqtDuFerpzZo/Bha2eahcD/5q LWFkfCvz28yZZmhzI2HYAPp195G/klul6minBV7H3QgV8c44hjcAGfbdA6i+P6oACtHP6+xfTDS wS2QkG7Eku68zd5pPbpdfeIBxGRsytWbadqRPVqCwBLd/zdar9onjlNPFqxrvFmxXGKjVf1k0bd +pEKjDoDh/EmFuJjefHOy7tRrrCvfp3cMbr7qXlRFTCMYV1+Bv6ilfvU9djeqpxtVCgn57Im8dd TxnoVg5CnO2h702dw0B3RpV04p+lhHZ4M3HL4uNbTirgW1MZQg6rwGpFlOeysZJla4n1qW3ELTv mU1u2B5meaUc+EW6bUqqhURofPFaLDnnu4Bija0NqKgTkdl65pxpzNXr8xT6YucF+RJ8XPghwaO TyDJr8U9O9T7XoLy6cZQOrr1dotHpI5GQMaUkTYblguJLrGnhSyxP8fTI2OnngKhAohinIrqDUJ MONHazMMs9P2b/A== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Re-work the RkISP1 Gamma Out Correction algorithm to use the new GammaAlgorithm class from libipa, which allows us to share an implementation with the other IPAs. Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v2: - Fixed includes - Dropped the comment explaining how gamma correction is implemented - it's on libipa gamma now - Dropped unused variable kDefaultGamma --- src/ipa/rkisp1/algorithms/goc.cpp | 63 +++++++++++---------------------------- src/ipa/rkisp1/algorithms/goc.h | 7 ++++- src/ipa/rkisp1/ipa_context.h | 10 ++----- 3 files changed, 26 insertions(+), 54 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/goc.cpp b/src/ipa/rkisp1/algorithms/goc.cpp index e8f64bf3d5e028c2013d37b7ef9fe90b737fad5f..e3bea4bbef186dde28fdd5a3aa14b7662a9ffe69 100644 --- a/src/ipa/rkisp1/algorithms/goc.cpp +++ b/src/ipa/rkisp1/algorithms/goc.cpp @@ -6,10 +6,11 @@ */ #include "goc.h" +#include #include #include -#include +#include #include @@ -29,21 +30,13 @@ namespace ipa::rkisp1::algorithms { * \class GammaOutCorrection * \brief RkISP1 Gamma out correction * - * This algorithm implements the gamma out curve for the RkISP1. It defaults to - * a gamma value of 2.2. - * - * As gamma is internally represented as a piecewise linear function with only - * 17 knots, the difference between gamma=2.2 and sRGB gamma is minimal. - * Therefore sRGB gamma was not implemented as special case. - * - * Useful links: - * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm - * - https://en.wikipedia.org/wiki/SRGB + * This algorithm implements the gamma out curve for the RkISP1 using the + * libipa GammaAlgorithm class. */ LOG_DEFINE_CATEGORY(RkISP1Gamma) -const float kDefaultGamma = 2.2f; +static constexpr unsigned int kNumLutSegments = RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10 - 1; /** * \copydoc libcamera::ipa::Algorithm::init @@ -57,10 +50,12 @@ int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) return -EINVAL; } - defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); - context.ctrlMap[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + std::array segments = { + 64, 64, 64, 64, 128, 128, 128, 128, + 256, 256, 256, 512, 512, 512, 512, 512 + }; - return 0; + return gammaAlgo_.init(context.ctrlMap, tuningData, segments); } /** @@ -69,7 +64,7 @@ int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) int GammaOutCorrection::configure(IPAContext &context, [[maybe_unused]] const IPACameraSensorInfo &configInfo) { - context.activeState.goc.gamma = defaultGamma_; + gammaAlgo_.configure(context.activeState.gamma); return 0; } @@ -80,17 +75,8 @@ void GammaOutCorrection::queueRequest(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const ControlList &controls) { - if (frame == 0) - frameContext.goc.update = true; - - const auto &gamma = controls.get(controls::Gamma); - if (gamma) { - context.activeState.goc.gamma = *gamma; - frameContext.goc.update = true; - LOG(RkISP1Gamma, Debug) << "Set gamma to " << *gamma; - } - - frameContext.goc.gamma = context.activeState.goc.gamma; + gammaAlgo_.queueRequest(context.activeState.gamma, frame, + frameContext.gamma, controls); } /** @@ -104,29 +90,14 @@ void GammaOutCorrection::prepare(IPAContext &context, ASSERT(context.hw.numGammaOutSamples == RKISP1_CIF_ISP_GAMMA_OUT_MAX_SAMPLES_V10); - if (!frameContext.goc.update) + if (!frameContext.gamma.update) return; - /* - * The logarithmic segments as specified in the reference. - * Plus an additional 0 to make the loop easier - */ - static constexpr std::array segments = { - 64, 64, 64, 64, 128, 128, 128, 128, 256, - 256, 256, 512, 512, 512, 512, 512, 0 - }; - auto config = params->block(); config.setEnabled(true); - __u16 *gamma_y = config->gamma_y; - - unsigned x = 0; - for (const auto [i, size] : utils::enumerate(segments)) { - gamma_y[i] = std::pow(x / 4096.0, 1.0 / frameContext.goc.gamma) * 1023.0; - x += size; - } - + Span lut{ config->gamma_y }; + gammaAlgo_.prepare(frameContext.gamma, lut); config->mode = RKISP1_CIF_ISP_GOC_MODE_LOGARITHMIC; } @@ -139,7 +110,7 @@ void GammaOutCorrection::process([[maybe_unused]] IPAContext &context, [[maybe_unused]] const rkisp1_stat_buffer *stats, ControlList &metadata) { - metadata.set(controls::Gamma, frameContext.goc.gamma); + gammaAlgo_.process(frameContext.gamma, metadata); } REGISTER_IPA_ALGORITHM(GammaOutCorrection, "GammaOutCorrection") diff --git a/src/ipa/rkisp1/algorithms/goc.h b/src/ipa/rkisp1/algorithms/goc.h index bd79fe19cc86b8aefa2603e98e9d7130b44105d9..767d707b2758a89db286ab62d8f20d7a01912a2a 100644 --- a/src/ipa/rkisp1/algorithms/goc.h +++ b/src/ipa/rkisp1/algorithms/goc.h @@ -9,6 +9,11 @@ #include "algorithm.h" +#include + +#include +#include + namespace libcamera { namespace ipa::rkisp1::algorithms { @@ -35,7 +40,7 @@ public: ControlList &metadata) override; private: - float defaultGamma_; + GammaAlgorithm> gammaAlgo_; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 7b4346c54ed2b8850ed147b0ce2b8a20155b80cf..8011c353c4eaade85d0a0007b44de8af7beaa9d6 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -30,6 +30,7 @@ #include "libipa/ccm.h" #include "libipa/fc_queue.h" #include "libipa/fixedpoint.h" +#include "libipa/gamma.h" #include "libipa/lsc.h" namespace libcamera { @@ -127,9 +128,7 @@ struct IPAActiveState { uint8_t sharpness; } filter; - struct { - double gamma; - } goc; + ipa::gamma::ActiveState gamma; struct { double lux; @@ -194,10 +193,7 @@ struct IPAFrameContext : public FrameContext { bool update; } filter; - struct { - double gamma; - bool update; - } goc; + ipa::gamma::FrameContext gamma; struct { uint32_t exposure; From patchwork Fri Jun 26 13:05:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27070 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 66926C3301 for ; Fri, 26 Jun 2026 13:06:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7E3EC65F02; Fri, 26 Jun 2026 15:06:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Z03J81rj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9975265EF7 for ; Fri, 26 Jun 2026 15:06:07 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B9CB71CEF; Fri, 26 Jun 2026 15:05:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479126; bh=KI5tgliKnQMEXyv5fryXUaUERQlQN30zJEGI22nu0SQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Z03J81rjHO2kuoBVHEnj3aNwHEKLtS+QnIVFFPiLEk/Hj8D00SAvCSjFqy1nB3uGZ M91jfZYGjfiRQhgttT9uk+53b/3jNd5PK8SLWgLkmqZ4/QOdr1y3GboWumjso7qJG6 +/EJRqTUdbNWH/mx7AGMo39Uu5xVzLosykVVfppY= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:57 +0100 Subject: [PATCH v2 10/12] ipa: ipu3: Don't pass sensorInfo to updateControls() MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-10-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3929; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=KI5tgliKnQMEXyv5fryXUaUERQlQN30zJEGI22nu0SQ=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk7kzJc6th1kwMjuPdUCnn3MiDgwrCNKggQv o0LDHeT1+WJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OwAKCRDISVd6bEV1 MixFD/4i+UbBsREcZXnseX5XOr6fgvEtbQ+Nfka93xeW+wx5yRg/ZrfcOrr7HxmlJb5PKh23AAt Pk9FDOvjmNK/qFYaIQp7ZuW/92HEzWxiuHYVk1WeTdtq+T+HJxiGXZ7BMzt0OrlW4oxmdK76322 YxylzLP1CiErzwarF52MoIpzymcI+/GdXPLh8Oj+kjOQG3EeO4/xGvinQIHdeMs8TeMR9rAuzaq r6iRk9EMB/4lEtHlzY840OEPPkNTNDdGx+5mslovq8zEoCAwq3VJpI6oPwVoVnaI9vMbJ35h/K+ iSmgSbOhtk5PnK7WLiseOoO9xqa1u3X8bW10TzcmT9JrbWW5yrF4rUDTU4QOvwKAfCG0Au3JieW ZcuH6tEuqQ/oZMrHUmlursnmeccj45X5FMlETKhRsoIWscbgC97T+WKpaCOPDBc5sZAREQPoBXa smQVv/Le0411pUw1ielxeoYveQaCBU2qza48p5eVgKDowMT/Ak0/Qq/HrdBuwqsw46NnYrdO8XE A9SyxdeBV1p4gkssPxoVlxGm0NOI5/W01vnrUJB4Hr4xe6QZnabpLd6YlPD/WNx8+RgbRSvCQoY xiGnvODfZclddtvdXdpXoCw4XkksFNj9yB7FUqpg7w02NYK8jaEzuYURbKm7ci2wHRnE5Tbw4YY WS7G1S6eLV+3glA== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The updateControls() function uses the configured output size of the sensor to set the ranges for vertical blanking and frame duration controls. We currently pass sensorInfo as a parameter, but it's also stored as a member of class IPAIPU3 and so doesn't need to be passed to updateControls at all(). Drop the parameter and just use the class member instead. Signed-off-by: Daniel Scally --- Changes in v2: - New patch --- src/ipa/ipu3/ipu3.cpp | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 4bdc4b7677fe9703fb968ca91967c68742454514..b1c176e68098b24d8fd37e7a0e278cabf0e1c3b2 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -165,8 +165,7 @@ protected: std::string logPrefix() const override; private: - void updateControls(const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, + void updateControls(const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); void updateSessionConfiguration(const ControlInfoMap &sensorControls); @@ -239,8 +238,7 @@ void IPAIPU3::updateSessionConfiguration(const ControlInfoMap &sensorControls) * - controls::ExposureTime * - controls::FrameDurationLimits */ -void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, +void IPAIPU3::updateControls(const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) { ControlInfoMap::Map controls{}; @@ -267,19 +265,19 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, */ const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; uint32_t hblank = v4l2HBlank.def().get(); - uint32_t lineLength = sensorInfo.outputSize.width + hblank; + uint32_t lineLength = sensorInfo_.outputSize.width + hblank; const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; std::array frameHeights{ - v4l2VBlank.min().get() + sensorInfo.outputSize.height, - v4l2VBlank.max().get() + sensorInfo.outputSize.height, - v4l2VBlank.def().get() + sensorInfo.outputSize.height, + v4l2VBlank.min().get() + sensorInfo_.outputSize.height, + v4l2VBlank.max().get() + sensorInfo_.outputSize.height, + v4l2VBlank.def().get() + sensorInfo_.outputSize.height, }; std::array frameDurations; for (unsigned int i = 0; i < frameHeights.size(); ++i) { uint64_t frameSize = lineLength * frameHeights[i]; - frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); + frameDurations[i] = frameSize / (sensorInfo_.pixelRate / 1000000U); } controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], frameDurations[1], @@ -313,6 +311,7 @@ int IPAIPU3::init(const IPASettings &settings, context_.configuration = {}; context_.configuration.sensor.lineDuration = sensorInfo.minLineLength * 1.0s / sensorInfo.pixelRate; + sensorInfo_ = sensorInfo; /* Load the tuning data file. */ File file(settings.configurationFile); @@ -346,7 +345,7 @@ int IPAIPU3::init(const IPASettings &settings, return ret; /* Initialize controls. */ - updateControls(sensorInfo, sensorControls, ipaControls); + updateControls(sensorControls, ipaControls); return 0; } @@ -488,7 +487,7 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, calculateBdsGrid(configInfo.bdsOutputSize); /* Update the camera controls using the new sensor settings. */ - updateControls(sensorInfo_, sensorCtrls_, ipaControls); + updateControls(sensorCtrls_, ipaControls); /* Update the IPASessionConfiguration using the sensor settings. */ updateSessionConfiguration(sensorCtrls_); From patchwork Fri Jun 26 13:05:58 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27071 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3E2D0C330F for ; Fri, 26 Jun 2026 13:06:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CD37765EEF; Fri, 26 Jun 2026 15:06:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cEgXHgkd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DACDC65EF9 for ; Fri, 26 Jun 2026 15:06:07 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 035D31D00; Fri, 26 Jun 2026 15:05:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479127; bh=A9hGuIqdIKWPEToYxKgGT5lgPd0mj/cV61qp5al+MzI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cEgXHgkdz8YFkciGH9i3Yd1YoyjB4DIjA+YrDeCHrreSLBPQkV6mSUkVcCx1Nq/qs ukaxMEmcbCmtUMEWNBm49bNVdMtzSmms2AayAF1PIiEDDZ7Nx7e3w6r8tN1gez0Hdu 0I6FQ7jSgHud8vElBEQtPwTWbxD/jlJwvR6NVJ80= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:58 +0100 Subject: [PATCH v2 11/12] ipa: ipu3: Add IPACameraSensorInfo to IPAContext MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-11-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4825; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=A9hGuIqdIKWPEToYxKgGT5lgPd0mj/cV61qp5al+MzI=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk7TS6rYD7kinKU0ZIJqrVX5t2HwQ0AYiP2X bAn3bc0AQKJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OwAKCRDISVd6bEV1 MnoeD/9jA8tzoMHWOdGpgOQJ+ZQOS4pBOAmlG5co0l6duQnJBusJkYAqw7Nh8g7naTnQUYSIRW8 81KkQAackTZsx1oF9a6yu1xxmIct7QSBMW+VCCTg/5ZUd+xJF1A5vWlbMSZCt5zBbFUZKR0VVAq HsXtn4ma9eoBF6AFvk78cfZ6fNbJ1Zku765CccyC2wPzwmRKwi4KGpw2SZziELTiISHW6dEWsZ+ +bcZen3b1pMjVNhNauZ5y5mROPeJauTgzk/D+4EpueHsABkRsxxayYfSTiJKg0RGCjfb1f3Q4nG POsUJv23wZrWk8FxQ/8NQrCxvHw6YKRg4w3AoiGIWmH6fjW/dGgfxmWYtXiAxiz2X8B+9RjVIO/ xkI5U9avJ/OmaoWtwRgMQE7qLCZPNqfW5ZfA0GRzLxNf1BleM89VAkZXZFkwd8rzMO4CNgzML/I jq7sTV8zdoK+rsuQCrssuJIfWwl1FOObBN9L9+YiBADwi3yNULseAYX3Gj0+X5HeHZDqjk/MXaI I6GT924OlLZMT01AWonmy6vhNLDmbHoPfbodg0eBnvkjAZBQqx+o3rGqYOKqLzlp7WnernJlU28 uLAyKmyTLKD6nOGjbaIXhd099O1Bh7I4qPeFanyNBPydd1kqhbKDRgOoLXJrxg7JbfaULeyAyds clObiOt0mW0559g== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add an instance of IPACameraSensorInfo to the IPAContext and fill it during IPAIPU3::init(). This will allow us to pass sensor information to algorithms at init time. This means that IPAIPU3::sensorInfo_ is no longer needed, as the instance stored in the context is accessible everywhere that that was previously used. Drop it. Signed-off-by: Daniel Scally --- Changes in v2: - Dropped the sensorInfo_ class member, since this replaces it --- src/ipa/ipu3/ipa_context.cpp | 3 +++ src/ipa/ipu3/ipa_context.h | 3 +++ src/ipa/ipu3/ipu3.cpp | 20 +++++++++----------- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index d3bcd7a81fe010d17d1437bf7f6cedce83a9be5c..9537802ceca5018118bdf4c71ef361c20fc44bfb 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -54,6 +54,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAContext::ctrlMap * \brief A ControlInfoMap::Map of controls populated by the algorithms + * + * \var IPAContext::sensorInfo + * \brief An IPACameraSensorInfo instance holding information about the sensor */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index ce496ed477f4cbe2653aedada3381258396d92c8..d650f2fe1ad8eab91b7128a47c9690e42b1595f1 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -15,6 +15,8 @@ #include #include +#include + #include #include #include @@ -91,6 +93,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + IPACameraSensorInfo sensorInfo; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b1c176e68098b24d8fd37e7a0e278cabf0e1c3b2..c2d3e62d12f4aba9e3d008025c8922a71d45d36f 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -177,8 +177,6 @@ private: ControlInfoMap sensorCtrls_; ControlInfoMap lensCtrls_; - IPACameraSensorInfo sensorInfo_; - /* Interface to the Camera Helper */ std::unique_ptr camHelper_; @@ -265,19 +263,19 @@ void IPAIPU3::updateControls(const ControlInfoMap &sensorControls, */ const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; uint32_t hblank = v4l2HBlank.def().get(); - uint32_t lineLength = sensorInfo_.outputSize.width + hblank; + uint32_t lineLength = context_.sensorInfo.outputSize.width + hblank; const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; std::array frameHeights{ - v4l2VBlank.min().get() + sensorInfo_.outputSize.height, - v4l2VBlank.max().get() + sensorInfo_.outputSize.height, - v4l2VBlank.def().get() + sensorInfo_.outputSize.height, + v4l2VBlank.min().get() + context_.sensorInfo.outputSize.height, + v4l2VBlank.max().get() + context_.sensorInfo.outputSize.height, + v4l2VBlank.def().get() + context_.sensorInfo.outputSize.height, }; std::array frameDurations; for (unsigned int i = 0; i < frameHeights.size(); ++i) { uint64_t frameSize = lineLength * frameHeights[i]; - frameDurations[i] = frameSize / (sensorInfo_.pixelRate / 1000000U); + frameDurations[i] = frameSize / (context_.sensorInfo.pixelRate / 1000000U); } controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], frameDurations[1], @@ -311,7 +309,7 @@ int IPAIPU3::init(const IPASettings &settings, context_.configuration = {}; context_.configuration.sensor.lineDuration = sensorInfo.minLineLength * 1.0s / sensorInfo.pixelRate; - sensorInfo_ = sensorInfo; + context_.sensorInfo = sensorInfo; /* Load the tuning data file. */ File file(settings.configurationFile); @@ -464,7 +462,7 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, return -ENODATA; } - sensorInfo_ = configInfo.sensorInfo; + context_.sensorInfo = configInfo.sensorInfo; lensCtrls_ = configInfo.lensControls; @@ -475,8 +473,8 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, /* Initialise the sensor configuration. */ context_.configuration.sensor.lineDuration = - sensorInfo_.minLineLength * 1.0s / sensorInfo_.pixelRate; - context_.configuration.sensor.size = sensorInfo_.outputSize; + context_.sensorInfo.minLineLength * 1.0s / context_.sensorInfo.pixelRate; + context_.configuration.sensor.size = context_.sensorInfo.outputSize; /* * Compute the sensor V4L2 controls to be used by the algorithms and From patchwork Fri Jun 26 13:05:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 27072 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D2041C3318 for ; Fri, 26 Jun 2026 13:06:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AD17965F19; Fri, 26 Jun 2026 15:06:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Wd749cbz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32D2565EE1 for ; Fri, 26 Jun 2026 15:06:08 +0200 (CEST) Received: from [127.0.1.1] (chfd-03-b2-v4wan-176392-cust229.vm15.cable.virginm.net [82.19.20.230]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3FD561E7; Fri, 26 Jun 2026 15:05:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1782479127; bh=FIBgslO/k1aow8xgwgmZBi8HwhIqX3xLhKNM4OUPFgY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Wd749cbzBOuQWBojgwuu2NovyKIT2IwMtS6FyRI6qrKDFeCDH6/V/US0K3sYVI2ib nLP4+GZJPA6dBPM8AzWcvjuBrsbC0n/ribQoHlBbB1SMZOkEQ+IVYT13P6Az4xpAl+ 0yMpuusagRFSmYXYBW2MO5KWl59eF9hv1mbz/AVc= From: Daniel Scally Date: Fri, 26 Jun 2026 14:05:59 +0100 Subject: [PATCH v2 12/12] ipa: ipu3: Add Lens Shading Correction algorithm MIME-Version: 1.0 Message-Id: <20260626-ipu3-libipa-rework-v2-12-41546e23de3e@ideasonboard.com> References: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> In-Reply-To: <20260626-ipu3-libipa-rework-v2-0-41546e23de3e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=15230; i=dan.scally@ideasonboard.com; h=from:subject:message-id; bh=FIBgslO/k1aow8xgwgmZBi8HwhIqX3xLhKNM4OUPFgY=; b=owEBbQKS/ZANAwAKAchJV3psRXUyAcsmYgBqPnk760DlwdVPMyl90d5Px7YI6hV2AJOk55IJl 5YZrN56TeiJAjMEAAEKAB0WIQQqyuwyDnZdb+mxmm/ISVd6bEV1MgUCaj55OwAKCRDISVd6bEV1 MlDlEACmNEom8sWYOyDUfkTwV/UiMxEpIJ95xQUhNNRS8hX+tpr7BViRCf3ZlbucC3LMN5Rn4E+ Dccld6KAjn/oWuLOCnWyIiASm5u3OGFDxFZkF6dpFxd9zFbCyoAIG+WBZSUvgQ6OFsF1rJgkbfJ lcxFGeyKASnYDjB/YBVpYuiVsE+V275l5TH8cIBR5s8HjHnTmL+gQZBhaVT4o5uuYQqIi1a/vr/ 5G95vMHhF2y85ExOZQ12B3CLfN85FA5QkrkskCAvV91yo6osUZkx8AmmV3fM/Og8HcBdla+qnki XfaOCymIJqZOtvEuXkroCdpHMap6wQ5GJ7Irh9U0XPdjOhanIrvmJ/U94k7U56pPEtCbG1OzDEE 7B0NmdMtoVaDY7qs3C9Vhsm6vCOSlm0280XKCCkiZLFGBWaJE4c5gk1Rw73LRKdPNNZuLMjhtEA QAqXBSJ434yilHyh56eQ8pXogG/Z1ZnNq59Tj6L07q8BqQ9Ipg/yvPDNwnDiSasq8/uAq8FVv38 uWUmZK6q5wg3YnR0djY+0x7TDEd2VhNPsZVJ+4C8NDfGkTakC2S3jzz998SHMNqP5saueSfg0o2 UkZQko4i5CQ7lc458lWEnhoAvJLnZ9fYYCMF5bg9SfuXod1TXuv1u+qR37ZAPQtU9OkJ5ydKubU HJjkuwrtIqg6yig== X-Developer-Key: i=dan.scally@ideasonboard.com; a=openpgp; fpr=EEC699ACA1B7CB5D31330C0BBD501C2A3546CCF6 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a lens shading correction algorithm for the IPU3 IPA, using the recently implemented libipa base algorithm class. Reviewed-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v2: - Fixed includes - Updated some documentation comments - Check tuning file for "type" key, not "polynomial" --- src/ipa/ipu3/algorithms/lsc.cpp | 296 ++++++++++++++++++++++++++++++++++++ src/ipa/ipu3/algorithms/lsc.h | 60 ++++++++ src/ipa/ipu3/algorithms/meson.build | 1 + src/ipa/ipu3/ipa_context.cpp | 10 ++ src/ipa/ipu3/ipa_context.h | 3 + 5 files changed, 370 insertions(+) diff --git a/src/ipa/ipu3/algorithms/lsc.cpp b/src/ipa/ipu3/algorithms/lsc.cpp new file mode 100644 index 0000000000000000000000000000000000000000..ab9775e6a64ed3d32e7ae3dd90f55f4128da40be --- /dev/null +++ b/src/ipa/ipu3/algorithms/lsc.cpp @@ -0,0 +1,296 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board, Oy + * + * IPU3 Lens Shading Correction algorithm + */ + +#include "lsc.h" + +#include +#include + +#include + +/** + * \file lsc.h + * \brief IPU3 Lens Shading Correction + */ + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +/** + * \class Lsc + * \brief IPU3 Lens Shading Correction algorithm + */ + +LOG_DEFINE_CATEGORY(IPU3Lsc) + +static constexpr unsigned int kMaxNumHCells = 73; +static constexpr unsigned int kMaxNumVCells = 56; +static constexpr int kColourTemperatureQuantization = 10; + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Lsc::init(IPAContext &context, const ValueNode &tuningData) +{ + /* + * The IPU3 lens shading block expects a table of data that isn't of a + * fixed size, but rather is configurable based on 4 parameters: + * + * block_width_log2: The log2 of the horizontal pixel count per cell + * block_height_log2: The log2 of the vertical pixel count per cell + * width: The number of horizontal cells + * height: The number of vertical cells + * + * The constructed grid should be capable of covering the image, but + * ideally won't extend past the edges of the image. Fixing either set + * of parameters for the algorithm as a whole is likely to result in + * suboptimal situations for some sensors, so let's determine them + * programmatically instead. + * + * What we want is the densest possible grid that ideally doesn't extend + * past the edges of the image at all. The maximum grid size is 73x56, + * which gives us the lower bounds on cell size. Unfortunately we can + * only specify sizes in powers of two, which can have the effect of + * making the grid much more coarse. For example for a 2592x1944 input + * image, 2592 / 73 = 35.5...which means we need to set blockWidthLog2 + * to 6 (I.E. 64) and have just 40.5 (or rather 41) cells horizontally. + */ + sensorWidth_ = context.sensorInfo.activeAreaSize.width; + sensorHeight_ = context.sensorInfo.activeAreaSize.height; + + unsigned int cellWidth = (sensorWidth_ + kMaxNumHCells - 1) / kMaxNumHCells; + unsigned int cellHeight = (sensorHeight_ + kMaxNumVCells - 1) / kMaxNumVCells; + + unsigned int minCellWidth = std::bit_ceil(cellWidth); + unsigned int minCellHeight = std::bit_ceil(cellHeight); + + numHCells_ = (sensorWidth_ + minCellWidth - 1) / minCellWidth; + numVCells_ = (sensorHeight_ + minCellHeight - 1) / minCellHeight; + + blockWidthLog2_ = std::bit_width(minCellWidth) - 1; + blockHeightLog2_ = std::bit_width(minCellHeight) - 1; + + LOG(IPU3Lsc, Debug) << "Calculated Grid configuration: " + << numHCells_ << "x" << numVCells_ << " cells of " + << minCellWidth << "x" << minCellHeight << " pixels"; + + /* + * We need to know if we're running the polynomial algorithm or not as + * things will behave slightly differently. + */ + polynomial_ = tuningData["type"].get(false); + + return lscAlgo_.init(tuningData, context.ctrlMap, { + .keys = { "r", "gr", "gb", "b" }, + .numHCells = numHCells_, + .numVCells = numVCells_, + .sensorSize = context.sensorInfo.activeAreaSize + }); +} + +std::vector Lsc::calculatePositions(unsigned int dimension) +{ + std::vector positions(dimension); + for (double i = 0.0; i < dimension; i++) + positions[i] = i / (dimension - 1); + + return positions; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Lsc::configure(IPAContext &context, const IPAConfigInfo &configInfo) +{ + cropWidth_ = configInfo.sensorInfo.analogCrop.width; + cropHeight_ = configInfo.sensorInfo.analogCrop.height; + std::vector xPos = calculatePositions(numHCells_); + std::vector yPos = calculatePositions(numVCells_); + + return lscAlgo_.configure(context.activeState.lsc, + configInfo.sensorInfo.analogCrop, xPos, yPos); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Lsc::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* + * The base algorithm defines the LensShadingCorrectionEnable control + * with a default value of true, but actually the IPU3 driver defaults + * it to off. If this is the first frame, check for the control, but if + * there isn't one, force it on to fulfil the advertised default. + */ + if (frame == 0) { + const auto &lscEnable = controls.get(controls::LensShadingCorrectionEnable); + if (!lscEnable) { + frameContext.lsc.enabled = true; + frameContext.lsc.update = true; + } + } + + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, controls); +} + +static unsigned int quantize(unsigned int value, unsigned int step) +{ + return std::lround(value / static_cast(step)) * step; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Lsc::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, ipu3_uapi_params *params) +{ + uint32_t ct = frameContext.awb.temperatureK; + unsigned int quantizedCt = quantize(ct, kColourTemperatureQuantization); + + if (!frameContext.lsc.update) { + if (!frameContext.lsc.enabled) + return; + + /* + * Add a threshold so that oscillations around a quantization + * step don't lead to constant changes. + */ + if (utils::abs_diff(ct, lastAppliedCt_) < kColourTemperatureQuantization / 2) + return; + + if (quantizedCt == lastAppliedQuantizedCt_) + return; + } + + /* + * This flag tells the kernel driver that it should read the LSC params + * passed from userspace instead of using its cached copy. + */ + params->use.acc_shd = 1; + + /* + * Pass the enabled flag. If we're not enabled, we can then just bail + * out. + */ + ipu3_uapi_shd_config_static *config = ¶ms->acc_param.shd.shd; + config->general.shd_enable = frameContext.lsc.enabled; + + if (!frameContext.lsc.enabled) + return; + + config->grid.width = numHCells_; + config->grid.height = numVCells_; + config->grid.block_width_log2 = blockWidthLog2_; + config->grid.block_height_log2 = blockHeightLog2_; + config->grid.grid_height_per_slice = IPU3_UAPI_SHD_MAX_CELLS_PER_SET / numHCells_; + + /* + * The IPU3's documentation describes the x_start and y_start members + * as follows: + * + * "[X/Y] value of top left corner of sensor relative to ROI + * s13, [-4096, 0], default 0, only negative values." + * + * I interpret that as allowing us to configure the cropped rectangle + * relative to the full grid. That's useful if we're running the tabular + * algorithm, which would otherwise apply the full grid inappropriately. + * If we're running the polynomial one though the calculated grid is + * probably more appropriate than a coarse application of the full grid + * so let's tell the hardware not to bother correcting in that case. + */ + if (polynomial_) { + config->grid.x_start = 0; + config->grid.y_start = 0; + } else { + config->grid.x_start = (cropWidth_ - sensorWidth_) / 2; + config->grid.y_start = (cropHeight_ - sensorHeight_) / 2; + } + + /* No idea what this is, but the docs say it should be set as so */ + config->general.init_set_vrt_offst_ul = + config->grid.y_start >> (config->grid.block_height_log2 % + config->grid.grid_height_per_slice); + + /* + * Values in the LUT cease taking effect at 4096, and a value of 0.0 is + * "no correction" rather than black. The gain factor is described by + * the documentation like so: + * + * "Shift calculated anti shading value. Precision u2. 0x0 - gain factor + * [1, 5], means no shift interpolated value. 0x1 - gain factor [1, 9], + * means shift interpolated by 1. 0x2 - gain factor [1, 17], means shift + * interpolated by 2." + * + * The simplest interpretation for those pieces of information is I + * think that the LUT stores 12-bit Q numbers who's represented values + * depend on the gain_factor setting like so: + * + * 0: UQ<2, 10> representing values in range [0, 4) + * 1: UQ<3, 9> representing values in range [0, 8) + * 2: UQ<4, 8> representing values in range [0, 16) + * + * And that a base gain of 1.0 is added to those configured values. As a + * gain of more than 5.0 is fairly unlikely, let's fix gain_factor to 0 + * for now and revisit if needed. + */ + config->general.gain_factor = 0; + + /* + * Disable the black level settings here - we do that through another + * parameters block. + */ + config->black_level.bl_r = 0; + config->black_level.bl_gr = 0; + config->black_level.bl_gb = 0; + config->black_level.bl_b = 0; + + ipu3_uapi_shd_lut *lut = ¶ms->acc_param.shd.shd_lut; + + const auto &set = lscAlgo_.interpolateComponents(quantizedCt); + + unsigned int totalCells = numHCells_ * numVCells_; + unsigned int cellsPerSet = numHCells_ * config->grid.grid_height_per_slice; + unsigned int numSets = (numHCells_ + config->grid.grid_height_per_slice - 1) / + config->grid.grid_height_per_slice; + unsigned int i = 0; + + for (unsigned int s = 0; s < numSets; s++) { + for (unsigned int c = 0; c < cellsPerSet && i < totalCells; c++, i++) { + lut->sets[s].r_and_gr[c].r = set.at("r")[i]; + lut->sets[s].r_and_gr[c].gr = set.at("gr")[i]; + lut->sets[s].gb_and_b[c].gb = set.at("gb")[i]; + lut->sets[s].gb_and_b[c].b = set.at("b")[i]; + } + } + + lastAppliedCt_ = ct; + lastAppliedQuantizedCt_ = quantizedCt; + LOG(IPU3Lsc, Debug) + << "ct is " << ct << ", quantized to " + << quantizedCt; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Lsc::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const ipu3_uapi_stats_3a *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(Lsc, "Lsc") + +} /* ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/lsc.h b/src/ipa/ipu3/algorithms/lsc.h new file mode 100644 index 0000000000000000000000000000000000000000..050ce91ad115a93224efb161ae0a3536fe32cc7d --- /dev/null +++ b/src/ipa/ipu3/algorithms/lsc.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board, Oy + * + * IPU3 Lens Shading Correction algorithm + */ + +#pragma once + +#include + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::ipu3::algorithms { + +class Lsc : public Algorithm +{ +public: + int init(IPAContext &context, const ValueNode &tuningData) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ipu3_uapi_stats_3a *stats, + ControlList &metadata) override; + +private: + std::vector calculatePositions(unsigned int dimension); + + unsigned int numHCells_; + unsigned int numVCells_; + unsigned int blockWidthLog2_; + unsigned int blockHeightLog2_; + + unsigned int lastAppliedCt_; + unsigned int lastAppliedQuantizedCt_; + + unsigned int sensorWidth_; + unsigned int sensorHeight_; + unsigned int cropWidth_; + unsigned int cropHeight_; + + bool polynomial_; + + LscAlgorithm> lscAlgo_; +}; + +} /* ipa::ipu3::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build index 3dafd2fda9897942cf87d9640665c4fcff383859..70177f50415259e0c2bd207205c385dde07d6624 100644 --- a/src/ipa/ipu3/algorithms/meson.build +++ b/src/ipa/ipu3/algorithms/meson.build @@ -6,5 +6,6 @@ ipu3_ipa_algorithms = files([ 'awb.cpp', 'blc.cpp', 'ccm.cpp', + 'lsc.cpp', 'tone_mapping.cpp', ]) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 9537802ceca5018118bdf4c71ef361c20fc44bfb..935264279a8a6346cb3e58ddb5577811b46163e8 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -127,6 +127,11 @@ namespace libcamera::ipa::ipu3 { * \brief Active gamma correction parameters for the IPA */ +/** + * \var IPAActiveState::lsc + * \brief Active lens shading correction parameters for the IPA + */ + /** * \var IPASessionConfiguration::sensor * \brief Sensor-specific configuration of the IPA @@ -186,4 +191,9 @@ namespace libcamera::ipa::ipu3 { * \brief Per-frame gamma correction parameters for the IPA */ +/** + * \var IPAFrameContext::lsc + * \brief Per-frame lens shading correction parameters for the IPA + */ + } /* namespace libcamera::ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index d650f2fe1ad8eab91b7128a47c9690e42b1595f1..d4bf2091a64fb1aad5c715a2d9d265ae3aecacb4 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -21,6 +21,7 @@ #include #include #include +#include namespace libcamera { @@ -68,6 +69,7 @@ struct IPAActiveState { ipa::awb::ActiveState awb; ipa::ccm::ActiveState ccm; ipa::gamma::ActiveState gamma; + ipa::lsc::ActiveState lsc; }; struct IPAFrameContext : public FrameContext { @@ -79,6 +81,7 @@ struct IPAFrameContext : public FrameContext { ipa::awb::FrameContext awb; ipa::ccm::FrameContext ccm; ipa::gamma::FrameContext gamma; + ipa::lsc::FrameContext lsc; }; struct IPAContext {