{"id":26493,"url":"https://patchwork.libcamera.org/api/1.1/patches/26493/?format=json","web_url":"https://patchwork.libcamera.org/patch/26493/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260407-kbingham-awb-split-v1-10-a39af3f4dc20@ideasonboard.com>","date":"2026-04-07T22:01:13","name":"[10/13] ipa: simple: Convert awb to libipa implementation","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"fefddd256709ad080c86886337ae10c82b313ec0","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/1.1/people/4/?format=json","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26493/mbox/","series":[{"id":5874,"url":"https://patchwork.libcamera.org/api/1.1/series/5874/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5874","date":"2026-04-07T22:01:03","name":"ipa: simple: Convert to libipa AWB implementation","version":1,"mbox":"https://patchwork.libcamera.org/series/5874/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26493/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26493/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 62AEFC32F8\n\tfor ;\n\tTue, 7 Apr 2026 22:03:01 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1918862D66;\n\tWed, 8 Apr 2026 00:02:55 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 33B1662D4E\n\tfor ;\n\tWed, 8 Apr 2026 00:02:43 +0200 (CEST)","from [192.168.0.204] (ams.linuxembedded.co.uk [209.38.108.23])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7FFD2257A;\n\tWed, 8 Apr 2026 00:01:15 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"HGg39Y8F\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1775599275;\n\tbh=rbs3mKEUMOc5h/sZ+nnL7K22ky+pgA3A36uWXG67KIM=;\n\th=From:Date:Subject:References:In-Reply-To:To:Cc:From;\n\tb=HGg39Y8FsAgy5bPKeao//ELLJx+E0zAa69LeKddrD/PbW94kaxVqQNy10djOcvRUw\n\t2JhnzfLJmAimZuJ4jv6vZsj/lPo1UDhFmNBH+ewUC+Iqq6AU2D26t17NzDXuKHGPIU\n\tg3AB/T0NsPdbh2Gio6lay6tW8UM+lHbTjSUbFkZo=","From":"Kieran Bingham ","Date":"Tue, 07 Apr 2026 23:01:13 +0100","Subject":"[PATCH 10/13] ipa: simple: Convert awb to libipa implementation","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"7bit","Message-Id":"<20260407-kbingham-awb-split-v1-10-a39af3f4dc20@ideasonboard.com>","References":"<20260407-kbingham-awb-split-v1-0-a39af3f4dc20@ideasonboard.com>","In-Reply-To":"<20260407-kbingham-awb-split-v1-0-a39af3f4dc20@ideasonboard.com>","To":"libcamera-devel@lists.libcamera.org","Cc":"Kieran Bingham ","X-Mailer":"b4 0.14.2","X-Developer-Signature":"v=1; a=ed25519-sha256; t=1775599360; l=12978;\n\ti=kieran.bingham@ideasonboard.com; s=20260204;\n\th=from:subject:message-id; \n\tbh=rbs3mKEUMOc5h/sZ+nnL7K22ky+pgA3A36uWXG67KIM=;\n\tb=6LbPOZq/ynlgBb2NlD2UGY0/hMIrkIZFVq+4/SokpqNOwtN4Y+pZsfyyAWWiCrRgyCCtf4JK2\n\t8AiDn5de/LUBw9J5H9P9BVhh+t4l6GztckKL26OaRabHKDPYPqX2kxJ","X-Developer-Key":"i=kieran.bingham@ideasonboard.com; a=ed25519;\n\tpk=IOxS2C6nWHNjLfkDR71Iesk904i6wJDfEERqV7hDBdY=","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This brings in both Greyworld and Bayes from libipa assuming the tuning\nfile provides the Bayesian priors from calibration.\n\nManual controls become available and enabled as well, and the storage in\nthe Context is moved to the new common types.\n\nSigned-off-by: Kieran Bingham \n---\n src/ipa/simple/algorithms/awb.cpp | 215 +++++++++++++++++++++++++++++++++-----\n src/ipa/simple/algorithms/awb.h | 12 +++\n src/ipa/simple/algorithms/ccm.cpp | 2 +-\n src/ipa/simple/ipa_context.h | 12 +--\n 4 files changed, 209 insertions(+), 32 deletions(-)","diff":"diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp\nindex 05155c83d172d64609053ba940a4c12a2248bb04..90c05e86bae6eefe4874feeb1263af07acd5fcfc 100644\n--- a/src/ipa/simple/algorithms/awb.cpp\n+++ b/src/ipa/simple/algorithms/awb.cpp\n@@ -14,6 +14,8 @@\n \n #include \n \n+#include \"libipa/awb_bayes.h\"\n+#include \"libipa/awb_grey.h\"\n #include \"libipa/colours.h\"\n #include \"simple/ipa_context.h\"\n \n@@ -23,24 +25,173 @@ LOG_DEFINE_CATEGORY(IPASoftAwb)\n \n namespace ipa::soft::algorithms {\n \n+constexpr int32_t kMinColourTemperature = 2500;\n+constexpr int32_t kMaxColourTemperature = 10000;\n+constexpr int32_t kDefaultColourTemperature = 5000;\n+\n+/* Identical to RKISP1AwbStats ... why ? */\n+class SimpleAwbStats final : public AwbStats\n+{\n+public:\n+\tSimpleAwbStats(const RGB &rgbMeans)\n+\t\t: rgbMeans_(rgbMeans)\n+\t{\n+\t\trg_ = rgbMeans_.r() / rgbMeans_.g();\n+\t\tbg_ = rgbMeans_.b() / rgbMeans_.g();\n+\t}\n+\n+\tdouble computeColourError(const RGB &gains) const override\n+\t{\n+\t\t/*\n+\t\t * Compute the sum of the squared colour error (non-greyness) as\n+\t\t * it appears in the log likelihood equation.\n+\t\t */\n+\t\tdouble deltaR = gains.r() * rg_ - 1.0;\n+\t\tdouble deltaB = gains.b() * bg_ - 1.0;\n+\t\tdouble delta2 = deltaR * deltaR + deltaB * deltaB;\n+\n+\t\treturn delta2;\n+\t}\n+\n+\tRGB rgbMeans() const override\n+\t{\n+\t\treturn rgbMeans_;\n+\t}\n+\n+private:\n+\tRGB rgbMeans_;\n+\tdouble rg_;\n+\tdouble bg_;\n+};\n+\n+int Awb::init(IPAContext &context, const YamlObject &tuningData)\n+{\n+\tauto &cmap = context.ctrlMap;\n+\n+\tcmap[&controls::ColourTemperature] = ControlInfo(kMinColourTemperature,\n+\t\t\t\t\t\t\t kMaxColourTemperature,\n+\t\t\t\t\t\t\t kDefaultColourTemperature);\n+\n+\tcmap[&controls::AwbEnable] = ControlInfo(false, true);\n+\tcmap[&controls::ColourGains] = ControlInfo(0.0f, 3.996f,\n+\t\t\t\t\t\t Span{ { 1.0f, 1.0f } });\n+\n+\tif (!tuningData.contains(\"algorithm\"))\n+\t\tLOG(IPASoftAwb, Info) << \"No AWB algorithm specified.\"\n+\t\t\t\t << \" Default to grey world\";\n+\n+\tauto mode = tuningData[\"algorithm\"].get(\"grey\");\n+\tif (mode == \"grey\") {\n+\t\tawbAlgo_ = std::make_unique();\n+\t} else if (mode == \"bayes\") {\n+\t\tawbAlgo_ = std::make_unique();\n+\t} else {\n+\t\tLOG(IPASoftAwb, Error) << \"Unknown AWB algorithm: \" << mode;\n+\t\treturn -EINVAL;\n+\t}\n+\tLOG(IPASoftAwb, Debug) << \"Using AWB algorithm: \" << mode;\n+\n+\tint ret = awbAlgo_->init(tuningData);\n+\tif (ret) {\n+\t\tLOG(IPASoftAwb, Error) << \"Failed to init AWB algorithm\";\n+\t\treturn ret;\n+\t}\n+\n+\tconst auto &src = awbAlgo_->controls();\n+\tcmap.insert(src.begin(), src.end());\n+\n+\treturn 0;\n+}\n+\n int Awb::configure(IPAContext &context,\n \t\t [[maybe_unused]] const IPAConfigInfo &configInfo)\n {\n-\tauto &gains = context.activeState.awb.gains;\n-\tgains = { { 1.0, 1.0, 1.0 } };\n+\tcontext.activeState.awb.manual.gains = RGB{ 1.0 };\n+\tauto gains = awbAlgo_->gainsFromColourTemperature(kDefaultColourTemperature);\n+\tif (gains)\n+\t\tcontext.activeState.awb.automatic.gains = *gains;\n+\telse\n+\t\tcontext.activeState.awb.automatic.gains = RGB{ 1.0 };\n+\n+\tcontext.activeState.awb.autoEnabled = true;\n+\tcontext.activeState.awb.manual.temperatureK = kDefaultColourTemperature;\n+\tcontext.activeState.awb.automatic.temperatureK = kDefaultColourTemperature;\n+\n+\tcontext.configuration.awb.enabled = true;\n \n \treturn 0;\n }\n \n+/**\n+ * \\copydoc libcamera::ipa::Algorithm::queueRequest\n+ */\n+void Awb::queueRequest(IPAContext &context,\n+\t\t [[maybe_unused]] const uint32_t frame,\n+\t\t [[maybe_unused]] IPAFrameContext &frameContext,\n+\t\t const ControlList &controls)\n+{\n+\tauto &awb = context.activeState.awb;\n+\n+\tconst auto &awbEnable = controls.get(controls::AwbEnable);\n+\tif (awbEnable && *awbEnable != awb.autoEnabled) {\n+\t\tawb.autoEnabled = *awbEnable;\n+\n+\t\tLOG(IPASoftAwb, Debug)\n+\t\t\t<< (*awbEnable ? \"Enabling\" : \"Disabling\") << \" AWB\";\n+\t}\n+\n+\tawbAlgo_->handleControls(controls);\n+\n+\tframeContext.awb.autoEnabled = awb.autoEnabled;\n+\n+\tif (awb.autoEnabled)\n+\t\treturn;\n+\n+\tconst auto &colourGains = controls.get(controls::ColourGains);\n+\tconst auto &colourTemperature = controls.get(controls::ColourTemperature);\n+\tbool update = false;\n+\tif (colourGains) {\n+\t\tawb.manual.gains.r() = (*colourGains)[0];\n+\t\tawb.manual.gains.b() = (*colourGains)[1];\n+\t\t/*\n+\t\t * \\todo Colour temperature reported in metadata is now\n+\t\t * incorrect, as we can't deduce the temperature from the gains.\n+\t\t * This will be fixed with the bayes AWB algorithm.\n+\t\t */\n+\t\tupdate = true;\n+\t} else if (colourTemperature) {\n+\t\tawb.manual.temperatureK = *colourTemperature;\n+\t\tconst auto &gains = awbAlgo_->gainsFromColourTemperature(*colourTemperature);\n+\t\tif (gains) {\n+\t\t\tawb.manual.gains.r() = gains->r();\n+\t\t\tawb.manual.gains.b() = gains->b();\n+\t\t\tupdate = true;\n+\t\t}\n+\t}\n+\n+\tif (update)\n+\t\tLOG(IPASoftAwb, Debug)\n+\t\t\t<< \"Set colour gains to \" << awb.manual.gains;\n+\n+\tframeContext.awb.gains = awb.manual.gains;\n+\tframeContext.awb.temperatureK = awb.manual.temperatureK;\n+}\n+\n void Awb::prepare(IPAContext &context,\n \t\t [[maybe_unused]] const uint32_t frame,\n \t\t IPAFrameContext &frameContext,\n \t\t DebayerParams *params)\n {\n-\tauto &gains = context.activeState.awb.gains;\n+\t/*\n+\t * When AutoAWB is enabled, this is the latest opportunity to take\n+\t * the most recent and up to date desired AWB gains.\n+\t */\n+\tif (frameContext.awb.autoEnabled) {\n+\t\tframeContext.awb.gains = context.activeState.awb.automatic.gains;\n+\t\tframeContext.awb.temperatureK = context.activeState.awb.automatic.temperatureK;\n+\t}\n \n-\tframeContext.gains = gains;\n-\tparams->gains = gains;\n+\tparams->gains = frameContext.awb.gains;\n }\n \n void Awb::process(IPAContext &context,\n@@ -49,15 +200,19 @@ void Awb::process(IPAContext &context,\n \t\t const SwIspStats *stats,\n \t\t ControlList &metadata)\n {\n-\tconst SwIspStats::Histogram &histogram = stats->yHistogram;\n-\tconst uint8_t blackLevel = context.activeState.blc.level;\n+\tIPAActiveState &activeState = context.activeState;\n+\tRGB gains = frameContext.awb.gains;\n \n-\tmetadata.set(controls::ColourGains, { frameContext.gains.r(),\n-\t\t\t\t\t frameContext.gains.b() });\n+\tmetadata.set(controls::AwbEnable, frameContext.awb.autoEnabled);\n+\tmetadata.set(controls::ColourGains, { gains.r(), gains.b() });\n+\tmetadata.set(controls::ColourTemperature, frameContext.awb.temperatureK);\n \n \tif (!stats->valid)\n \t\treturn;\n \n+\tconst SwIspStats::Histogram &histogram = stats->yHistogram;\n+\tconst uint8_t blackLevel = context.activeState.blc.level;\n+\n \t/*\n \t * Black level must be subtracted to get the correct AWB ratios, they\n \t * would be off if they were computed from the whole brightness range\n@@ -67,30 +222,42 @@ void Awb::process(IPAContext &context,\n \t\thistogram.begin(), histogram.end(), uint64_t(0));\n \tconst uint64_t offset = blackLevel * nPixels;\n \tconst uint64_t minValid = 1;\n+\n \t/*\n \t * Make sure the sums are at least minValid, while preventing unsigned\n \t * integer underflow.\n \t */\n \tconst RGB sum = stats->sum_.max(offset + minValid) - offset;\n \n-\t/*\n-\t * Calculate red and blue gains for AWB.\n-\t * Clamp max gain at 4.0, this also avoids 0 division.\n-\t */\n-\tauto &gains = context.activeState.awb.gains;\n-\tgains = { {\n-\t\tsum.r() <= sum.g() / 4 ? 4.0f : static_cast(sum.g()) / sum.r(),\n-\t\t1.0,\n-\t\tsum.b() <= sum.g() / 4 ? 4.0f : static_cast(sum.g()) / sum.b(),\n-\t} };\n+\tRGB rgbMeans = { { static_cast(sum.r() / nPixels),\n+\t\t\t\t static_cast(sum.g() / nPixels),\n+\t\t\t\t static_cast(sum.b() / nPixels) } };\n \n-\tRGB rgbGains{ { 1 / gains.r(), 1 / gains.g(), 1 / gains.b() } };\n-\tcontext.activeState.awb.temperatureK = estimateCCT(rgbGains);\n-\tmetadata.set(controls::ColourTemperature, context.activeState.awb.temperatureK);\n+\t/*\n+\t * Todo: Determine the minimum allowed thresholds from the mean\n+\t * but we currently have the sum - not the mean value!\n+\t */\n+\tSimpleAwbStats awbStats{ rgbMeans };\n+\n+\tAwbResult awbResult = awbAlgo_->calculateAwb(awbStats, frameContext.lux.lux);\n+\n+\t/* Todo: Check if clamping required */\n+\n+\t/* Filter the values to avoid oscillations. */\n+\tdouble speed = 0.2;\n+\tdouble ct = awbResult.colourTemperature;\n+\tct = ct * speed + activeState.awb.automatic.temperatureK * (1 - speed);\n+\tawbResult.gains = awbResult.gains * speed +\n+\t\t\t activeState.awb.automatic.gains * (1 - speed);\n+\n+\tactiveState.awb.automatic.temperatureK = static_cast(ct);\n+\tactiveState.awb.automatic.gains = awbResult.gains;\n \n \tLOG(IPASoftAwb, Debug)\n-\t\t<< \"gain R/B: \" << gains << \"; temperature: \"\n-\t\t<< context.activeState.awb.temperatureK;\n+\t\t<< std::showpoint\n+\t\t<< \"Means \" << rgbMeans << \", gains \"\n+\t\t<< activeState.awb.automatic.gains << \", temp \"\n+\t\t<< activeState.awb.automatic.temperatureK << \"K\";\n }\n \n REGISTER_IPA_ALGORITHM(Awb, \"Awb\")\ndiff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h\nindex ad993f39c18002547301b0588dfde143382854a9..fa8f38f65d6e9fdd18418361711e683916b9a9ba 100644\n--- a/src/ipa/simple/algorithms/awb.h\n+++ b/src/ipa/simple/algorithms/awb.h\n@@ -7,6 +7,8 @@\n \n #pragma once\n \n+#include \"libipa/awb.h\"\n+\n #include \"algorithm.h\"\n \n namespace libcamera {\n@@ -19,7 +21,14 @@ public:\n \tAwb() = default;\n \t~Awb() = default;\n \n+\tint init(IPAContext &context,\n+\t\t const YamlObject &tuningData) override;\n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n+\n+\tvoid queueRequest(IPAContext &context,\n+\t\t\t [[maybe_unused]] const uint32_t frame,\n+\t\t\t IPAFrameContext &frameContext,\n+\t\t\t const ControlList &controls) override;\n \tvoid prepare(IPAContext &context,\n \t\t const uint32_t frame,\n \t\t IPAFrameContext &frameContext,\n@@ -29,6 +38,9 @@ public:\n \t\t IPAFrameContext &frameContext,\n \t\t const SwIspStats *stats,\n \t\t ControlList &metadata) override;\n+\n+private:\n+\tstd::unique_ptr awbAlgo_;\n };\n \n } /* namespace ipa::soft::algorithms */\ndiff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp\nindex 911a5af2c90b55187f0d98519f3c29b8e0804567..6dace73202a800b2c6375c30083e1ed50ef425b1 100644\n--- a/src/ipa/simple/algorithms/ccm.cpp\n+++ b/src/ipa/simple/algorithms/ccm.cpp\n@@ -44,7 +44,7 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData\n void Ccm::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \t\t IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params)\n {\n-\tconst unsigned int ct = context.activeState.awb.temperatureK;\n+\tconst unsigned int ct = frameContext.awb.temperatureK;\n \n \t/* Change CCM only on bigger temperature changes. */\n \tif (!currentCcm_ ||\ndiff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h\nindex 2bd7c4642b118d7bb94b1b16cdf4ede5fb2554b5..67b03b5b835f59cf2e339d21377e06d1bbe79b6f 100644\n--- a/src/ipa/simple/ipa_context.h\n+++ b/src/ipa/simple/ipa_context.h\n@@ -16,6 +16,7 @@\n #include \"libcamera/internal/matrix.h\"\n #include \"libcamera/internal/vector.h\"\n \n+#include \n #include \n #include \n \n@@ -26,6 +27,8 @@ namespace libcamera {\n namespace ipa::soft {\n \n struct IPASessionConfiguration {\n+\tipa::awb::Session awb;\n+\n \tstruct {\n \t\tint32_t exposureMin, exposureMax;\n \t\tdouble againMin, againMax, again10, againMinStep;\n@@ -38,6 +41,7 @@ struct IPASessionConfiguration {\n \n struct IPAActiveState {\n \tipa::lux::ActiveState lux;\n+\tipa::awb::ActiveState awb;\n \n \tstruct {\n \t\tint32_t exposure;\n@@ -51,11 +55,6 @@ struct IPAActiveState {\n \t\tdouble lastGain;\n \t} blc;\n \n-\tstruct {\n-\t\tRGB gains;\n-\t\tunsigned int temperatureK;\n-\t} awb;\n-\n \tMatrix combinedMatrix;\n \n \tstruct {\n@@ -68,6 +67,7 @@ struct IPAActiveState {\n \n struct IPAFrameContext : public FrameContext {\n \tipa::lux::FrameContext lux;\n+\tipa::awb::FrameContext awb;\n \n \tMatrix ccm;\n \n@@ -76,8 +76,6 @@ struct IPAFrameContext : public FrameContext {\n \t\tdouble gain;\n \t} sensor;\n \n-\tRGB gains;\n-\n \tfloat gamma;\n \tstd::optional contrast;\n \tstd::optional saturation;\n","prefixes":["10/13"]}