{"id":26882,"url":"https://patchwork.libcamera.org/api/1.1/patches/26882/?format=json","web_url":"https://patchwork.libcamera.org/patch/26882/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260615-libipa-algorithms-v1-5-e949c937422e@ideasonboard.com>","date":"2026-06-15T14:05:30","name":"[05/11] ipa: libipa: Add CcmAlgorithm to libipa","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"accb3d6e2c193c6592538b03f928d049f4353487","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/1.1/people/143/?format=json","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26882/mbox/","series":[{"id":5992,"url":"https://patchwork.libcamera.org/api/1.1/series/5992/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5992","date":"2026-06-15T14:05:25","name":"ipa: libipa: Introduce libipa algorithms","version":1,"mbox":"https://patchwork.libcamera.org/series/5992/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26882/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26882/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E4E3EC324C\n\tfor ;\n\tMon, 15 Jun 2026 14:06:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 379D4623F4;\n\tMon, 15 Jun 2026 16:05:56 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 42789623E1\n\tfor ;\n\tMon, 15 Jun 2026 16:05:48 +0200 (CEST)","from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it\n\t[93.65.100.155])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7B94B1E1B;\n\tMon, 15 Jun 2026 16:05:15 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"IYZucAFK\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1781532315;\n\tbh=Ll5r+4v/REyWSytwnhKQKhR6P2gts8C2571acQh/v1s=;\n\th=From:Date:Subject:References:In-Reply-To:To:Cc:From;\n\tb=IYZucAFKDnPBrP+Dp2b2kBXUNy/1HWW4ebY3XHmYOa18sQkinxPbPXIfOBy32jNGj\n\t5jkRbURQb0cIwEqWb7J67sRDSAkNeCa9n+rByoNvYbj5GqI02UYXw9y+jPnuWfzn4Y\n\tYAMCE+duGyzPd+YUZt4sXGmb8H1gzlXJCAX44/fE=","From":"Jacopo Mondi ","Date":"Mon, 15 Jun 2026 16:05:30 +0200","Subject":"[PATCH 05/11] ipa: libipa: Add CcmAlgorithm to libipa","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"7bit","Message-Id":"<20260615-libipa-algorithms-v1-5-e949c937422e@ideasonboard.com>","References":"<20260615-libipa-algorithms-v1-0-e949c937422e@ideasonboard.com>","In-Reply-To":"<20260615-libipa-algorithms-v1-0-e949c937422e@ideasonboard.com>","To":"libcamera-devel@lists.libcamera.org","Cc":"Jacopo Mondi ","X-Mailer":"b4 0.14.3","X-Developer-Signature":"v=1; a=openpgp-sha256; l=12961;\n\ti=jacopo.mondi@ideasonboard.com; h=from:subject:message-id;\n\tbh=Ll5r+4v/REyWSytwnhKQKhR6P2gts8C2571acQh/v1s=;\n\tb=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqMAa5VeUv7idtLyoxzUuIZU6RUOwHU14UvL7zw\n\tXu+Xd45vrOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajAGuQAKCRByNAaPFqFW\n\tPHeGD/9c3/rI2F8j2PKZa9ohpirm6zJuugWz09Y0Z0x4G+UHv1vknl5QN18KLZVtBRnaG+QZFjW\n\tdXpM7XWEnFOHNeFWyQYLLVW6s03ONkTXhEglH2f+dyg6i/3p1lftCME8PQVRoALp6+/hAFXouVr\n\tTq5gn1EUUWIbTG7ghtg4DAFwljvQ9LGPopAeluIakyDmwEoVjjUziyhk1Fu5RjlZ7Lwi9qOmk8n\n\tm8zxBKBSKEGzFMpL5RRANnVkc73IckKYTJhfjwya3KD8Tdrg/PHcBLIe2YAPfHmGLz9QBP6nYrH\n\tS1h2XBg6enCLQKBj/CDrnlLI6wgxib2clUhNGUmeMSy3zh7QlT9YFz9MxWjWOd16AaujJQXSlNh\n\t+kQSbEvQCkbDc9+3K4Ys8jHHuE6ketAWlkWZoRFxXjrOtNemD8pQgHcuuyNJ6NG+k1FpHBoVeAN\n\t9N68sPmkqRSjgCVtUTXg4UwJMTlvOMRV7KIs5LMtfAbgAlyxG0XRWVauKEmNXDG+A02NW1JNhVc\n\tHenGAQeFx7VuO7H2r7x7Sm5vlSaVpHH8XAca3L3Vy6qios0ytqK5Yp1hZUl8M+dyMtx2IxckZzo\n\tF/rlBX2I1ndq9L6TIBb37lApfE4uncP8C+VTnr4aTVtsbHBDgwukZfb++OhuRyCN2f8RHX5kAB8\n\t0yKGrAcFT/j9mWQ==","X-Developer-Key":"i=jacopo.mondi@ideasonboard.com; a=openpgp;\n\tfpr=72392EDC88144A65C701EA9BA5826A2587AD026B","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add a CcmAlgorithm to libipa.\n\nThe CcmAlgorithm performs interpolation of the colour correction\nmatrices as loaded from tuning file on a colour temperature.\n\nThe implementation is based on the existing RkISP1 CCM algorihtm.\n\nSigned-off-by: Jacopo Mondi \n---\n src/ipa/libipa/ccm.cpp | 273 +++++++++++++++++++++++++++++++++++++++++++++\n src/ipa/libipa/ccm.h | 87 +++++++++++++++\n src/ipa/libipa/meson.build | 2 +\n 3 files changed, 362 insertions(+)","diff":"diff --git a/src/ipa/libipa/ccm.cpp b/src/ipa/libipa/ccm.cpp\nnew file mode 100644\nindex 000000000000..278f8534e899\n--- /dev/null\n+++ b/src/ipa/libipa/ccm.cpp\n@@ -0,0 +1,273 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2026 Ideas on Board Oy\n+ *\n+ * libIPA CCM algorithm\n+ */\n+\n+#include \"ccm.h\"\n+\n+/**\n+ * \\file ccm.h\n+ * \\brief libipa ccm (Colour Correction Matrix) algorithm\n+ */\n+\n+namespace libcamera {\n+\n+namespace ipa {\n+\n+LOG_DEFINE_CATEGORY(Ccm)\n+\n+namespace ccm {\n+\n+/**\n+ * \\struct ActiveState\n+ * \\brief Active ccm state\n+ *\n+ * \\var ActiveState::manual\n+ * \\brief The most recent manually requested ccm state\n+ *\n+ * \\var ActiveState::automatic\n+ * \\brief The most recent automatically calculated ccm state\n+ */\n+\n+/**\n+ * \\struct ActiveState::CcmState\n+ * \\brief Ccm coefficients and offsets\n+ *\n+ * \\var ActiveState::CcmState::ccm\n+ * \\brief Matrix of 3x3 ccm coefficients\n+ *\n+ * \\var ActiveState::CcmState::offsets\n+ * \\brief Vector of RGB ccm offsets\n+ */\n+\n+/**\n+ * \\struct FrameContext\n+ * \\brief Per-frame ccm state\n+ *\n+ * \\var FrameContext::ccm\n+ * \\brief Matrix of 3x3 ccm coefficients\n+ *\n+ * \\var FrameContext::offsets\n+ * \\brief Vector of RGB ccm offsets\n+ */\n+\n+} /* namespace ccm */\n+\n+/**\n+ * \\class CcmAlgorithmBase\n+ * \\brief Base class for CcmAlgorithm for non-templated functions implementation\n+ *\n+ * Base class for CcmAlgorithm where non-templated functions are implemented.\n+ * IPA implementations shall use CcmAlgorithm and not this class.\n+ */\n+\n+/**\n+ * \\brief Initialize the algorithm with the given tuning data\n+ * \\param[in] tuningData The tuning data to use for the algorithm\n+ *\n+ * Parse \\a tuningData to initialize the ccm algorithm and register controls.\n+ * IPA modules are expected to call this function as part of their\n+ * implementation of Algorithm::init().\n+ *\n+ * \\return 0 on success, a negative error code otherwise\n+ */\n+int CcmAlgorithmBase::init(const ValueNode &tuningData)\n+{\n+\tint ret = ccm_.readYaml(tuningData[\"ccms\"], \"ct\", \"ccm\");\n+\tif (ret < 0) {\n+\t\tLOG(Ccm, Warning)\n+\t\t\t<< \"Failed to parse 'ccm' \"\n+\t\t\t<< \"parameter from tuning file; falling back to unit matrix\";\n+\t\tccm_.setData({ { 0, Matrix::identity() } });\n+\t}\n+\n+\tret = offsets_.readYaml(tuningData[\"ccms\"], \"ct\", \"offsets\");\n+\tif (ret < 0) {\n+\t\tLOG(Ccm, Warning)\n+\t\t\t<< \"Failed to parse 'offsets' \"\n+\t\t\t<< \"parameter from tuning file; falling back to zero offsets\";\n+\n+\t\toffsets_.setData({ { 0, Matrix({ 0, 0, 0 }) } });\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/**\n+ * \\brief Configure the ccm algorithm\n+ * \\param[in] state The ccm active state\n+ * \\param[in] temperatureK The colour temperature in Kelvin\n+ *\n+ * Configure the ccm algorithm by initializing the manual and automatic\n+ * states in \\a state by interpolating the default colour correction matrix\n+ * with the given colour temperature \\a temperatureK.\n+ *\n+ * \\return 0 if successful, an error code otherwise\n+ */\n+int CcmAlgorithmBase::configure(ccm::ActiveState &state, unsigned int temperatureK)\n+{\n+\tstate.manual.ccm = ccm_.getInterpolated(temperatureK);\n+\tstate.manual.offsets = offsets_.getInterpolated(temperatureK);\n+\tstate.automatic.ccm = ccm_.getInterpolated(temperatureK);\n+\tstate.automatic.offsets = offsets_.getInterpolated(temperatureK);\n+\n+\treturn 0;\n+}\n+\n+/**\n+ * \\brief Queue a Request to the ccm algorithm\n+ * \\param[in] state The ccm active state\n+ * \\param[in] context The ccm frame context\n+ * \\param[in] controls The list of controls part of the Request\n+ *\n+ * Queue a new Request to the ccm algorithm and store the manual colour\n+ * correction matrix and temperature in \\a frameContext.\n+ *\n+ * The currently handled controls are:\n+ * - controls::ColourTemperature\n+ * - controls::ColourCorrectionMatrix\n+ *\n+ * When controls::ColourCorrectionMatrix is passed in the supplied matrix is\n+ * stored in \\a state and \\a context.\n+ *\n+ * When controls::ColourTemperature is passed in, the matrices loaded from\n+ * configuration file are interpolated with the give temperature and the result\n+ * is stored in \\a state and \\a context.\n+ *\n+ * If the IPA is running in manual mode, the IPA ccm algorithm implementations\n+ * can use the matrix coefficients and offsets directly from \\a context after\n+ * calling this function to program the HW ccm engine, without calling prepare().\n+ */\n+void CcmAlgorithmBase::queueRequest(ccm::ActiveState &state,\n+\t\t\t\t ccm::FrameContext &context,\n+\t\t\t\t const ControlList &controls)\n+{\n+\tconst auto &colourTemperature = controls.get(controls::ColourTemperature);\n+\tconst auto &ccmMatrix = controls.get(controls::ColourCorrectionMatrix);\n+\tif (ccmMatrix) {\n+\t\tstate.manual.ccm = Matrix(*ccmMatrix);\n+\t\tLOG(Ccm, Debug) << \"Setting manual CCM from CCM control to \"\n+\t\t\t\t<< state.manual.ccm;\n+\t} else if (colourTemperature) {\n+\t\tstate.manual.ccm = ccm_.getInterpolated(*colourTemperature);\n+\t\tLOG(Ccm, Debug) << \"Setting manual CCM from CT control to \"\n+\t\t\t\t<< state.manual.ccm;\n+\t}\n+\n+\tcontext.ccm = state.manual.ccm;\n+\tcontext.offsets = state.manual.offsets;\n+}\n+\n+/**\n+ * \\brief Calculate the matrix coefficients for a colour temperature\n+ * \\param[in] state The ccm active state\n+ * \\param[in] context The ccm frame context\n+ * \\param[in] frame The frame number\n+ * \\param[in] temperatureK The colour temperature in Kelvin\n+ *\n+ * Interpolate the colour correction matrices as loaded from configuration file\n+ * for colour temperature \\a temperatureK.\n+ *\n+ * The function shall only be called if the IPA algorithm is running in auto\n+ * mode. If running in manual mode the application supplied correction matrix is\n+ * stored in \\a frameContext at queueRequest() time.\n+ */\n+void CcmAlgorithmBase::prepare(ccm::ActiveState &state,\n+\t\t\t ccm::FrameContext &context,\n+\t\t\t unsigned int frame, unsigned int temperatureK)\n+{\n+\tif (frame > 0 && temperatureK == ct_) {\n+\t\tcontext.ccm = state.automatic.ccm;\n+\t\treturn;\n+\t}\n+\n+\tct_ = temperatureK;\n+\tcontext.ccm = ccm_.getInterpolated(ct_);\n+\tcontext.offsets = offsets_.getInterpolated(ct_);\n+\n+\tstate.automatic.ccm = context.ccm;\n+\tstate.automatic.offsets = context.offsets;\n+}\n+\n+/**\n+ * \\brief Populate metadata with the latest correction matrix coefficients\n+ * \\param[in] context The ccm frame context\n+ * \\param[out] metadata The metadata list\n+ */\n+void CcmAlgorithmBase::process(ccm::FrameContext &context, ControlList &metadata)\n+{\n+\tmetadata.set(controls::ColourCorrectionMatrix, context.ccm.data());\n+}\n+\n+/**\n+ * \\var CcmAlgorithmBase::coeffMin_\n+ * \\brief The minimum supported coefficients value\n+ *\n+ * Minimum coefficient value used to clamp the ccm algorithm calculation results\n+ * in the range supported by the platform ccm engine.\n+ *\n+ * The min and max gain values are initialized by CcmAlgorithm::init().\n+ */\n+\n+/**\n+ * \\var CcmAlgorithmBase::coeffMax_\n+ * \\brief The maximum supported coefficients value\n+ *\n+ * Maximum coefficient value used to clamp the ccm algorithm calculation results\n+ * in the range supported by the platform ccm engine.\n+ *\n+ * The min and max gain values are initialized by CcmAlgorithm::init().\n+ */\n+\n+/**\n+ * \\class CcmAlgorithm\n+ * \\brief The libipa ccm algorithm\n+ * \\tparam Q The fixedpoint register representation of the colour correction\n+ * coefficients\n+ *\n+ * Implement the ccm algorithm for libipa.\n+ *\n+ * The CcmAlgorithm class implement an interface similar in spirit to the one\n+ * of the Algorithm class. IPA modules are expected to store an instance of\n+ * CcmAlgorithm as class member, template it with the ccm coefficients register\n+ * representation and call its function in their implementations of the\n+ * Algorithm interface.\n+ *\n+ * The CcmAlgorithm class provides an init() function where tuning data are\n+ * parsed and the per-colour temperature correction matrices are loaded from\n+ * the tuning file.\n+ *\n+ * CcmAlgorithm supports both automatic and manual colour correction operations,\n+ * but doesn't offer a way to select one of them. Enabling or disabling\n+ * automatic ccm operations usually goes through the Awb algorithm\n+ * enable/disable as the two algorithms should work with the same mode.\n+ *\n+ * When the Awb algorithm runs in manual mode a custom colour correction matrix\n+ * or a custom colour temperature can be supplied to the ccm algorithm at\n+ * queueRequest() time. If the Request contains a color correction matrix\n+ * (controls::ColourCorrectionMatrix) then the matrix coefficients gets saved in\n+ * the FrameContext and the IPA module can immediately use them and doesn't need\n+ * to call process(). If a custom colour temperature is provided\n+ * (controls::ColourTemperature) then the matrices loaded from configuration are\n+ * interpolated with it and the result is saved in the FrameContext. In this\n+ * case as well IPA modules can use the result immediately and should avoid\n+ * calling process().\n+ *\n+ * When the Awb algorithm runs in automatic mode instead, it estimates the scene\n+ * colour temperature. The estimated colour temperature shall be passed to\n+ * process(), where it is used to interpolate the matrices loaded from the\n+ * tuning file. The resulting coefficients are stored in the FrameContext for\n+ * the IPA algorithm to use them to program their ccm engine registers.\n+ */\n+\n+/**\n+ * \\fn CcmAlgorithm::init()\n+ * \\param[in] controls The info map of the IPA controls\n+ * \\copydoc CcmAlgorithmBase::init()\n+ */\n+\n+} /* namespace ipa */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/libipa/ccm.h b/src/ipa/libipa/ccm.h\nnew file mode 100644\nindex 000000000000..26ea0789481d\n--- /dev/null\n+++ b/src/ipa/libipa/ccm.h\n@@ -0,0 +1,87 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2026 Ideas on Board Oy\n+ *\n+ * libIPA CCM algorithm\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/matrix.h\"\n+\n+#include \"fixedpoint.h\"\n+#include \"interpolator.h\"\n+\n+namespace libcamera {\n+\n+namespace ipa {\n+\n+namespace ccm {\n+\n+struct ActiveState {\n+\tstruct CcmState {\n+\t\tMatrix ccm;\n+\t\tMatrix offsets;\n+\t};\n+\n+\tstruct CcmState manual;\n+\tstruct CcmState automatic;\n+};\n+\n+struct FrameContext {\n+\tMatrix ccm;\n+\tMatrix offsets;\n+};\n+\n+} /* namespace ccm */\n+\n+class CcmAlgorithmBase\n+{\n+public:\n+\tint init(const ValueNode &tuningData);\n+\tint configure(ccm::ActiveState &state, unsigned int temperatureK);\n+\tvoid queueRequest(ccm::ActiveState &state, ccm::FrameContext &context,\n+\t\t\t const ControlList &controls);\n+\n+\tvoid prepare(ccm::ActiveState &state, ccm::FrameContext &context,\n+\t\t unsigned int frame, unsigned int temperatureK);\n+\tvoid process(ccm::FrameContext &context, ControlList &metadata);\n+\n+protected:\n+\tfloat coeffMin_;\n+\tfloat coeffMax_;\n+\n+private:\n+\tunsigned int ct_;\n+\tInterpolator> ccm_;\n+\tInterpolator> offsets_;\n+};\n+\n+template\n+class CcmAlgorithm : public CcmAlgorithmBase\n+{\n+public:\n+\tint init(const ValueNode &tuningData, ControlInfoMap::Map &controls)\n+\t{\n+\t\tint ret = CcmAlgorithmBase::init(tuningData);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\n+\t\tcoeffMin_ = Q::TraitsType::min;\n+\t\tcoeffMax_ = Q::TraitsType::max;\n+\n+\t\tcontrols[&controls::ColourCorrectionMatrix] =\n+\t\t\tControlInfo(ControlValue(coeffMin_),\n+\t\t\t\t ControlValue(coeffMax_),\n+\t\t\t\t ControlValue(Matrix::identity().data()));\n+\n+\t\treturn 0;\n+\t}\n+};\n+\n+} /* namespace ipa */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build\nindex 963c5ee73063..edf8eabd8b78 100644\n--- a/src/ipa/libipa/meson.build\n+++ b/src/ipa/libipa/meson.build\n@@ -7,6 +7,7 @@ libipa_headers = files([\n 'awb_grey.h',\n 'awb.h',\n 'camera_sensor_helper.h',\n+ 'ccm.h',\n 'colours.h',\n 'exposure_mode_helper.h',\n 'fc_queue.h',\n@@ -28,6 +29,7 @@ libipa_sources = files([\n 'awb_grey.cpp',\n 'awb.cpp',\n 'camera_sensor_helper.cpp',\n+ 'ccm.cpp',\n 'colours.cpp',\n 'exposure_mode_helper.cpp',\n 'fc_queue.cpp',\n","prefixes":["05/11"]}