From patchwork Mon Jun 15 14:05:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26885 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ABD92C32D4 for ; Mon, 15 Jun 2026 14:06:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5C133623F1; Mon, 15 Jun 2026 16:06:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uRIXCLuD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 187EF623EB for ; Mon, 15 Jun 2026 16:05:49 +0200 (CEST) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4E93B1E48; Mon, 15 Jun 2026 16:05:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781532316; bh=6bqc2/XcD2qmj+9SAp2g4fk3X/yoP9cnFEchCTiEYUo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=uRIXCLuD5TeV1+Ht7PoY/b6NpW9xAOTbUpoCIl7R2nzDbt3rRrxs6dLoHggT5ai5f C/MifBgHBn6zBQ4aQxCG3INlgBXEHNPsl0fOSYwBLeMZss8D2Ai8RoAu98nXwtl9SJ rYEKziLSlnRTvp7MbRqdR+Z6hiE2RyaRq7zpQcjs= From: Jacopo Mondi Date: Mon, 15 Jun 2026 16:05:33 +0200 Subject: [PATCH 08/11] ipa: mali-c55: Implement Ccm algorithm MIME-Version: 1.0 Message-Id: <20260615-libipa-algorithms-v1-8-e949c937422e@ideasonboard.com> References: <20260615-libipa-algorithms-v1-0-e949c937422e@ideasonboard.com> In-Reply-To: <20260615-libipa-algorithms-v1-0-e949c937422e@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9110; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=6bqc2/XcD2qmj+9SAp2g4fk3X/yoP9cnFEchCTiEYUo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqMAa51zvayb9KaJ77YW3k6C3D6StTZBnxiWg7l 17aHHcep7yJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajAGuQAKCRByNAaPFqFW PELeD/9k3vWVtL6j3UBKlORsjdZWL9hU+WqKFEHuFASyrjx6dvW0Du8V5EjUWPF+oNMhcKxxUbL FRIYvMRkQUXxjKNjTDKUXg6Vd1XHJMLy3l4R6dwtarcN5wFn1/6uWrrZehJeFTQr8Fh6FLd3lhR oKgBOo4seibw4itU6EFs+PgWmDEuL/kfjCGLPsWITvwZWHy4Bzj9LMqv2rf8pKIcGiozovqdNkF s3rdW5GBFupI+bESgJ9RQhqJdD63a9TeLb/Fn07JexqZC7Nps1lLa2RhnlnI4RtzqsVUgSCnB6E EDzrxIZnyFRjV/+Yuqws/WVLcPp1FBm0Azy1eX2WtLTqNKGSZONrJf+ehJOOQf0ueC/DTnYzQqX ce61uGAbRPoIRmAhoyoSDL4F+L5tnPoJK7YjwMUvqzRg52MlbPXSmy2OmgSp2OyNEYODdaTiCEM LEKxaigHVXeBdhotfxBSYq5gwYshMevu3v4FjaMrO2rf95lDwL/kHxbt+hjWMkZiNebJStu1O/8 /KF6ruVFSWxANhwVpiPKBQgoCblvvCwgcQDdfYqRHPspJKzd954zur69Gvt/1A2n+l9qJQYRjfT ThqdrYF2SQyDHaQsDm9yIIrSPAGr6HngPjHpB55jQV5E9ruSOlF+fRQxiM2I8BD7EWdz7WOT+UY dE29+68tj01lh7g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement Ccm algorithm for Mali-C55 base on the libipa CcmAlgorithm class. Mali has configurable colour gains as part of the CCM hardware block. Fix them to 1.0 for the time being as white balance is performed by the dedicated Awb algorithm in the Bayer colour space. The existing Mali Ccm algorithm implementation is a bit more strict on when to re-interpolate the CCM coefficients. Keep the algorithm a little more strict and only interpolate on 20% color temperature changes. Signed-off-by: Jacopo Mondi --- src/ipa/mali-c55/algorithms/ccm.cpp | 175 ++++++++++++++++++++++++++++++++ src/ipa/mali-c55/algorithms/ccm.h | 66 ++++++++++++ src/ipa/mali-c55/algorithms/meson.build | 1 + src/ipa/mali-c55/ipa_context.h | 5 + 4 files changed, 247 insertions(+) diff --git a/src/ipa/mali-c55/algorithms/ccm.cpp b/src/ipa/mali-c55/algorithms/ccm.cpp new file mode 100644 index 000000000000..008ab2d1e31e --- /dev/null +++ b/src/ipa/mali-c55/algorithms/ccm.cpp @@ -0,0 +1,175 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * Mali C55 Color Correction Matrix control algorithm + */ + +#include "ccm.h" + +#include + +/** + * \file ccm.h + * \brief Mali-C55 CCM implementation + */ + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +LOG_DEFINE_CATEGORY(MaliC55Ccm) + +/** + * \class Ccm + * \brief Mali-C55 color correction matrix algorithm + */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init(IPAContext &context, const ValueNode &tuningData) +{ + auto &cmap = context.ctrlMap; + int ret = ccmAlgo_.init(tuningData, cmap); + if (ret) + return ret; + + /* + * Mali-C55 allows to perform WB in the RGB color space as part of the + * CCM. Fix the gains at 1.0 as we perform White Balance in the Bayer + * domain. + */ + gain_ = 1.0; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + lastCt_ = context.activeState.awb.automatic.temperatureK; + return ccmAlgo_.configure(context.activeState.ccm, lastCt_); +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* Nothing to do here, the ccm will be calculated in prepare() */ + if (frameContext.awb.autoEnabled) + return; + + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(MaliC55Params *params, const IPAFrameContext &frameContext) +{ + auto config = params->block(); + + for (unsigned int i = 0; i < 3; i++) + config->gains[i] = UQ<4, 8>(gain_).quantized(); + + for (unsigned int i = 0; i < 3; i++) + config->offs[i] = frameContext.ccm.offsets[i][0]; + + const Matrix &ccm = frameContext.ccm.ccm; + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) { + uint16_t val = Q<5, 8>(ccm[i][j]).quantized(); + + if (val && ccm[i][j] < 0) { + /* + * CCM coefficients are expected to be in + * sign/magnitude format. + * + * As we're here handling the case where + * ccm[i][j] is negative, its quantized + * representation is stored in 'val' as + * 2's complement. + * + * We need to invert the 2's complement by + * re-complementing it to 1 and adding 1 back. + * + * Let's make a practical example on how to + * reverse the 2's complement of -7 with 4 bits + * and BIT(5) for sign. + * + * 2's complement (ignore sign bit) + * - 1's complement of abs(-7) + * 2^4 - 1 - 7 = 8 -> 1000 + * - Add one: 8 + 1 = 9 -> 1001 + * + * -7 is then [1 1001] in memory in 2's complement + * + * Reverse 2's complement (ignore sign bit) + * - 1's complement of the 2's complement representation + * 2^4 - 1 - 9 = 6 -> 0110 + * - add one: 6 + 1 = 7 -> 0111 + * + * -7 is then [1 0111] in memory in Sign/Magnitude + */ + val = ((~(val & ~(1 << 12)) & 0x1ff) + 1) | (1 << 12); + } + config->coeffs[i][j] = val; + } + } + + config.setEnabled(true); + + LOG(MaliC55Ccm, Debug) << "Setting ccm: " << ccm << " offsets: " + << frameContext.ccm.offsets + << " with fixed gain " << gain_; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, MaliC55Params *params) +{ + if (!frameContext.awb.autoEnabled) { + setParameters(params, frameContext); + return; + } + + /* + * The Mali-C55 Ccm implementation is slightly stricter than the + * CcmAlgorithm class and only re-interpolates if the colour temperature + * changes of a certain amount. + */ + float ct = frameContext.awb.temperatureK * 1.0f; + if (frame > 0 && (ct < lastCt_ * 1.2 && ct > lastCt_ * 0.8)) { + frameContext.ccm.ccm = context.activeState.ccm.automatic.ccm; + frameContext.ccm.offsets = context.activeState.ccm.automatic.offsets; + lastCt_ = ct; + + return; + } + + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, frame, ct); + setParameters(params, frameContext); + lastCt_ = ct; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const mali_c55_stats_buffer *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/ccm.h b/src/ipa/mali-c55/algorithms/ccm.h new file mode 100644 index 000000000000..73649204a7ee --- /dev/null +++ b/src/ipa/mali-c55/algorithms/ccm.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * Mali C55 Color Correction Matrix control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/ccm.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::mali_c55::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() {} + ~Ccm() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + MaliC55Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const mali_c55_stats_buffer *stats, + ControlList &metadata) override; + +private: + void setParameters(MaliC55Params *params, const IPAFrameContext &context); + + /* + * The CCM coefficient registers are said to be in Q<4,8> but this + * doesn't include the sign bit as the register is 13 bits wide + * (Q-format TI variant). + * + * As the Quantized class uses the ARM variant of the Q-format notation, + * make it <5, 8> to include the sign bit. + */ + CcmAlgorithm> ccmAlgo_; + float gain_; + float lastCt_; +}; + +} /* namespace ipa::mali_c55::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build index 1665da071634..24a27ce66562 100644 --- a/src/ipa/mali-c55/algorithms/meson.build +++ b/src/ipa/mali-c55/algorithms/meson.build @@ -4,5 +4,6 @@ mali_c55_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'blc.cpp', + 'ccm.cpp', 'lsc.cpp', ]) diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index 3d884ea17eb8..2d3e91d56baa 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -16,6 +16,7 @@ #include #include "libipa/awb.h" +#include "libipa/ccm.h" #include "libipa/fixedpoint.h" namespace libcamera { @@ -59,6 +60,8 @@ struct IPAActiveState { } agc; ipa::awb::ActiveState awb; + + ipa::ccm::ActiveState ccm; }; struct IPAFrameContext : public FrameContext { @@ -69,6 +72,8 @@ struct IPAFrameContext : public FrameContext { } agc; ipa::awb::FrameContext awb; + + ipa::ccm::FrameContext ccm; }; struct IPAContext {