From patchwork Thu Nov 20 10:45:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 25093 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7E253C3330 for ; Thu, 20 Nov 2025 10:46:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3294260AAE; Thu, 20 Nov 2025 11:46:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KdBqqX8p"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AAC9D60AAB for ; Thu, 20 Nov 2025 11:46:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763635579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nzU1JruVmpvV/SjwbmJAKlBGk479eMSAy9Kc0Vk3tLM=; b=KdBqqX8phevXWiGdqkLKwVl7rSB6W0E2OupOxhkk0qHuIcFM/aPY1MVXoTdSeUXnpd0aM/ Cn9w86Dd+mcCUxTjzJ0pgaRUn7NtLwgAeO2AdJGVvLDcIBIofMiaUZnVTMhUvqulQ20DuO vHLr2H7L38JQTAVJnxukY61hXABXXHU= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-137-e5Wv2KO_N7qk3maF7r1bqQ-1; Thu, 20 Nov 2025 05:46:17 -0500 X-MC-Unique: e5Wv2KO_N7qk3maF7r1bqQ-1 X-Mimecast-MFC-AGG-ID: e5Wv2KO_N7qk3maF7r1bqQ_1763635576 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 80754195606B; Thu, 20 Nov 2025 10:46:16 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.39]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A58DE1800451; Thu, 20 Nov 2025 10:46:14 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [RFC PATCH v2 07/13] libcamera: ipa: simple: Separate saturation from CCM Date: Thu, 20 Nov 2025 11:45:42 +0100 Message-ID: <20251120104548.80268-8-mzamazal@redhat.com> In-Reply-To: <20251120104548.80268-1-mzamazal@redhat.com> References: <20251120104548.80268-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: O8-Yyx59ZQ7qOq3_Cee8Tc3TLG_e-F-mssGMQZ5Y4Hs_1763635576 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Saturation adjustments are implemented using matrix operations. They are currently applied to the colour correction matrix. Let's move them to a newly introduced separate "Adjust" algorithm. This separation has the following advantages: - It allows disabling general colour adjustments algorithms without disabling the CCM algorithm. - It keeps the CCM separated from other corrections. - It's generally cleaner. Reviewed-by: Kieran Bingham Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/adjust.cpp | 106 ++++++++++++++++++++++++++ src/ipa/simple/algorithms/adjust.h | 52 +++++++++++++ src/ipa/simple/algorithms/ccm.cpp | 58 +------------- src/ipa/simple/algorithms/ccm.h | 9 --- src/ipa/simple/algorithms/meson.build | 1 + src/ipa/simple/data/uncalibrated.yaml | 1 + 6 files changed, 163 insertions(+), 64 deletions(-) create mode 100644 src/ipa/simple/algorithms/adjust.cpp create mode 100644 src/ipa/simple/algorithms/adjust.h diff --git a/src/ipa/simple/algorithms/adjust.cpp b/src/ipa/simple/algorithms/adjust.cpp new file mode 100644 index 000000000..b9a7864ad --- /dev/null +++ b/src/ipa/simple/algorithms/adjust.cpp @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * Copyright (C) 2024-2025, Red Hat Inc. + * + * Common image adjustments + */ + +#include "adjust.h" + +#include +#include + +#include + +#include "libcamera/internal/matrix.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftAdjust) + +int Adjust::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + if (context.ccmEnabled) + context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); + return 0; +} + +int Adjust::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + context.activeState.knobs.saturation = std::optional(); + + return 0; +} + +void Adjust::queueRequest(typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, + const ControlList &controls) +{ + const auto &saturation = controls.get(controls::Saturation); + if (saturation.has_value()) { + context.activeState.knobs.saturation = saturation; + LOG(IPASoftAdjust, Debug) << "Setting saturation to " << saturation.value(); + } +} + +void Adjust::applySaturation(Matrix &matrix, float saturation) +{ + /* https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion */ + const Matrix rgb2ycbcr{ + { 0.256788235294, 0.504129411765, 0.0979058823529, + -0.148223529412, -0.290992156863, 0.439215686275, + 0.439215686275, -0.367788235294, -0.0714274509804 } + }; + const Matrix ycbcr2rgb{ + { 1.16438356164, 0, 1.59602678571, + 1.16438356164, -0.391762290094, -0.812967647235, + 1.16438356164, 2.01723214285, 0 } + }; + const Matrix saturationMatrix{ + { 1, 0, 0, + 0, saturation, 0, + 0, 0, saturation } + }; + matrix = + ycbcr2rgb * saturationMatrix * rgb2ycbcr * matrix; +} + +void Adjust::prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] DebayerParams *params) +{ + if (!context.ccmEnabled) + return; + + auto &saturation = context.activeState.knobs.saturation; + frameContext.saturation = saturation; + if (saturation) + applySaturation(context.activeState.combinedMatrix, saturation.value()); + + if (frame == 0 || saturation != lastSaturation_) { + context.activeState.matrixChanged = true; + lastSaturation_ = saturation; + } +} + +void Adjust::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + const auto &saturation = frameContext.saturation; + metadata.set(controls::Saturation, saturation.value_or(1.0)); +} + +REGISTER_IPA_ALGORITHM(Adjust, "Adjust") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/adjust.h b/src/ipa/simple/algorithms/adjust.h new file mode 100644 index 000000000..c4baa2503 --- /dev/null +++ b/src/ipa/simple/algorithms/adjust.h @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024-2025, Red Hat Inc. + * + * Color correction matrix + */ + +#pragma once + +#include + +#include "libcamera/internal/matrix.h" + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class Adjust : public Algorithm +{ +public: + Adjust() = default; + ~Adjust() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void queueRequest(typename Module::Context &context, + const uint32_t frame, + typename Module::FrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; + +private: + void applySaturation(Matrix &ccm, float saturation); + + std::optional lastSaturation_; +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index 52dd11aaa..50bb3164f 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -3,7 +3,7 @@ * Copyright (C) 2024, Ideas On Board * Copyright (C) 2024-2025, Red Hat Inc. * - * Color correction matrix + saturation + * Color correction matrix */ #include "ccm.h" @@ -37,75 +37,26 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData } context.ccmEnabled = true; - context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); return 0; } -int Ccm::configure(IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) -{ - context.activeState.knobs.saturation = std::optional(); - - return 0; -} - -void Ccm::queueRequest(typename Module::Context &context, - [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] typename Module::FrameContext &frameContext, - const ControlList &controls) -{ - const auto &saturation = controls.get(controls::Saturation); - if (saturation.has_value()) { - context.activeState.knobs.saturation = saturation; - LOG(IPASoftCcm, Debug) << "Setting saturation to " << saturation.value(); - } -} - -void Ccm::applySaturation(Matrix &ccm, float saturation) -{ - /* https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion */ - const Matrix rgb2ycbcr{ - { 0.256788235294, 0.504129411765, 0.0979058823529, - -0.148223529412, -0.290992156863, 0.439215686275, - 0.439215686275, -0.367788235294, -0.0714274509804 } - }; - const Matrix ycbcr2rgb{ - { 1.16438356164, 0, 1.59602678571, - 1.16438356164, -0.391762290094, -0.812967647235, - 1.16438356164, 2.01723214285, 0 } - }; - const Matrix saturationMatrix{ - { 1, 0, 0, - 0, saturation, 0, - 0, 0, saturation } - }; - ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm; -} - void Ccm::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) { - auto &saturation = context.activeState.knobs.saturation; - const unsigned int ct = context.activeState.awb.temperatureK; - /* Change CCM only on saturation or bigger temperature changes. */ + /* Change CCM only on bigger temperature changes. */ if (frame == 0 || - utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold || - saturation != lastSaturation_) { + utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold) { currentCcm_ = ccm_.getInterpolated(ct); - if (saturation) - applySaturation(currentCcm_, saturation.value()); lastCt_ = ct; - lastSaturation_ = saturation; context.activeState.matrixChanged = true; } context.activeState.combinedMatrix = currentCcm_ * context.activeState.combinedMatrix; context.activeState.ccm = currentCcm_; - frameContext.saturation = saturation; frameContext.ccm = currentCcm_; } @@ -116,9 +67,6 @@ void Ccm::process([[maybe_unused]] IPAContext &context, ControlList &metadata) { metadata.set(controls::ColourCorrectionMatrix, frameContext.ccm.data()); - - const auto &saturation = frameContext.saturation; - metadata.set(controls::Saturation, saturation.value_or(1.0)); } REGISTER_IPA_ALGORITHM(Ccm, "Ccm") diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h index c29f394f0..a7613f0f9 100644 --- a/src/ipa/simple/algorithms/ccm.h +++ b/src/ipa/simple/algorithms/ccm.h @@ -26,12 +26,6 @@ public: ~Ccm() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - int configure(IPAContext &context, - const IPAConfigInfo &configInfo) override; - void queueRequest(typename Module::Context &context, - const uint32_t frame, - typename Module::FrameContext &frameContext, - const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, @@ -42,10 +36,7 @@ public: ControlList &metadata) override; private: - void applySaturation(Matrix &ccm, float saturation); - unsigned int lastCt_; - std::optional lastSaturation_; Interpolator> ccm_; Matrix currentCcm_; }; diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index 2d0adb059..ebe9f20dd 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 soft_simple_ipa_algorithms = files([ + 'adjust.cpp', 'awb.cpp', 'agc.cpp', 'blc.cpp', diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index 5508e6686..f0410fe61 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -14,6 +14,7 @@ algorithms: # ccm: [ 1, 0, 0, # 0, 1, 0, # 0, 0, 1] + - Adjust: - Lut: - Agc: ...