From patchwork Mon Apr 7 08:58:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23154 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8C94EC327D for ; Mon, 7 Apr 2025 08:58:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 369C468A54; Mon, 7 Apr 2025 10:58:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="W94KqSDY"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E8316689A8 for ; Mon, 7 Apr 2025 10:58:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016290; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vU/wIqwfm1co2AE+x5OlG+iqbSHfl5NlTF4KIJlDUos=; b=W94KqSDYbMfyGIxeFAyXKY7JbER4EfYXkblZvy0tvqU4A/KbA05Gr+aXlZXkHNfpA0cwUE p/bKUlZSx6tr0nsjHQCD9ViQy1DJu2Y0fEv/ApweEN9qZrYTeLlTCJo0sIeJyfVVAg2hih KyNbiyXLyb7GgkwVgii42KCKU8Q/Esw= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-628-a_5XSCVfOjqZVpmoav635A-1; Mon, 07 Apr 2025 04:58:08 -0400 X-MC-Unique: a_5XSCVfOjqZVpmoav635A-1 X-Mimecast-MFC-AGG-ID: a_5XSCVfOjqZVpmoav635A_1744016287 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A371E19560AB for ; Mon, 7 Apr 2025 08:58:07 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7736E180A803; Mon, 7 Apr 2025 08:58:06 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH] libcamera: software_isp: Add saturation control Date: Mon, 7 Apr 2025 10:58:02 +0200 Message-ID: <20250407085802.16269-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 7c1ZrCax0UWVUcEglVFbLxuB2Kocl0i_Hc7KuUJDY9M_1744016287 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Saturation control is added on top of the colour correction matrix. A way of saturation adjustment that can be fully integrated into the colour correction matrix is used. The control is available only if Ccm algorithm is enabled. The control uses 0.0-2.0 value range, with 1.0 being unmodified saturation, 0.0 full desaturation and 2.0 quite saturated. The matrix for saturation adjustment is taken from https://www.graficaobscura.com/matrix/index.html. The colour correction matrix is applied before gamma and the given matrix is suitable for such a case. Alternatively, the transformation used in libcamera rpi ccm.cpp could be used. Signed-off-by: Milan Zamazal Tested-by: Kieran Bingham Reviewed-by: Kieran Bingham --- src/ipa/simple/algorithms/ccm.cpp | 57 +++++++++++++++++++++++++++++-- src/ipa/simple/algorithms/ccm.h | 11 ++++++ src/ipa/simple/ipa_context.h | 4 +++ 3 files changed, 69 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index d5ba928d..2700a247 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -3,7 +3,7 @@ * Copyright (C) 2024, Ideas On Board * Copyright (C) 2024-2025, Red Hat Inc. * - * Color correction matrix + * Color correction matrix + saturation */ #include "ccm.h" @@ -13,6 +13,8 @@ #include +#include "libcamera/internal/matrix.h" + namespace { constexpr unsigned int kTemperatureThreshold = 100; @@ -35,28 +37,73 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData } context.ccmEnabled = true; + context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); + + return 0; +} + +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + context.activeState.knobs.saturation = std::optional(); return 0; } +void Ccm::queueRequest(typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, + const ControlList &controls) +{ + const auto &saturation = controls.get(controls::Saturation); + if (saturation.has_value()) { + context.activeState.knobs.saturation = saturation; + LOG(IPASoftCcm, Debug) << "Setting saturation to " << saturation.value(); + } +} + +void Ccm::updateSaturation(Matrix &ccm, float saturation) +{ + /* + * See https://www.graficaobscura.com/matrix/index.html. + * This is applied before gamma thus a matrix for linear RGB must be used. + * The saturation range is 0..2, with 1 being an unchanged saturation and 0 + * no saturation (monochrome). + */ + constexpr float r = 0.3086; + constexpr float g = 0.6094; + constexpr float b = 0.0820; + const float s1 = 1.0 - saturation; + ccm = ccm * Matrix{ { s1 * r + saturation, s1 * g, s1 * b, + s1 * r, s1 * g + saturation, s1 * b, + s1 * r, s1 * g, s1 * b + saturation } }; +} + void Ccm::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) { + auto &saturation = context.activeState.knobs.saturation; + const unsigned int ct = context.activeState.awb.temperatureK; - /* Change CCM only on bigger temperature changes. */ + /* Change CCM only on saturation or bigger temperature changes. */ if (frame > 0 && - utils::abs_diff(ct, lastCt_) < kTemperatureThreshold) { + utils::abs_diff(ct, lastCt_) < kTemperatureThreshold && + saturation == lastSaturation_) { frameContext.ccm.ccm = context.activeState.ccm.ccm; context.activeState.ccm.changed = false; return; } lastCt_ = ct; + lastSaturation_ = saturation; Matrix ccm = ccm_.getInterpolated(ct); + if (saturation) + updateSaturation(ccm, saturation.value()); context.activeState.ccm.ccm = ccm; frameContext.ccm.ccm = ccm; + frameContext.saturation = saturation; context.activeState.ccm.changed = true; } @@ -67,6 +114,10 @@ void Ccm::process([[maybe_unused]] IPAContext &context, ControlList &metadata) { metadata.set(controls::ColourCorrectionMatrix, frameContext.ccm.ccm.data()); + + const auto &saturation = frameContext.saturation; + if (saturation) + metadata.set(controls::Saturation, saturation.value()); } REGISTER_IPA_ALGORITHM(Ccm, "Ccm") diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h index f4e2b85b..2c5d2170 100644 --- a/src/ipa/simple/algorithms/ccm.h +++ b/src/ipa/simple/algorithms/ccm.h @@ -7,6 +7,8 @@ #pragma once +#include + #include "libcamera/internal/matrix.h" #include @@ -24,6 +26,12 @@ public: ~Ccm() = default; int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void queueRequest(typename Module::Context &context, + const uint32_t frame, + typename Module::FrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, @@ -34,7 +42,10 @@ public: ControlList &metadata) override; private: + void updateSaturation(Matrix &ccm, float saturation); + unsigned int lastCt_; + std::optional lastSaturation_; Interpolator> ccm_; }; diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 88cc6c35..56792981 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -63,6 +63,7 @@ struct IPAActiveState { struct { /* 0..2 range, 1.0 = normal */ std::optional contrast; + std::optional saturation; } knobs; }; @@ -75,11 +76,14 @@ struct IPAFrameContext : public FrameContext { int32_t exposure; double gain; } sensor; + struct { double red; double blue; } gains; + std::optional contrast; + std::optional saturation; }; struct IPAContext {