From patchwork Thu May 15 16:04:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23377 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8CD1AC3220 for ; Thu, 15 May 2025 16:04:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 43DAE68B6F; Thu, 15 May 2025 18:04:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="YZ2/exj4"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4DB168B4F for ; Thu, 15 May 2025 18:04:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1747325086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E8rqiQAHcdJ8edqF8tKFSbZIleKMy9ZrenykOPkVZis=; b=YZ2/exj4F7I4TNArFHZrjUSznqdZKCSJnYenSisFn3CUyazLsBBtIYxDx3ZFqKpHWlUir2 bWAFLHevyoPTS2HOxocqiTuhAedzG7PJuqlaWkK3z+BDx1kuqiSUcbyCjJhsSIC5BV/mBT kSAQnNnhMcATtgFpWXvSGEzJagpkM6A= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-610-kx5F9P05MCS10Dr3LSN1YA-1; Thu, 15 May 2025 12:04:43 -0400 X-MC-Unique: kx5F9P05MCS10Dr3LSN1YA-1 X-Mimecast-MFC-AGG-ID: kx5F9P05MCS10Dr3LSN1YA_1747325082 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1DB921955DAD; Thu, 15 May 2025 16:04:42 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.124]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2D85B1956066; Thu, 15 May 2025 16:04:39 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Paul Elder , Laurent Pinchart , Kieran Bingham Subject: [PATCH v3 1/1] libcamera: software_isp: Add saturation control Date: Thu, 15 May 2025 18:04:31 +0200 Message-ID: <20250515160432.119022-2-mzamazal@redhat.com> In-Reply-To: <20250515160432.119022-1-mzamazal@redhat.com> References: <20250515160432.119022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 1nwrRwEemyeGqjLi1vDtQG4_VxuC-Ukfab4hXAVg_bQ_1747325082 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Saturation control is added on top of the colour correction matrix. A method of saturation adjustment that can be fully integrated into the colour correction matrix is used. The control is available only if Ccm algorithm is enabled. The control uses 0.0-2.0 value range, with 1.0 being unmodified saturation, 0.0 full desaturation and 2.0 quite saturated. The saturation is adjusted by converting to Y'CbCr colour space, applying the saturation value on the colour axes, and converting back to RGB. ITU-R BT.601 conversion is used to convert between the colour spaces, for no particular reason. The colour correction matrix is applied before gamma and the given matrix is suitable for such a case. Alternatively, the transformation used in libcamera rpi ccm.cpp could be used. Signed-off-by: Milan Zamazal Reviewed-by: Paul Elder --- src/ipa/simple/algorithms/ccm.cpp | 60 +++++++++++++++++++++++++++++-- src/ipa/simple/algorithms/ccm.h | 11 ++++++ src/ipa/simple/ipa_context.h | 4 +++ 3 files changed, 72 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index d5ba928d..0a98406c 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -3,7 +3,7 @@ * Copyright (C) 2024, Ideas On Board * Copyright (C) 2024-2025, Red Hat Inc. * - * Color correction matrix + * Color correction matrix + saturation */ #include "ccm.h" @@ -13,6 +13,8 @@ #include +#include "libcamera/internal/matrix.h" + namespace { constexpr unsigned int kTemperatureThreshold = 100; @@ -35,28 +37,77 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData } context.ccmEnabled = true; + context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); + + return 0; +} + +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + context.activeState.knobs.saturation = std::optional(); return 0; } +void Ccm::queueRequest(typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, + const ControlList &controls) +{ + const auto &saturation = controls.get(controls::Saturation); + if (saturation.has_value()) { + context.activeState.knobs.saturation = saturation; + LOG(IPASoftCcm, Debug) << "Setting saturation to " << saturation.value(); + } +} + +void Ccm::applySaturation(Matrix &ccm, float saturation) +{ + /* https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion */ + const Matrix rgb2ycbcr{ + { 0.256788235294, 0.504129411765, 0.0979058823529, + -0.148223529412, -0.290992156863, 0.439215686275, + 0.439215686275, -0.367788235294, -0.0714274509804 } + }; + const Matrix ycbcr2rgb{ + { 1.16438356164, 0, 1.59602678571, + 1.16438356164, -0.391762290094, -0.812967647235, + 1.16438356164, 2.01723214285, 0 } + }; + const Matrix saturationMatrix{ + { 1, 0, 0, + 0, saturation, 0, + 0, 0, saturation } + }; + ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm; +} + void Ccm::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) { + auto &saturation = context.activeState.knobs.saturation; + const unsigned int ct = context.activeState.awb.temperatureK; - /* Change CCM only on bigger temperature changes. */ + /* Change CCM only on saturation or bigger temperature changes. */ if (frame > 0 && - utils::abs_diff(ct, lastCt_) < kTemperatureThreshold) { + utils::abs_diff(ct, lastCt_) < kTemperatureThreshold && + saturation == lastSaturation_) { frameContext.ccm.ccm = context.activeState.ccm.ccm; context.activeState.ccm.changed = false; return; } lastCt_ = ct; + lastSaturation_ = saturation; Matrix ccm = ccm_.getInterpolated(ct); + if (saturation) + applySaturation(ccm, saturation.value()); context.activeState.ccm.ccm = ccm; frameContext.ccm.ccm = ccm; + frameContext.saturation = saturation; context.activeState.ccm.changed = true; } @@ -67,6 +118,9 @@ void Ccm::process([[maybe_unused]] IPAContext &context, ControlList &metadata) { metadata.set(controls::ColourCorrectionMatrix, frameContext.ccm.ccm.data()); + + const auto &saturation = frameContext.saturation; + metadata.set(controls::Saturation, saturation.value_or(1.0)); } REGISTER_IPA_ALGORITHM(Ccm, "Ccm") diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h index f4e2b85b..8279a3d5 100644 --- a/src/ipa/simple/algorithms/ccm.h +++ b/src/ipa/simple/algorithms/ccm.h @@ -7,6 +7,8 @@ #pragma once +#include + #include "libcamera/internal/matrix.h" #include @@ -24,6 +26,12 @@ public: ~Ccm() = default; int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void queueRequest(typename Module::Context &context, + const uint32_t frame, + typename Module::FrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, @@ -34,7 +42,10 @@ public: ControlList &metadata) override; private: + void applySaturation(Matrix &ccm, float saturation); + unsigned int lastCt_; + std::optional lastSaturation_; Interpolator> ccm_; }; diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 88cc6c35..a471b80a 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -63,6 +63,7 @@ struct IPAActiveState { struct { /* 0..2 range, 1.0 = normal */ std::optional contrast; + std::optional saturation; } knobs; }; @@ -75,11 +76,14 @@ struct IPAFrameContext : public FrameContext { int32_t exposure; double gain; } sensor; + struct { double red; double blue; } gains; + std::optional contrast; + std::optional saturation; }; struct IPAContext {