From patchwork Wed Apr 30 11:20:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23307 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB476C327D for ; Wed, 30 Apr 2025 11:21:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F6ED68AD2; Wed, 30 Apr 2025 13:21:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="cjQgKrlx"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 16A8268AD1 for ; Wed, 30 Apr 2025 13:21:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1746012065; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s/nfj9Fkob4YeCdnjJB6V1yWyRrqGaZIFw8aYcurbnU=; b=cjQgKrlxUS+ZqLPF8xo2z5k7u5mxXR773Q6j3e6ETOw65vFCJgIYPZ0AdvnfGWbsE8nir7 agprRYXrUohdKhoc0jQSX/xbEecsZlNFlaJ0gRs+Oa/g6VLWiHQWoAKmpkZiYsjAw1QfSK qUZ3/Gh9zKCFWWb+1mqZROax0RzQkfI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-631-xSMIJdnnN8-MhbsielpTMA-1; Wed, 30 Apr 2025 07:20:59 -0400 X-MC-Unique: xSMIJdnnN8-MhbsielpTMA-1 X-Mimecast-MFC-AGG-ID: xSMIJdnnN8-MhbsielpTMA_1746012058 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 67A9B19560AD; Wed, 30 Apr 2025 11:20:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.120]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E1BA11800359; Wed, 30 Apr 2025 11:20:56 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v2 1/1] libcamera: software_isp: Add saturation control Date: Wed, 30 Apr 2025 13:20:42 +0200 Message-ID: <20250430112043.23807-2-mzamazal@redhat.com> In-Reply-To: <20250430112043.23807-1-mzamazal@redhat.com> References: <20250430112043.23807-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9MQV1ZaxnbUzRdXoCAiEow3NFVoFOmwzlvoCEBZhgAo_1746012058 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Saturation control is added on top of the colour correction matrix. A way of saturation adjustment that can be fully integrated into the colour correction matrix is used. The control is available only if Ccm algorithm is enabled. The control uses 0.0-2.0 value range, with 1.0 being unmodified saturation, 0.0 full desaturation and 2.0 quite saturated. The saturation is adjusted by converting to Y'CbCr colour space, applying the saturation value on the colour axes, and converting back to RGB. ITU-R BT.601 conversion is used to convert between the colour spaces, for no particular reason. The colour correction matrix is applied before gamma and the given matrix is suitable for such a case. Alternatively, the transformation used in libcamera rpi ccm.cpp could be used. Signed-off-by: Milan Zamazal --- src/ipa/simple/algorithms/ccm.cpp | 60 +++++++++++++++++++++++++++++-- src/ipa/simple/algorithms/ccm.h | 11 ++++++ src/ipa/simple/ipa_context.h | 4 +++ 3 files changed, 72 insertions(+), 3 deletions(-) diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index d5ba928d..021aefc9 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -3,7 +3,7 @@ * Copyright (C) 2024, Ideas On Board * Copyright (C) 2024-2025, Red Hat Inc. * - * Color correction matrix + * Color correction matrix + saturation */ #include "ccm.h" @@ -13,6 +13,8 @@ #include +#include "libcamera/internal/matrix.h" + namespace { constexpr unsigned int kTemperatureThreshold = 100; @@ -35,28 +37,77 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData } context.ccmEnabled = true; + context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); + + return 0; +} + +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPAConfigInfo &configInfo) +{ + context.activeState.knobs.saturation = std::optional(); return 0; } +void Ccm::queueRequest(typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, + const ControlList &controls) +{ + const auto &saturation = controls.get(controls::Saturation); + if (saturation.has_value()) { + context.activeState.knobs.saturation = saturation; + LOG(IPASoftCcm, Debug) << "Setting saturation to " << saturation.value(); + } +} + +void Ccm::updateSaturation(Matrix &ccm, float saturation) +{ + /* https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion */ + const Matrix rgb2ycbcr{ + { 0.256788235294, 0.504129411765, 0.0979058823529, + -0.148223529412, -0.290992156863, 0.439215686275, + 0.439215686275, -0.367788235294, -0.0714274509804 } + }; + const Matrix ycbcr2rgb{ + { 1.16438356164, 0, 1.59602678571, + 1.16438356164, -0.391762290094, -0.812967647235, + 1.16438356164, 2.01723214285, 0 } + }; + const Matrix saturationMatrix{ + { 1, 0, 0, + 0, saturation, 0, + 0, 0, saturation } + }; + ccm = ycbcr2rgb * saturationMatrix * rgb2ycbcr * ccm; +} + void Ccm::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) { + auto &saturation = context.activeState.knobs.saturation; + const unsigned int ct = context.activeState.awb.temperatureK; - /* Change CCM only on bigger temperature changes. */ + /* Change CCM only on saturation or bigger temperature changes. */ if (frame > 0 && - utils::abs_diff(ct, lastCt_) < kTemperatureThreshold) { + utils::abs_diff(ct, lastCt_) < kTemperatureThreshold && + saturation == lastSaturation_) { frameContext.ccm.ccm = context.activeState.ccm.ccm; context.activeState.ccm.changed = false; return; } lastCt_ = ct; + lastSaturation_ = saturation; Matrix ccm = ccm_.getInterpolated(ct); + if (saturation) + updateSaturation(ccm, saturation.value()); context.activeState.ccm.ccm = ccm; frameContext.ccm.ccm = ccm; + frameContext.saturation = saturation; context.activeState.ccm.changed = true; } @@ -67,6 +118,9 @@ void Ccm::process([[maybe_unused]] IPAContext &context, ControlList &metadata) { metadata.set(controls::ColourCorrectionMatrix, frameContext.ccm.ccm.data()); + + const auto &saturation = frameContext.saturation; + metadata.set(controls::Saturation, saturation.value_or(1.0)); } REGISTER_IPA_ALGORITHM(Ccm, "Ccm") diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h index f4e2b85b..2c5d2170 100644 --- a/src/ipa/simple/algorithms/ccm.h +++ b/src/ipa/simple/algorithms/ccm.h @@ -7,6 +7,8 @@ #pragma once +#include + #include "libcamera/internal/matrix.h" #include @@ -24,6 +26,12 @@ public: ~Ccm() = default; int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPAConfigInfo &configInfo) override; + void queueRequest(typename Module::Context &context, + const uint32_t frame, + typename Module::FrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, @@ -34,7 +42,10 @@ public: ControlList &metadata) override; private: + void updateSaturation(Matrix &ccm, float saturation); + unsigned int lastCt_; + std::optional lastSaturation_; Interpolator> ccm_; }; diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 88cc6c35..56792981 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -63,6 +63,7 @@ struct IPAActiveState { struct { /* 0..2 range, 1.0 = normal */ std::optional contrast; + std::optional saturation; } knobs; }; @@ -75,11 +76,14 @@ struct IPAFrameContext : public FrameContext { int32_t exposure; double gain; } sensor; + struct { double red; double blue; } gains; + std::optional contrast; + std::optional saturation; }; struct IPAContext {