From patchwork Thu Jan 30 18:14:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22704 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5A672BD808 for ; Thu, 30 Jan 2025 18:15:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C9E8A68578; Thu, 30 Jan 2025 19:15:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="I0MOpqSJ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7C21F6856B for ; Thu, 30 Jan 2025 19:15:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738260927; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ff4csIfc6p7Y4CYFK+G1ddx3hrfg1CFFjFgWzkdPMF8=; b=I0MOpqSJYyRNmZvPihEGNXz3dRl3AYYs+zR5SmGxsbXXmVdcQjjubkMdm8N1eeT/JCK2DN n3GKcXPU2eOjaCSEpgXTzdo0dAtFr9RISqb+Yaj7bDnLgB6T5riVj/YngJrovY58vMKVHR Ke3B1EvVj3sb39fxj9YuuQa5Ak4XLmI= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-292-KOiem_i6N8ueJml8Zx_UFA-1; Thu, 30 Jan 2025 13:15:24 -0500 X-MC-Unique: KOiem_i6N8ueJml8Zx_UFA-1 X-Mimecast-MFC-AGG-ID: KOiem_i6N8ueJml8Zx_UFA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5181719560AA; Thu, 30 Jan 2025 18:15:23 +0000 (UTC) Received: from mzamazal-thinkpadp1gen3.tpbc.com (unknown [10.45.224.122]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ECDF030001BE; Thu, 30 Jan 2025 18:15:20 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Robert Mader , Hans de Goede , Laurent Pinchart , Kieran Bingham Subject: [PATCH v5 07/10] libcamera: software_isp: Add CCM algorithm Date: Thu, 30 Jan 2025 19:14:44 +0100 Message-ID: <20250130181449.130492-8-mzamazal@redhat.com> In-Reply-To: <20250130181449.130492-1-mzamazal@redhat.com> References: <20250130181449.130492-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: PxZLcGmpUQIpobUFLMur07-QOhcFyJe9G_sX5ig9KTo_1738260923 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds color correction matrix (CCM) algorithm to software ISP. It is based on the corresponding algorithm in rkisp1. The primary difference against hardware pipelines is that applying the CCM is optional. Applying CCM causes a significant slowdown, time needed to process a frame raises by 40-90% on tested platforms. If CCM is really needed, it can be applied, if not, it's better to stick without it. This can be configured by presence or omission of Ccm algorithm in the tuning file. CCM is changed only if the determined temperature changes by at least 100 K (an arbitrarily selected value), to avoid recomputing the matrices and lookup tables all the time. The outputs of the algorithm are not used yet, they will be enabled in followup patches. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/ipa/simple/algorithms/ccm.cpp | 79 +++++++++++++++++++++++++++ src/ipa/simple/algorithms/ccm.h | 43 +++++++++++++++ src/ipa/simple/algorithms/meson.build | 1 + src/ipa/simple/ipa_context.h | 12 ++++ 4 files changed, 135 insertions(+) create mode 100644 src/ipa/simple/algorithms/ccm.cpp create mode 100644 src/ipa/simple/algorithms/ccm.h diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp new file mode 100644 index 00000000..069a12f8 --- /dev/null +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas On Board + * Copyright (C) 2024-2025, Red Hat Inc. + * + * Color correction matrix + */ + +#include "ccm.h" + +#include +#include + +#include + +namespace { + +constexpr unsigned int kTemperatureThreshold = 100; + +} + +namespace libcamera { + +namespace ipa::soft::algorithms { + +LOG_DEFINE_CATEGORY(IPASoftCcm) + +int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + int ret = ccm_.readYaml(tuningData["ccms"], "ct", "ccm"); + if (ret < 0) { + LOG(IPASoftCcm, Error) + << "Failed to parse 'ccm' parameter from tuning file."; + return ret; + } + + return 0; +} + +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) +{ + const unsigned int ct = context.activeState.awb.temperatureK; + + /* Change CCM only on bigger temperature changes. */ + if (frame > 0 && + utils::abs_diff(ct, lastCt_) < kTemperatureThreshold) { + frameContext.ccm.ccm = context.activeState.ccm.ccm; + context.activeState.ccm.changed = false; + return; + } + + lastCt_ = ct; + Matrix ccm = ccm_.getInterpolated(ct); + + context.activeState.ccm.ccm = ccm; + frameContext.ccm.ccm = ccm; + context.activeState.ccm.changed = true; +} + +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const SwIspStats *stats, + ControlList &metadata) +{ + float m[9]; + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + m[i * 3 + j] = frameContext.ccm.ccm[i][j]; + } + metadata.set(controls::ColourCorrectionMatrix, m); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/ccm.h b/src/ipa/simple/algorithms/ccm.h new file mode 100644 index 00000000..e5dc6506 --- /dev/null +++ b/src/ipa/simple/algorithms/ccm.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024-2025, Red Hat Inc. + * + * Color correction matrix + */ + +#pragma once + +#include "libcamera/internal/matrix.h" + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::soft::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() = default; + ~Ccm() = default; + + int init(IPAContext &context, const YamlObject &tuningData) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const SwIspStats *stats, + ControlList &metadata) override; + +private: + unsigned int lastCt_; + Interpolator> ccm_; +}; + +} /* namespace ipa::soft::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index 37a2eb53..2d0adb05 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -4,5 +4,6 @@ soft_simple_ipa_algorithms = files([ 'awb.cpp', 'agc.cpp', 'blc.cpp', + 'ccm.cpp', 'lut.cpp', ]) diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index df0552db..bd6c66d8 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -13,6 +13,8 @@ #include +#include "libcamera/internal/matrix.h" + #include #include @@ -47,6 +49,12 @@ struct IPAActiveState { uint8_t blackLevel; double contrast; } gamma; + + struct { + Matrix ccm; + bool changed; + } ccm; + struct { /* 0..2 range, 1.0 = normal */ std::optional contrast; @@ -54,6 +62,10 @@ struct IPAActiveState { }; struct IPAFrameContext : public FrameContext { + struct { + Matrix ccm; + } ccm; + struct { int32_t exposure; double gain;