From patchwork Thu Nov 20 10:45:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 25099 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B5AC9C3330 for ; Thu, 20 Nov 2025 10:46:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B99860AAE; Thu, 20 Nov 2025 11:46:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="V2PLB+u8"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 870BE60AB5 for ; Thu, 20 Nov 2025 11:46:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763635592; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WEe0dQuM8uQflGfUsVvTSuaPJ9If8Vzqz7dkDyx+yQ8=; b=V2PLB+u8mJtMlq3jZjFSoozx3/3oUOg31rZMYGtnoLCapJt86Pz6E8p9THGXd/UhsBid9H mY3CI0XQE0f32oeQ8XsMGrZvSf1tksy3e+mBBQ//Vi0nY6eQJNdQiighuxqBt4eYXpGVCk OFqmPoxVMhP66GfQnDOoszUI/Chfz0Q= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-339-zDYgTBBhMAivND3Mb8DPww-1; Thu, 20 Nov 2025 05:46:30 -0500 X-MC-Unique: zDYgTBBhMAivND3Mb8DPww-1 X-Mimecast-MFC-AGG-ID: zDYgTBBhMAivND3Mb8DPww_1763635589 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CC7611956048; Thu, 20 Nov 2025 10:46:29 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.34.39]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BFAC11800451; Thu, 20 Nov 2025 10:46:27 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [RFC PATCH v2 13/13] libcamera: ipa: simple: Remove Lut algorithm Date: Thu, 20 Nov 2025 11:45:48 +0100 Message-ID: <20251120104548.80268-14-mzamazal@redhat.com> In-Reply-To: <20251120104548.80268-1-mzamazal@redhat.com> References: <20251120104548.80268-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 0s4j3XLeE9OKvMdoVpWKoF6XTxs7ZovdcJyiDDsSIOs_1763635589 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Lut algorithm is not really an algorithm. Moreover, algorithms may be enabled or disabled but with Lut disabled, nothing will work. Let's move the construction of lookup tables to debayering, where it is used. The implied and related changes are: - DebayerParams is changed to contain the real params rather than lookup tables. - The params must be initialised so that debayering gets meaningful parameter values even when some algorithms are disabled. - combinedMatrix must be put to params everywhere where it is modified. - Matrix changes needn't be tracked in the algorithms any more. - Debayering must watch for changes of the corresponding parameters to update the lookup tables when and only when needed. - Swapping red and blue is integrated into lookup table constructions. Signed-off-by: Milan Zamazal --- .../internal/software_isp/debayer_params.h | 42 +---- src/ipa/simple/algorithms/adjust.cpp | 20 +-- src/ipa/simple/algorithms/adjust.h | 2 - src/ipa/simple/algorithms/awb.cpp | 7 +- src/ipa/simple/algorithms/blc.cpp | 8 + src/ipa/simple/algorithms/blc.h | 6 +- src/ipa/simple/algorithms/ccm.cpp | 4 +- src/ipa/simple/algorithms/lut.cpp | 134 -------------- src/ipa/simple/algorithms/lut.h | 35 ---- src/ipa/simple/algorithms/meson.build | 1 - src/ipa/simple/data/uncalibrated.yaml | 1 - src/ipa/simple/ipa_context.h | 9 - src/ipa/simple/soft_simple.cpp | 10 +- src/libcamera/software_isp/debayer.cpp | 71 ++------ src/libcamera/software_isp/debayer_cpu.cpp | 165 +++++++++++++----- src/libcamera/software_isp/debayer_cpu.h | 30 +++- src/libcamera/software_isp/software_isp.cpp | 17 -- 17 files changed, 202 insertions(+), 360 deletions(-) delete mode 100644 src/ipa/simple/algorithms/lut.cpp delete mode 100644 src/ipa/simple/algorithms/lut.h diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 217cd5d92..e10f7f9e9 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -10,45 +10,19 @@ #pragma once -#include #include +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + namespace libcamera { struct DebayerParams { - static constexpr unsigned int kRGBLookupSize = 256; - - struct CcmColumn { - int16_t r; - int16_t g; - int16_t b; - }; - - using LookupTable = std::array; - using CcmLookupTable = std::array; - - /* - * Color lookup tables when CCM is not used. - * - * Each color of a debayered pixel is amended by the corresponding - * value in the given table. - */ - LookupTable red; - LookupTable green; - LookupTable blue; - - /* - * Color and gamma lookup tables when CCM is used. - * - * Each of the CcmLookupTable's corresponds to a CCM column; together they - * make a complete 3x3 CCM lookup table. The CCM is applied on debayered - * pixels and then the gamma lookup table is used to set the resulting - * values of all the three colors. - */ - CcmLookupTable redCcm; - CcmLookupTable greenCcm; - CcmLookupTable blueCcm; - LookupTable gammaLut; + uint8_t blackLevel; + float gamma; + float contrast; + RGB gains; + Matrix combinedMatrix; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/adjust.cpp b/src/ipa/simple/algorithms/adjust.cpp index 96c0bde0a..81f97feeb 100644 --- a/src/ipa/simple/algorithms/adjust.cpp +++ b/src/ipa/simple/algorithms/adjust.cpp @@ -90,24 +90,22 @@ void Adjust::applySaturation(Matrix &matrix, float saturation) } void Adjust::prepare(IPAContext &context, - const uint32_t frame, + [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, - [[maybe_unused]] DebayerParams *params) + DebayerParams *params) { frameContext.contrast = context.activeState.knobs.contrast; - if (!context.ccmEnabled) - return; - auto &saturation = context.activeState.knobs.saturation; - frameContext.saturation = saturation; - if (saturation) + if (context.ccmEnabled && saturation) { applySaturation(context.activeState.combinedMatrix, saturation.value()); - - if (frame == 0 || saturation != lastSaturation_) { - context.activeState.matrixChanged = true; - lastSaturation_ = saturation; + frameContext.saturation = saturation; } + + params->gamma = context.activeState.knobs.gamma.value_or(kDefaultGamma); + params->contrast = + context.activeState.knobs.contrast.value_or(kDefaultContrast); + params->combinedMatrix = context.activeState.combinedMatrix; } void Adjust::process([[maybe_unused]] IPAContext &context, diff --git a/src/ipa/simple/algorithms/adjust.h b/src/ipa/simple/algorithms/adjust.h index 11d8297ca..02e469a50 100644 --- a/src/ipa/simple/algorithms/adjust.h +++ b/src/ipa/simple/algorithms/adjust.h @@ -47,8 +47,6 @@ public: private: void applySaturation(Matrix &ccm, float saturation); - - std::optional lastSaturation_; }; } /* namespace ipa::soft::algorithms */ diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index a0af8a603..b743a69cc 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -37,7 +37,7 @@ int Awb::configure(IPAContext &context, void Awb::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, - [[maybe_unused]] DebayerParams *params) + DebayerParams *params) { auto &gains = context.activeState.awb.gains; Matrix gainMatrix = { { gains.r(), 0, 0, @@ -45,9 +45,12 @@ void Awb::prepare(IPAContext &context, 0, 0, gains.b() } }; context.activeState.combinedMatrix = context.activeState.combinedMatrix * gainMatrix; - /* Just report, the gains are applied in LUT algorithm. */ + frameContext.gains.red = gains.r(); frameContext.gains.blue = gains.b(); + + params->gains = gains; + params->combinedMatrix = context.activeState.combinedMatrix; } void Awb::process(IPAContext &context, diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 370385afc..365360f47 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -47,6 +47,14 @@ int BlackLevel::configure(IPAContext &context, return 0; } +void BlackLevel::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + DebayerParams *params) +{ + params->blackLevel = context.activeState.blc.level; +} + void BlackLevel::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/simple/algorithms/blc.h b/src/ipa/simple/algorithms/blc.h index db9e6d639..8c919452f 100644 --- a/src/ipa/simple/algorithms/blc.h +++ b/src/ipa/simple/algorithms/blc.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2024, Red Hat Inc. + * Copyright (C) 2024-2025 Red Hat Inc. * * Black level handling */ @@ -24,6 +24,10 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + DebayerParams *params) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index 50bb3164f..561c6e123 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -42,7 +42,7 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData } void Ccm::prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, [[maybe_unused]] DebayerParams *params) + IPAFrameContext &frameContext, DebayerParams *params) { const unsigned int ct = context.activeState.awb.temperatureK; @@ -51,13 +51,13 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame, utils::abs_diff(ct, lastCt_) >= kTemperatureThreshold) { currentCcm_ = ccm_.getInterpolated(ct); lastCt_ = ct; - context.activeState.matrixChanged = true; } context.activeState.combinedMatrix = currentCcm_ * context.activeState.combinedMatrix; context.activeState.ccm = currentCcm_; frameContext.ccm = currentCcm_; + params->combinedMatrix = context.activeState.combinedMatrix; } void Ccm::process([[maybe_unused]] IPAContext &context, diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp deleted file mode 100644 index bbf7f168a..000000000 --- a/src/ipa/simple/algorithms/lut.cpp +++ /dev/null @@ -1,134 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2024-2025, Red Hat Inc. - * - * Color lookup tables construction - */ - -#include "lut.h" - -#include -#include -#include -#include - -#include - -#include - -#include "simple/ipa_context.h" - -#include "adjust.h" - -namespace libcamera { - -LOG_DEFINE_CATEGORY(IPASoftLut) - -namespace ipa::soft::algorithms { - -int Lut::configure(IPAContext &context, - [[maybe_unused]] const IPAConfigInfo &configInfo) -{ - updateGammaTable(context); - - return 0; -} - -void Lut::updateGammaTable(IPAContext &context) -{ - auto &gammaTable = context.activeState.gamma.gammaTable; - const auto blackLevel = context.activeState.blc.level; - const unsigned int blackIndex = blackLevel * gammaTable.size() / 256; - const auto gamma = - 1.0 / context.activeState.knobs.gamma.value_or(kDefaultGamma); - const auto contrast = context.activeState.knobs.contrast.value_or(1.0); - - const float divisor = gammaTable.size() - blackIndex - 1.0; - for (unsigned int i = blackIndex; i < gammaTable.size(); i++) { - double normalized = (i - blackIndex) / divisor; - /* Convert 0..2 to 0..infinity; avoid actual inifinity at tan(pi/2) */ - double contrastExp = tan(std::clamp(contrast * M_PI_4, 0.0, M_PI_2 - 0.00001)); - /* Apply simple S-curve */ - if (normalized < 0.5) - normalized = 0.5 * std::pow(normalized / 0.5, contrastExp); - else - normalized = 1.0 - 0.5 * std::pow((1.0 - normalized) / 0.5, contrastExp); - gammaTable[i] = UINT8_MAX * - std::pow(normalized, gamma); - } - /* - * Due to CCM operations, the table lookup may reach indices below the black - * level. Let's set the table values below black level to the minimum - * non-black value to prevent problems when the minimum value is - * significantly non-zero (for example, when the image should be all grey). - */ - std::fill(gammaTable.begin(), gammaTable.begin() + blackIndex, - gammaTable[blackIndex]); - - context.activeState.gamma.gamma = gamma; - context.activeState.gamma.blackLevel = blackLevel; - context.activeState.gamma.contrast = contrast; -} - -int16_t Lut::matrixValue(unsigned int i, float ccm) const -{ - return std::round(i * ccm); -} - -void Lut::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - DebayerParams *params) -{ - /* - * Update the gamma table if needed. This means if black level changes - * and since the black level gets updated only if a lower value is - * observed, it's not permanently prone to minor fluctuations or - * rounding errors. - */ - const bool gammaUpdateNeeded = - context.activeState.gamma.blackLevel != context.activeState.blc.level || - context.activeState.gamma.contrast != context.activeState.knobs.contrast; - if (gammaUpdateNeeded) - updateGammaTable(context); - - auto &gains = context.activeState.awb.gains; - auto &gammaTable = context.activeState.gamma.gammaTable; - const unsigned int gammaTableSize = gammaTable.size(); - const double div = static_cast(DebayerParams::kRGBLookupSize) / - gammaTableSize; - - if (!context.ccmEnabled) { - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - /* Apply gamma after gain! */ - const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); - params->red[i] = gammaTable[static_cast(lutGains.r())]; - params->green[i] = gammaTable[static_cast(lutGains.g())]; - params->blue[i] = gammaTable[static_cast(lutGains.b())]; - } - } else if (context.activeState.matrixChanged || gammaUpdateNeeded) { - auto &matrix = context.activeState.combinedMatrix; - auto &red = params->redCcm; - auto &green = params->greenCcm; - auto &blue = params->blueCcm; - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - red[i].r = matrixValue(i, matrix[0][0]); - red[i].g = matrixValue(i, matrix[1][0]); - red[i].b = matrixValue(i, matrix[2][0]); - green[i].r = matrixValue(i, matrix[0][1]); - green[i].g = matrixValue(i, matrix[1][1]); - green[i].b = matrixValue(i, matrix[2][1]); - blue[i].r = matrixValue(i, matrix[0][2]); - blue[i].g = matrixValue(i, matrix[1][2]); - blue[i].b = matrixValue(i, matrix[2][2]); - params->gammaLut[i] = gammaTable[i / div]; - } - context.activeState.matrixChanged = false; - } -} - -REGISTER_IPA_ALGORITHM(Lut, "Lut") - -} /* namespace ipa::soft::algorithms */ - -} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lut.h b/src/ipa/simple/algorithms/lut.h deleted file mode 100644 index ad16d1e8e..000000000 --- a/src/ipa/simple/algorithms/lut.h +++ /dev/null @@ -1,35 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2024, Red Hat Inc. - * - * Color lookup tables construction - */ - -#pragma once - -#include "algorithm.h" - -namespace libcamera { - -namespace ipa::soft::algorithms { - -class Lut : public Algorithm -{ -public: - Lut() = default; - ~Lut() = default; - - int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, - const uint32_t frame, - IPAFrameContext &frameContext, - DebayerParams *params) override; - -private: - void updateGammaTable(IPAContext &context); - int16_t matrixValue(unsigned int i, float ccm) const; -}; - -} /* namespace ipa::soft::algorithms */ - -} /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/meson.build b/src/ipa/simple/algorithms/meson.build index ebe9f20dd..73c637220 100644 --- a/src/ipa/simple/algorithms/meson.build +++ b/src/ipa/simple/algorithms/meson.build @@ -6,5 +6,4 @@ soft_simple_ipa_algorithms = files([ 'agc.cpp', 'blc.cpp', 'ccm.cpp', - 'lut.cpp', ]) diff --git a/src/ipa/simple/data/uncalibrated.yaml b/src/ipa/simple/data/uncalibrated.yaml index f0410fe61..fc90ca526 100644 --- a/src/ipa/simple/data/uncalibrated.yaml +++ b/src/ipa/simple/data/uncalibrated.yaml @@ -15,6 +15,5 @@ algorithms: # 0, 1, 0, # 0, 0, 1] - Adjust: - - Lut: - Agc: ... diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index fb063b091..c59e684f8 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -53,17 +53,8 @@ struct IPAActiveState { unsigned int temperatureK; } awb; - static constexpr unsigned int kGammaLookupSize = 1024; - struct { - std::array gammaTable; - uint8_t blackLevel; - float gamma; - float contrast; - } gamma; - Matrix ccm; Matrix combinedMatrix; - bool matrixChanged = false; struct { std::optional gamma; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 98582587c..574b972e5 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -26,6 +26,7 @@ #include "libcamera/internal/software_isp/swisp_stats.h" #include "libcamera/internal/yaml_parser.h" +#include "algorithms/adjust.h" #include "libipa/camera_sensor_helper.h" #include "module.h" @@ -158,6 +159,11 @@ int IPASoftSimple::init(const IPASettings &settings, } params_ = static_cast(mem); + params_->blackLevel = 0; + params_->gamma = algorithms::kDefaultGamma; + params_->contrast = algorithms::kDefaultContrast; + params_->gains = { { 1.0, 1.0, 1.0 } }; + /* combinedMatrix is reset for each frame. */ } { @@ -296,7 +302,9 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro void IPASoftSimple::computeParams(const uint32_t frame) { - context_.activeState.combinedMatrix = Matrix::identity(); + Matrix combinedMatrix = Matrix::identity(); + context_.activeState.combinedMatrix = combinedMatrix; + params_->combinedMatrix = combinedMatrix; IPAFrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index e9e18c488..298875384 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -19,79 +19,28 @@ namespace libcamera { */ /** - * \var DebayerParams::kRGBLookupSize - * \brief Size of a color lookup table + * \var DebayerParams::blackLevel + * \brief 8-bit black level value */ /** - * \struct DebayerParams::CcmColumn - * \brief Type of a single column of a color correction matrix (CCM) - * - * When multiplying an input pixel, columns in the CCM correspond to the red, - * green or blue component of input pixel values, while rows correspond to the - * red, green or blue components of the output pixel values. The members of the - * CcmColumn structure are named after the colour components of the output pixel - * values they correspond to. - */ - -/** - * \var DebayerParams::CcmColumn::r - * \brief Red (first) component of a CCM column - */ - -/** - * \var DebayerParams::CcmColumn::g - * \brief Green (second) component of a CCM column - */ - -/** - * \var DebayerParams::CcmColumn::b - * \brief Blue (third) component of a CCM column - */ - -/** - * \typedef DebayerParams::LookupTable - * \brief Type of the lookup tables for single lookup values - */ - -/** - * \typedef DebayerParams::CcmLookupTable - * \brief Type of the CCM lookup tables for red, green, blue values - */ - -/** - * \var DebayerParams::red - * \brief Lookup table for red color, mapping input values to output values - */ - -/** - * \var DebayerParams::green - * \brief Lookup table for green color, mapping input values to output values - */ - -/** - * \var DebayerParams::blue - * \brief Lookup table for blue color, mapping input values to output values - */ - -/** - * \var DebayerParams::redCcm - * \brief Lookup table for the CCM red column, mapping input values to output values + * \var DebayerParams::gamma + * \brief Gamma value */ /** - * \var DebayerParams::greenCcm - * \brief Lookup table for the CCM green column, mapping input values to output values + * \var DebayerParams::contrast + * \brief Contrast to be applied */ /** - * \var DebayerParams::blueCcm - * \brief Lookup table for the CCM blue column, mapping input values to output values + * \var DebayerParams::gains + * \brief Colour channel gains */ /** - * \var DebayerParams::gammaLut - * \brief Gamma lookup table used with color correction matrix + * \var DebayerParams::combinedMatrix + * \brief Colour correction matrix, including other adjustments */ /** diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 67a303c65..a0e845f67 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -64,14 +64,6 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, const GlobalConfigurat framesToMeasure_ = configuration.option( { "software_isp", "measure", "number" }) .value_or(framesToMeasure_); - - /* Initialize color lookup tables */ - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - red_[i] = green_[i] = blue_[i] = i; - redCcm_[i] = { static_cast(i), 0, 0 }; - greenCcm_[i] = { 0, static_cast(i), 0 }; - blueCcm_[i] = { 0, 0, static_cast(i) }; - } } DebayerCpu::~DebayerCpu() = default; @@ -84,21 +76,21 @@ DebayerCpu::~DebayerCpu() = default; #define GAMMA(value) \ *dst++ = gammaLut_[std::clamp(value, 0, static_cast(gammaLut_.size()) - 1)] -#define STORE_PIXEL(b_, g_, r_) \ - if constexpr (ccmEnabled) { \ - const DebayerParams::CcmColumn &blue = blueCcm_[b_]; \ - const DebayerParams::CcmColumn &green = greenCcm_[g_]; \ - const DebayerParams::CcmColumn &red = redCcm_[r_]; \ - GAMMA(blue.b + green.b + red.b); \ - GAMMA(blue.g + green.g + red.g); \ - GAMMA(blue.r + green.r + red.r); \ - } else { \ - *dst++ = blue_[b_]; \ - *dst++ = green_[g_]; \ - *dst++ = red_[r_]; \ - } \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define STORE_PIXEL(b_, g_, r_) \ + if constexpr (ccmEnabled) { \ + const CcmColumn &blue = blueCcm_[b_]; \ + const CcmColumn &green = greenCcm_[g_]; \ + const CcmColumn &red = redCcm_[r_]; \ + GAMMA(blue.b + green.b + red.b); \ + GAMMA(blue.g + green.g + red.g); \ + GAMMA(blue.r + green.r + red.r); \ + } else { \ + *dst++ = blue_[b_]; \ + *dst++ = green_[g_]; \ + *dst++ = red_[r_]; \ + } \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; /* @@ -547,6 +539,16 @@ int DebayerCpu::configure(const StreamConfiguration &inputCfg, if (ret != 0) return -EINVAL; + ccmEnabled_ = ccmEnabled; + + /* + * Lookup tables must be initialized because the initial value is used for + * the first two frames, i.e. until stats processing starts providing its + * own parameters. Let's enforce recomputing lookup tables by setting the + * stored last used gamma to an out-of-range value. + */ + params_.gamma = 0.0; + window_.x = ((inputCfg.size.width - outputCfg.size.width) / 2) & ~(inputConfig_.patternSize.width - 1); window_.y = ((inputCfg.size.height - outputCfg.size.height) / 2) & @@ -775,6 +777,103 @@ inline int64_t timeDiff(timespec &after, timespec &before) } /* namespace */ +void DebayerCpu::updateGammaTable(DebayerParams ¶ms) +{ + const uint8_t blackLevel = params.blackLevel; + const unsigned int blackIndex = blackLevel * gammaTable_.size() / 256; + const double gamma = 1.0 / params.gamma; + const double contrast = params.contrast; + + const float divisor = gammaTable_.size() - blackIndex - 1.0; + for (unsigned int i = blackIndex; i < gammaTable_.size(); i++) { + double normalized = (i - blackIndex) / divisor; + /* Convert 0..2 to 0..infinity; avoid actual inifinity at tan(pi/2) */ + double contrastExp = tan(std::clamp(contrast * M_PI_4, 0.0, M_PI_2 - 0.00001)); + /* Apply simple S-curve */ + if (normalized < 0.5) + normalized = 0.5 * std::pow(normalized / 0.5, contrastExp); + else + normalized = 1.0 - 0.5 * std::pow((1.0 - normalized) / 0.5, contrastExp); + gammaTable_[i] = UINT8_MAX * + std::pow(normalized, gamma); + } + /* + * Due to CCM operations, the table lookup may reach indices below the black + * level. Let's set the table values below black level to the minimum + * non-black value to prevent problems when the minimum value is + * significantly non-zero (for example, when the image should be all grey). + */ + std::fill(gammaTable_.begin(), gammaTable_.begin() + blackIndex, + gammaTable_[blackIndex]); +} + +void DebayerCpu::updateLookupTables(DebayerParams ¶ms) +{ + const bool gammaUpdateNeeded = + params.gamma != params_.gamma || + params.blackLevel != params_.blackLevel || + params.contrast != params_.contrast; + if (gammaUpdateNeeded) + updateGammaTable(params); + + auto matrixChanged = [](Matrix m1, Matrix m2) -> bool { + for (unsigned int i = 0; i < 3; i++) + for (unsigned int j = 0; j < 3; j++) + if (m1[i][j] != m2[i][j]) + return true; + return false; + }; + + const unsigned int gammaTableSize = gammaTable_.size(); + const double div = static_cast(kRGBLookupSize) / gammaTableSize; + if (ccmEnabled_) { + if (gammaUpdateNeeded || + matrixChanged(params.combinedMatrix, params_.combinedMatrix)) { + auto &red = swapRedBlueGains_ ? blueCcm_ : redCcm_; + auto &green = greenCcm_; + auto &blue = swapRedBlueGains_ ? redCcm_ : blueCcm_; + const unsigned int redIndex = swapRedBlueGains_ ? 2 : 0; + const unsigned int greenIndex = 1; + const unsigned int blueIndex = swapRedBlueGains_ ? 0 : 2; + for (unsigned int i = 0; i < kRGBLookupSize; i++) { + red[i].r = std::round(i * params.combinedMatrix[redIndex][0]); + red[i].g = std::round(i * params.combinedMatrix[greenIndex][0]); + red[i].b = std::round(i * params.combinedMatrix[blueIndex][0]); + green[i].r = std::round(i * params.combinedMatrix[redIndex][1]); + green[i].g = std::round(i * params.combinedMatrix[greenIndex][1]); + green[i].b = std::round(i * params.combinedMatrix[blueIndex][1]); + blue[i].r = std::round(i * params.combinedMatrix[redIndex][2]); + blue[i].g = std::round(i * params.combinedMatrix[greenIndex][2]); + blue[i].b = std::round(i * params.combinedMatrix[blueIndex][2]); + gammaLut_[i] = gammaTable_[i / div]; + } + } + } else { + if (gammaUpdateNeeded || params.gains != params_.gains) { + auto &gains = params.gains; + auto &red = swapRedBlueGains_ ? blue_ : red_; + auto &green = green_; + auto &blue = swapRedBlueGains_ ? red_ : blue_; + for (unsigned int i = 0; i < kRGBLookupSize; i++) { + /* Apply gamma after gain! */ + const RGB lutGains = (gains * i / div).min(gammaTableSize - 1); + red[i] = gammaTable_[static_cast(lutGains.r())]; + green[i] = gammaTable_[static_cast(lutGains.g())]; + blue[i] = gammaTable_[static_cast(lutGains.b())]; + } + } + } + + LOG(Debayer, Debug) + << "Debayer parameters: blackLevel=" << static_cast(params.blackLevel) + << "; gamma=" << params.gamma + << "; contrast=" << params.contrast + << "; gains=" << params.gains + << "; matrix=" << params.combinedMatrix; + + params_ = params; +} + void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) { timespec frameStartTime; @@ -794,25 +893,7 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output for (const FrameBuffer::Plane &plane : output->planes()) dmaSyncers.emplace_back(plane.fd, DmaSyncer::SyncType::Write); - green_ = params.green; - greenCcm_ = params.greenCcm; - if (swapRedBlueGains_) { - red_ = params.blue; - blue_ = params.red; - redCcm_ = params.blueCcm; - blueCcm_ = params.redCcm; - for (unsigned int i = 0; i < 256; i++) { - std::swap(redCcm_[i].r, redCcm_[i].b); - std::swap(greenCcm_[i].r, greenCcm_[i].b); - std::swap(blueCcm_[i].r, blueCcm_[i].b); - } - } else { - red_ = params.red; - blue_ = params.blue; - redCcm_ = params.redCcm; - blueCcm_ = params.blueCcm; - } - gammaLut_ = params.gammaLut; + updateLookupTables(params); /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 03e0d7843..941f08b7b 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -19,6 +19,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/global_configuration.h" +#include "libcamera/internal/software_isp/debayer_params.h" #include "debayer.h" #include "swstats_cpu.h" @@ -135,17 +136,32 @@ private: void memcpyNextLine(const uint8_t *linePointers[]); void process2(uint32_t frame, const uint8_t *src, uint8_t *dst); void process4(uint32_t frame, const uint8_t *src, uint8_t *dst); + void updateGammaTable(DebayerParams ¶ms); + void updateLookupTables(DebayerParams ¶ms); /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - DebayerParams::LookupTable red_; - DebayerParams::LookupTable green_; - DebayerParams::LookupTable blue_; - DebayerParams::CcmLookupTable redCcm_; - DebayerParams::CcmLookupTable greenCcm_; - DebayerParams::CcmLookupTable blueCcm_; - DebayerParams::LookupTable gammaLut_; + static constexpr unsigned int kRGBLookupSize = 256; + static constexpr unsigned int kGammaLookupSize = 1024; + struct CcmColumn { + int16_t r; + int16_t g; + int16_t b; + }; + using LookupTable = std::array; + using CcmLookupTable = std::array; + LookupTable red_; + LookupTable green_; + LookupTable blue_; + CcmLookupTable redCcm_; + CcmLookupTable greenCcm_; + CcmLookupTable blueCcm_; + std::array gammaTable_; + LookupTable gammaLut_; + bool ccmEnabled_; + DebayerParams params_; + debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 341c0352c..ad40471d3 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -80,23 +80,6 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) { - /* - * debayerParams_ must be initialized because the initial value is used for - * the first two frames, i.e. until stats processing starts providing its - * own parameters. - * - * \todo This should be handled in the same place as the related - * operations, in the IPA module. - */ - std::array gammaTable; - for (unsigned int i = 0; i < 256; i++) - gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - debayerParams_.red[i] = gammaTable[i]; - debayerParams_.green[i] = gammaTable[i]; - debayerParams_.blue[i] = gammaTable[i]; - } - if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaBufAllocator object"; return;