From patchwork Tue May 12 12:36:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 26733 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7CB18BDB1C for ; Tue, 12 May 2026 12:37:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E13E63039; Tue, 12 May 2026 14:37:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="XPIlyzmz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4C75463039 for ; Tue, 12 May 2026 14:37:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778589424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dy9SKuAzEW9xA5akmA/eiuI7dLFSDoVPtineW4JKjb0=; b=XPIlyzmzwKVjUM4XOxDxvCHRNY59cr1qo8Km4DWdybNzqhPApeDwnBaYI6qTrAns//wDdj YYQ0WfRwg/vFoprA31306B18VgEnEPAa8Y0k4lGq9Gyg8FEdFZmaqO7RXXPUV29jLk58jk 7iyNt5C1t7OPFJPKPIUJHB+D6hF0BpM= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-9-fcP3_nY7NTi4pjLs8V223Q-1; Tue, 12 May 2026 08:37:00 -0400 X-MC-Unique: fcP3_nY7NTi4pjLs8V223Q-1 X-Mimecast-MFC-AGG-ID: fcP3_nY7NTi4pjLs8V223Q_1778589419 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 855981956095; Tue, 12 May 2026 12:36:59 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.50.44]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BBB4B1955D84; Tue, 12 May 2026 12:36:57 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Bryan O'Donoghue , Hans de Goede , Laurent Pinchart Subject: [RFC PATCH v5 10/10] libcamera: software_isp: Add polynomial LSC Date: Tue, 12 May 2026 14:36:14 +0200 Message-ID: <20260512123619.120068-11-mzamazal@redhat.com> In-Reply-To: <20260512123619.120068-1-mzamazal@redhat.com> References: <20260512123619.120068-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 6at_ebdfqeqeXSjiFvdquvdX6yuUjoMgrFAzh07YtnE_1778589419 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In addition to the already implemented table based lens shading correction, let's implement polynomial lens shading correction. The primary differences between the two are: - Table based correction is based on a 16x16 grid of values, while polynomial correction uses 3 coefficients for polynomial computation of the scaling factor. - The polynomial correction implemented here is faster (at least in my environment). - Table based correction allows specifying non-symmetric corrections. This patch implements just the simplest form of the polynomial correction, for ease of implementation and speed. It assumes that lens shading is centred and uses only 3-item polynomial: lsc0 + lsc1 * dist^2 + lsc2 * dist^4, where dist is the distance from the centre. The distance is measured in kilopixels, which is definitely debatable, but it makes easy to deal with different vertical and horizontal sizes and with cropping, assuming the pixels have the same horizontal and vertical sizes. Shifted crops are not considered but can be added easily in another patch by specifying the shift as another uniform. The coefficients are specified for each of the RGB colours. The YAML definition looks like: - Lsc: type: polynomial sets: - ct: temperature-1 r: [ lsc0, lsc1, lsc2 ] g: [ ... ] b: [ ... ] - ct: temperature-2 r: [ ... ] g: [ ... ] b: [ ... ] - ... Signed-off-by: Milan Zamazal --- .../internal/software_isp/debayer_params.h | 4 ++ src/ipa/simple/algorithms/lsc.cpp | 69 +++++++++++++------ src/ipa/simple/algorithms/lsc.h | 4 ++ src/libcamera/shaders/bayer_1x_packed.frag | 9 +++ src/libcamera/shaders/bayer_unpacked.frag | 9 +++ src/libcamera/software_isp/debayer.cpp | 15 ++++ src/libcamera/software_isp/debayer_egl.cpp | 28 +++++++- src/libcamera/software_isp/debayer_egl.h | 4 ++ 8 files changed, 121 insertions(+), 21 deletions(-) diff --git a/include/libcamera/internal/software_isp/debayer_params.h b/include/libcamera/internal/software_isp/debayer_params.h index 37fff49a2..bef6820cb 100644 --- a/include/libcamera/internal/software_isp/debayer_params.h +++ b/include/libcamera/internal/software_isp/debayer_params.h @@ -37,8 +37,12 @@ struct DebayerParams { enum LscType : uint32_t { LscNone, LscTable, + LscPolynomial, }; LscLookupTable lscLut{}; + + static constexpr unsigned int kNLscCoefficients = 3; + std::array, kNLscCoefficients> lscCoefficients; }; } /* namespace libcamera */ diff --git a/src/ipa/simple/algorithms/lsc.cpp b/src/ipa/simple/algorithms/lsc.cpp index ae5179cfc..0470b5e76 100644 --- a/src/ipa/simple/algorithms/lsc.cpp +++ b/src/ipa/simple/algorithms/lsc.cpp @@ -18,23 +18,27 @@ LOG_DEFINE_CATEGORY(IPASoftLsc) int Lsc::init(IPAContext &context, const ValueNode &tuningData) { std::string type = tuningData["type"].get("table"); + int retR, retG, retB; if (type == "table") { - int retR = lscR_.readYaml(tuningData["sets"], "ct", "r"); - int retG = lscG_.readYaml(tuningData["sets"], "ct", "g"); - int retB = lscB_.readYaml(tuningData["sets"], "ct", "b"); - - if (retR < 0 || retG < 0 || retB < 0) { - LOG(IPASoftLsc, Error) - << "Failed to parse 'lsc' parameter from tuning file."; - return -EINVAL; - } - + retR = lscR_.readYaml(tuningData["sets"], "ct", "r"); + retG = lscG_.readYaml(tuningData["sets"], "ct", "g"); + retB = lscB_.readYaml(tuningData["sets"], "ct", "b"); type_ = DebayerParams::LscTable; + } else if (type == "polynomial") { + retR = lscCoefR_.readYaml(tuningData["sets"], "ct", "r"); + retG = lscCoefG_.readYaml(tuningData["sets"], "ct", "g"); + retB = lscCoefB_.readYaml(tuningData["sets"], "ct", "b"); + type_ = DebayerParams::LscPolynomial; } else { LOG(IPASoftLsc, Error) << "LSC: type " << type << " not supported"; return -EINVAL; } + if (retR < 0 || retG < 0 || retB < 0) { + LOG(IPASoftLsc, Error) + << "Failed to parse 'lsc' parameter from tuning file."; + return -EINVAL; + } context.lscType = type_; @@ -53,18 +57,43 @@ void Lsc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, unsigned int ct = context.activeState.awb.temperatureK.value_or(kDefaultTemperature); - const LscMatrix matrixR = lscR_.getInterpolated(ct); - const LscMatrix matrixG = lscG_.getInterpolated(ct); - const LscMatrix matrixB = lscB_.getInterpolated(ct); + switch (type_) { + case DebayerParams::LscNone: + break; + + case DebayerParams::LscTable: { + const LscMatrix matrixR = lscR_.getInterpolated(ct); + const LscMatrix matrixG = lscG_.getInterpolated(ct); + const LscMatrix matrixB = lscB_.getInterpolated(ct); + + DebayerParams::LscLookupTable lut; + constexpr unsigned int gridSize = DebayerParams::kLscGridSize; + for (unsigned int i = 0, j = 0; i < gridSize * gridSize; i++) { + lut[j++] = matrixR.data()[i]; + lut[j++] = matrixG.data()[i]; + lut[j++] = matrixB.data()[i]; + } + params->lscLut = lut; - DebayerParams::LscLookupTable lut; - constexpr unsigned int gridSize = DebayerParams::kLscGridSize; - for (unsigned int i = 0, j = 0; i < gridSize * gridSize; i++) { - lut[j++] = matrixR.data()[i]; - lut[j++] = matrixG.data()[i]; - lut[j++] = matrixB.data()[i]; + break; + } + + case DebayerParams::LscPolynomial: { + const Vector coefR = + lscCoefR_.getInterpolated(ct); + const Vector coefG = + lscCoefG_.getInterpolated(ct); + const Vector coefB = + lscCoefB_.getInterpolated(ct); + + for (unsigned int i = 0; i < DebayerParams::kNLscCoefficients; i++) { + params->lscCoefficients[i].r() = coefR[i]; + params->lscCoefficients[i].g() = coefG[i]; + params->lscCoefficients[i].b() = coefB[i]; + } + break; + } } - params->lscLut = lut; } REGISTER_IPA_ALGORITHM(Lsc, "Lsc") diff --git a/src/ipa/simple/algorithms/lsc.h b/src/ipa/simple/algorithms/lsc.h index d7d7c9559..e1981dec7 100644 --- a/src/ipa/simple/algorithms/lsc.h +++ b/src/ipa/simple/algorithms/lsc.h @@ -6,6 +6,7 @@ #pragma once #include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" #include @@ -35,6 +36,9 @@ private: Interpolator lscR_; Interpolator lscG_; Interpolator lscB_; + Interpolator> lscCoefR_; + Interpolator> lscCoefG_; + Interpolator> lscCoefB_; }; } /* namespace ipa::soft::algorithms */ diff --git a/src/libcamera/shaders/bayer_1x_packed.frag b/src/libcamera/shaders/bayer_1x_packed.frag index dc2b582f8..bc45c1993 100644 --- a/src/libcamera/shaders/bayer_1x_packed.frag +++ b/src/libcamera/shaders/bayer_1x_packed.frag @@ -72,6 +72,11 @@ uniform float contrastExp; #if defined(APPLY_LSC_TABLE) uniform sampler2D lsc_tex; +#elif defined(APPLY_LSC_POLYNOMIAL) +uniform vec2 lscScale; +uniform vec3 lsc0; +uniform vec3 lsc1; +uniform vec3 lsc2; #endif float apply_contrast(float value) @@ -233,6 +238,10 @@ void main(void) #if defined(APPLY_LSC_TABLE) rgb = rgb * texture2D(lsc_tex, textureOut).rgb; +#elif defined(APPLY_LSC_POLYNOMIAL) + vec2 offCenter = (textureOut - vec2(0.5, 0.5)) * lscScale; + float dist2 = dot(offCenter, offCenter); + rgb = rgb * (lsc0 + lsc1 * dist2 + lsc2 * dist2 * dist2); #endif /* diff --git a/src/libcamera/shaders/bayer_unpacked.frag b/src/libcamera/shaders/bayer_unpacked.frag index df324e6e1..5a1d82325 100644 --- a/src/libcamera/shaders/bayer_unpacked.frag +++ b/src/libcamera/shaders/bayer_unpacked.frag @@ -31,6 +31,11 @@ uniform float contrastExp; #if defined(APPLY_LSC_TABLE) uniform sampler2D lsc_tex; +#elif defined(APPLY_LSC_POLYNOMIAL) +uniform vec2 lscScale; +uniform vec3 lsc0; +uniform vec3 lsc1; +uniform vec3 lsc2; #endif float apply_contrast(float value) @@ -136,6 +141,10 @@ void main(void) { #if defined(APPLY_LSC_TABLE) rgb = rgb * texture2D(lsc_tex, center.xy).rgb; +#elif defined(APPLY_LSC_POLYNOMIAL) + vec2 offCenter = (center.xy - vec2(0.5, 0.5)) * lscScale; + float dist2 = dot(offCenter, offCenter); + rgb = rgb * (lsc0 + lsc1 * dist2 + lsc2 * dist2 * dist2); #endif /* diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index a77b3c9fc..ec0c618ba 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -63,6 +63,11 @@ namespace libcamera { * \brief Lens shading correction using a lookup table */ +/** + * \var DebayerParams::LscPolynomial + * \brief Lens shading correction using polynomial coefficients + */ + /** * \typedef DebayerParams::LscValueType * \brief Type of LSC grid values @@ -86,6 +91,16 @@ namespace libcamera { * \brief Lens shading lookup table */ +/** + * \var DebayerParams::kNLscCoefficients + * \brief Number of the lens shading correction polynomial coefficients + */ + +/** + * \var DebayerParams::lscCoefficients + * \brief Polynomial coefficients for lens shading correction, one per colour channel + */ + /** * \class Debayer * \brief Base debayering class diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 03ec5c5da..560a87d66 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -116,6 +116,10 @@ int DebayerEGL::getShaderVariableLocations(void) textureUniformProjMatrix_ = glGetUniformLocation(programId_, "proj_matrix"); textureUniformLsc_ = glGetUniformLocation(programId_, "lsc_tex"); + lscScale_ = glGetUniformLocation(programId_, "lscScale"); + lsc0_ = glGetUniformLocation(programId_, "lsc0"); + lsc1_ = glGetUniformLocation(programId_, "lsc1"); + lsc2_ = glGetUniformLocation(programId_, "lsc2"); LOG(Debayer, Debug) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ << " tex_y " << textureUniformBayerDataIn_ @@ -128,7 +132,11 @@ int DebayerEGL::getShaderVariableLocations(void) << " stride_factor " << textureUniformStrideFactor_ << " tex_bayer_first_red " << textureUniformBayerFirstRed_ << " proj_matrix " << textureUniformProjMatrix_ - << " lsc " << textureUniformLsc_; + << " lscTexture " << textureUniformLsc_ + << " lscScale " << lscScale_ + << " lsc0 " << lsc0_ + << " lsc1 " << lsc1_ + << " lsc2 " << lsc2_; return 0; } @@ -153,6 +161,9 @@ int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputForm case DebayerParams::LscTable: egl_.pushEnv(shaderEnv, "#define APPLY_LSC_TABLE"); break; + case DebayerParams::LscPolynomial: + egl_.pushEnv(shaderEnv, "#define APPLY_LSC_POLYNOMIAL"); + break; } /* @@ -513,6 +524,21 @@ void DebayerEGL::setShaderVariableValues(const DebayerParams ¶ms) params.lscLut.data(), GL_LINEAR); glUniform1i(textureUniformLsc_, eglImageLscLookup_->texture_unit_uniform_id_); break; + case DebayerParams::LscPolynomial: + glUniform2f(lscScale_, imgSize[0] / 1000.0, imgSize[1] / 1000.0); + glUniform3f(lsc0_, + params.lscCoefficients[0].r(), + params.lscCoefficients[0].g(), + params.lscCoefficients[0].b()); + glUniform3f(lsc1_, + params.lscCoefficients[1].r(), + params.lscCoefficients[1].g(), + params.lscCoefficients[1].b()); + glUniform3f(lsc2_, + params.lscCoefficients[2].r(), + params.lscCoefficients[2].g(), + params.lscCoefficients[2].b()); + break; } /* diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index d5cfcad1e..7d1fa31cb 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -95,6 +95,10 @@ private: GLint textureUniformBayerDataIn_; GLint textureUniformLsc_; + GLint lscScale_; + GLint lsc0_; + GLint lsc1_; + GLint lsc2_; /* Represent per-frame CCM as a uniform vector of floats 3 x 3 */ GLint ccmUniformDataIn_;