From patchwork Sat Jun 20 23:00:33 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 27002 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7211BC330A for ; Sat, 20 Jun 2026 23:00:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2076A656E2; Sun, 21 Jun 2026 01:00:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WAlsjo1L"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C58C762899 for ; Sun, 21 Jun 2026 01:00:36 +0200 (CEST) Received: from [192.168.0.240] (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1113018A2; Sun, 21 Jun 2026 01:00:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781996400; bh=Ibjo4eaudDdOYgNiLtnzyC7qbdq6Mx1seSA2HTgq6CQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=WAlsjo1LTsg/Y+TxpG48P9inSzpo8/2PtCLQtit8CXqXBOv1ZfUV8v035CvbabkBC x7NU9a6EKdnujVnNMR3SNB8eUKdCfzdJtGFDNusaYkuu7PLerHvjFyKuxA92n+Em1N hGkt2dCyvShG/q0FQ+X8syhynETCSQ/6MqGH+DV4= From: Kieran Bingham Date: Sun, 21 Jun 2026 00:00:33 +0100 Subject: [PATCH 6/7] softisp: Split AWB from Combined Matrix MIME-Version: 1.0 Message-Id: <20260621-kbingham-awb-saturation-v1-6-b91ea59c6cfb@ideasonboard.com> References: <20260621-kbingham-awb-saturation-v1-0-b91ea59c6cfb@ideasonboard.com> In-Reply-To: <20260621-kbingham-awb-saturation-v1-0-b91ea59c6cfb@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Milan Zamazal X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1781996434; l=8995; i=kieran.bingham@ideasonboard.com; s=20260204; h=from:subject:message-id; bh=Ibjo4eaudDdOYgNiLtnzyC7qbdq6Mx1seSA2HTgq6CQ=; b=zU/62TsF+OPfE4Rbl/iSEY9v+TlOtzKto93HoUQy95EqYuC6/d8ZPLYmXF/7gwNrVRxHHQZmu yyM60/8VskSAyyUlNUMJ1r8O8e8P16MHGg1KgvInZTtO3wHkhAvEjfV X-Developer-Key: i=kieran.bingham@ideasonboard.com; a=ed25519; pk=IOxS2C6nWHNjLfkDR71Iesk904i6wJDfEERqV7hDBdY= X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB and CCM are currently combined into a single matrix which gets passed to the GPU and CPU ISP implementations. This unfortunately can cause saturation bugs between the AWB and CCM in areas of high brightness in a given scene, where one colour channel will saturate before the others unevenly. Remove AWB gains from the combined Matrix to allow a specific AWB Gain stage to be applied, which is then clamped equally on all channels. For CPU ISP the gains are applied and clamped before applying the CCM. Reviewed-by: Milan Zamazal Signed-off-by: Milan Zamazal Signed-off-by: Kieran Bingham --- src/ipa/simple/algorithms/awb.cpp | 16 +++------------- src/ipa/simple/ipa_context.h | 5 +---- src/libcamera/shaders/bayer_1x_packed.frag | 4 ++++ src/libcamera/shaders/bayer_unpacked.frag | 4 ++++ src/libcamera/software_isp/debayer_cpu.cpp | 10 +++++----- src/libcamera/software_isp/debayer_egl.cpp | 5 +++++ src/libcamera/software_isp/debayer_egl.h | 3 +++ 7 files changed, 25 insertions(+), 22 deletions(-) diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index f5c88ea6f896ff8a7ffdc328e09a4d4aa99df5e4..05155c83d172d64609053ba940a4c12a2248bb04 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -38,15 +38,8 @@ void Awb::prepare(IPAContext &context, DebayerParams *params) { auto &gains = context.activeState.awb.gains; - Matrix gainMatrix = { { gains.r(), 0, 0, - 0, gains.g(), 0, - 0, 0, gains.b() } }; - context.activeState.combinedMatrix = - gainMatrix * context.activeState.combinedMatrix; - - frameContext.gains.red = gains.r(); - frameContext.gains.blue = gains.b(); + frameContext.gains = gains; params->gains = gains; } @@ -59,11 +52,8 @@ void Awb::process(IPAContext &context, const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; - const float mdGains[] = { - static_cast(frameContext.gains.red), - static_cast(frameContext.gains.blue) - }; - metadata.set(controls::ColourGains, mdGains); + metadata.set(controls::ColourGains, { frameContext.gains.r(), + frameContext.gains.b() }); if (!stats->valid) return; diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index 34f7403a41d690cb3f0c271827ae2e915b6ea49d..8ccfacb46a59cedb5a0ad051d67f7c1f40af4b52 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -71,10 +71,7 @@ struct IPAFrameContext : public FrameContext { double gain; } sensor; - struct { - double red; - double blue; - } gains; + RGB gains; float gamma; std::optional contrast; diff --git a/src/libcamera/shaders/bayer_1x_packed.frag b/src/libcamera/shaders/bayer_1x_packed.frag index fbd15b32e5b3b510ba0dd5a75704f9ff3942e00c..6b3f7532c177f277e43c27e498dfadcb9cd4e26c 100644 --- a/src/libcamera/shaders/bayer_1x_packed.frag +++ b/src/libcamera/shaders/bayer_1x_packed.frag @@ -65,6 +65,7 @@ uniform vec2 tex_step; uniform vec2 tex_bayer_first_red; uniform sampler2D tex_y; +uniform vec3 awb; uniform mat3 ccm; uniform vec3 blacklevel; uniform float gamma; @@ -231,6 +232,9 @@ void main(void) */ rgb = (rgb - blacklevel) / (1.0 - blacklevel); + /* Apply AWB gains, and saturate each channel at sensor range */ + rgb = clamp(rgb * awb, vec3(0.0), vec3(1.0)); + /* * CCM is a 3x3 in the format * diff --git a/src/libcamera/shaders/bayer_unpacked.frag b/src/libcamera/shaders/bayer_unpacked.frag index 0f85e9a4ded7f43ce4fdabf1e045275ae2bc8f53..44535312125d0e7e5cca372f4e8e0f8ad9fce8b3 100644 --- a/src/libcamera/shaders/bayer_unpacked.frag +++ b/src/libcamera/shaders/bayer_unpacked.frag @@ -24,6 +24,7 @@ uniform sampler2D tex_y; varying vec4 center; varying vec4 yCoord; varying vec4 xCoord; +uniform vec3 awb; uniform mat3 ccm; uniform vec3 blacklevel; uniform float gamma; @@ -134,6 +135,9 @@ void main(void) { */ rgb = (rgb - blacklevel) / (1.0 - blacklevel); + /* Apply AWB gains, and saturate each channel at sensor range */ + rgb = clamp(rgb * awb, vec3(0.0), vec3(1.0)); + /* * CCM is a 3x3 in the format * diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 9d6a08b3333f988570cb5d3f7cea4c117f7c2530..49382b4c2719bf02f6a806c9a9ac560c8daae866 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -1011,20 +1011,22 @@ void DebayerCpu::updateLookupTables(const DebayerParams ¶ms) const unsigned int gammaTableSize = gammaTable_.size(); const RGB blackIndex = params.blackLevel * kRGBLookupSize; + const RGB gains = params.gains; const RGB div = (RGB(kRGBLookupSize) - blackIndex).max(1.0); if (ccmEnabled_) { if (gammaUpdateNeeded || - matrixChanged(params.combinedMatrix, params_.combinedMatrix)) { + matrixChanged(params.combinedMatrix, params_.combinedMatrix) || + params.gains != params_.gains) { auto &red = swapRedBlueGains_ ? blueCcm_ : redCcm_; auto &green = greenCcm_; auto &blue = swapRedBlueGains_ ? redCcm_ : blueCcm_; const unsigned int redIndex = swapRedBlueGains_ ? 2 : 0; const unsigned int greenIndex = 1; const unsigned int blueIndex = swapRedBlueGains_ ? 0 : 2; - for (unsigned int i = 0; i < kRGBLookupSize; i++) { - const RGB rgb = ((RGB(i) - blackIndex) * kRGBLookupSize / div).max(0.0); + const RGB rgb = (gains * (RGB(i) - blackIndex) * kRGBLookupSize / div) + .clamp(0.0, kRGBLookupSize - 1); red[i].r = std::round(rgb.r() * params.combinedMatrix[redIndex][0]); red[i].g = std::round(rgb.r() * params.combinedMatrix[greenIndex][0]); red[i].b = std::round(rgb.r() * params.combinedMatrix[blueIndex][0]); @@ -1039,11 +1041,9 @@ void DebayerCpu::updateLookupTables(const DebayerParams ¶ms) } } else { if (gammaUpdateNeeded || params.gains != params_.gains) { - auto &gains = params.gains; auto &red = swapRedBlueGains_ ? blue_ : red_; auto &green = green_; auto &blue = swapRedBlueGains_ ? red_ : blue_; - for (unsigned int i = 0; i < kRGBLookupSize; i++) { const RGB lutGains = (gains * (RGB(i) - blackIndex) * gammaTableSize / div) diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 5213166578b6aadc26078443181b96a2c59fa9d5..6ba197966d85e1ab3bdc337b9e4ddeb20f1ec2fa 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -115,6 +115,7 @@ int DebayerEGL::getShaderVariableLocations(void) attributeTexture_ = glGetAttribLocation(programId_, "textureIn"); textureUniformBayerDataIn_ = glGetUniformLocation(programId_, "tex_y"); + awbUniformDataIn_ = glGetUniformLocation(programId_, "awb"); ccmUniformDataIn_ = glGetUniformLocation(programId_, "ccm"); blackLevelUniformDataIn_ = glGetUniformLocation(programId_, "blacklevel"); gammaUniformDataIn_ = glGetUniformLocation(programId_, "gamma"); @@ -128,6 +129,7 @@ int DebayerEGL::getShaderVariableLocations(void) LOG(Debayer, Debug) << "vertexIn " << attributeVertex_ << " textureIn " << attributeTexture_ << " tex_y " << textureUniformBayerDataIn_ + << " awb " << awbUniformDataIn_ << " ccm " << ccmUniformDataIn_ << " blacklevel " << blackLevelUniformDataIn_ << " gamma " << gammaUniformDataIn_ @@ -504,6 +506,9 @@ void DebayerEGL::setShaderVariableValues(const DebayerParams ¶ms) glUniform3f(blackLevelUniformDataIn_, params.blackLevel[0], params.blackLevel[1], params.blackLevel[2]); LOG(Debayer, Debug) << " blackLevelUniformDataIn_ " << blackLevelUniformDataIn_ << " data " << params.blackLevel; + glUniform3f(awbUniformDataIn_, params.gains[0], params.gains[1], params.gains[2]); + LOG(Debayer, Debug) << " awbUniformDataIn_ " << awbUniformDataIn_ << " data " << params.gains; + /* * Gamma */ diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index fbd5430e4270678cf05d59cc98dff48315d2bb04..4c7c864726b7d5de1e92d4e74b0bc92860dc04c0 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -90,6 +90,9 @@ private: GLint textureUniformBayerDataIn_; + /* Per-frame AWB gains */ + GLint awbUniformDataIn_; + /* Represent per-frame CCM as a uniform vector of floats 3 x 3 */ GLint ccmUniformDataIn_;