[{"id":38523,"web_url":"https://patchwork.libcamera.org/comment/38523/","msgid":"<20260408001702.GN1268443@killaraus.ideasonboard.com>","date":"2026-04-08T00:17:02","subject":"Re: [PATCH 05/13] shaders: bayer: Use native matrix multiplication","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Apr 07, 2026 at 11:01:08PM +0100, Kieran Bingham wrote:\n> Remove the open coded matrix multiplication as glsl can perform this\n> directly. Adapt the uniform ccmUniformDataIn_ to ensure we correctly\n> upload the matrix in row major ordering.\n> \n> Signed-off-by: Kieran Bingham \n> ---\n> src/libcamera/shaders/bayer_1x_packed.frag | 45 ++----------------------------\n> src/libcamera/shaders/bayer_unpacked.frag | 44 ++---------------------------\n> src/libcamera/software_isp/debayer_egl.cpp | 12 ++------\n> 3 files changed, 7 insertions(+), 94 deletions(-)\n> \n> diff --git a/src/libcamera/shaders/bayer_1x_packed.frag b/src/libcamera/shaders/bayer_1x_packed.frag\n> index 9a1992e219dd066945b3f46ec509f47a31590385..57f0be9b264bdbdf523291dc9b35ab5fd6c9ef6a 100644\n> --- a/src/libcamera/shaders/bayer_1x_packed.frag\n> +++ b/src/libcamera/shaders/bayer_1x_packed.frag\n> @@ -231,49 +231,8 @@ void main(void)\n> \t/* Apply AWB gains, and saturate each channel at sensor range */\n> \trgb = clamp(rgb * awb, vec3(0.0), vec3(1.0) - blacklevel);\n> \n> -\t/*\n> -\t * CCM is a 3x3 in the format\n> -\t *\n> -\t * +--------------+----------------+---------------+\n> -\t * | RedRedGain | RedGreenGain | RedBlueGain |\n> -\t * +--------------+----------------+---------------+\n> -\t * | GreenRedGain | GreenGreenGain | GreenBlueGain |\n> -\t * +--------------+----------------+---------------+\n> -\t * | BlueRedGain | BlueGreenGain | BlueBlueGain |\n> -\t * +--------------+----------------+---------------+\n> -\t *\n> -\t * Rout = RedRedGain * Rin + RedGreenGain * Gin + RedBlueGain * Bin\n> -\t * Gout = GreenRedGain * Rin + GreenGreenGain * Gin + GreenBlueGain * Bin\n> -\t * Bout = BlueRedGain * Rin + BlueGreenGain * Gin + BlueBlueGain * Bin\n> -\t *\n> -\t * We upload to the GPU without transposition glUniformMatrix3f(.., .., GL_FALSE, ccm);\n> -\t *\n> -\t * CPU\n> -\t * float ccm [] = {\n> -\t * RedRedGain, RedGreenGain, RedBlueGain,\n> -\t * GreenRedGain, GreenGreenGain, GreenBlueGain,\n> -\t * BlueRedGain, BlueGreenGain, BlueBlueGain,\n> -\t * };\n> -\t *\n> -\t * GPU\n> -\t * ccm = {\n> -\t * RedRedGain, GreenRedGain, BlueRedGain,\n> -\t * RedGreenGain, GreenGreenGain, BlueGreenGain,\n> -\t * RedBlueGain, GreenBlueGain, BlueBlueGain,\n> -\t * }\n> -\t *\n> -\t * However the indexing for the mat data-type is column major hence\n> -\t * ccm[0][0] = RedRedGain, ccm[0][1] = RedGreenGain, ccm[0][2] = RedBlueGain\n> -\t *\n> -\t */\n> -\tfloat rin, gin, bin;\n> -\trin = rgb.r;\n> -\tgin = rgb.g;\n> -\tbin = rgb.b;\n> -\n> -\trgb.r = (rin * ccm[0][0]) + (gin * ccm[0][1]) + (bin * ccm[0][2]);\n> -\trgb.g = (rin * ccm[1][0]) + (gin * ccm[1][1]) + (bin * ccm[1][2]);\n> -\trgb.b = (rin * ccm[2][0]) + (gin * ccm[2][1]) + (bin * ccm[2][2]);\n> +\t/* Row major Colour Correction Matrix multiplication */\n\nRow-major is related to how data is stored in memory. When writing\n\n\tccm * rgb\n\nthat's not relevant any more. You can drop it from the comment. Same\nbelow.\n\n> +\trgb = ccm * rgb;\n\nI wonder why that was open-coded. Any idea ? I suppose there's no\nnegative performance impact from this patch, as the compiler should\ngenerate the same or better instructions ?\n\n> \n> \t/*\n> \t * Contrast\n> diff --git a/src/libcamera/shaders/bayer_unpacked.frag b/src/libcamera/shaders/bayer_unpacked.frag\n> index 87e4e4915fe19679943fdcc3d213a0224b89065e..52de0a7901cfac33dbc7b306f1840ec0048cd8ea 100644\n> --- a/src/libcamera/shaders/bayer_unpacked.frag\n> +++ b/src/libcamera/shaders/bayer_unpacked.frag\n> @@ -134,49 +134,9 @@ void main(void) {\n> \t/* Apply AWB gains, and saturate each channel at sensor range */\n> \trgb = clamp(rgb * awb, vec3(0.0), vec3(1.0) - blacklevel);\n> \n> -\t/*\n> -\t * CCM is a 3x3 in the format\n> -\t *\n> -\t * +--------------+----------------+---------------+\n> -\t * | RedRedGain | RedGreenGain | RedBlueGain |\n> -\t * +--------------+----------------+---------------+\n> -\t * | GreenRedGain | GreenGreenGain | GreenBlueGain |\n> -\t * +--------------+----------------+---------------+\n> -\t * | BlueRedGain | BlueGreenGain | BlueBlueGain |\n> -\t * +--------------+----------------+---------------+\n> -\t *\n> -\t * Rout = RedRedGain * Rin + RedGreenGain * Gin + RedBlueGain * Bin\n> -\t * Gout = GreenRedGain * Rin + GreenGreenGain * Gin + GreenBlueGain * Bin\n> -\t * Bout = BlueRedGain * Rin + BlueGreenGain * Gin + BlueBlueGain * Bin\n> -\t *\n> -\t * We upload to the GPU without transposition glUniformMatrix3f(.., .., GL_FALSE, ccm);\n> -\t *\n> -\t * CPU\n> -\t * float ccm [] = {\n> -\t * RedRedGain, RedGreenGain, RedBlueGain,\n> -\t * GreenRedGain, GreenGreenGain, GreenBlueGain,\n> -\t * BlueRedGain, BlueGreenGain, BlueBlueGain,\n> -\t * };\n> -\t *\n> -\t * GPU\n> -\t * ccm = {\n> -\t * RedRedGain, GreenRedGain, BlueRedGain,\n> -\t * RedGreenGain, GreenGreenGain, BlueGreenGain,\n> -\t * RedBlueGain, GreenBlueGain, BlueBlueGain,\n> -\t * }\n> -\t *\n> -\t * However the indexing for the mat data-type is column major hence\n> -\t * ccm[0][0] = RedRedGain, ccm[0][1] = RedGreenGain, ccm[0][2] = RedBlueGain\n> -\t *\n> -\t */\n> -\tfloat rin, gin, bin;\n> -\trin = rgb.r;\n> -\tgin = rgb.g;\n> -\tbin = rgb.b;\n> \n> -\trgb.r = (rin * ccm[0][0]) + (gin * ccm[0][1]) + (bin * ccm[0][2]);\n> -\trgb.g = (rin * ccm[1][0]) + (gin * ccm[1][1]) + (bin * ccm[1][2]);\n> -\trgb.b = (rin * ccm[2][0]) + (gin * ccm[2][1]) + (bin * ccm[2][2]);\n> +\t/* Row major Colour Correction Matrix multiplication */\n> +\trgb = ccm * rgb;\n> \n> \t/*\n> \t * Contrast\n> diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp\n> index 738036e649224f370bdf9a0cc59b399f8b1066de..1022f595cc71751cd0c9133b3e5755f591224dc4 100644\n> --- a/src/libcamera/software_isp/debayer_egl.cpp\n> +++ b/src/libcamera/software_isp/debayer_egl.cpp\n> @@ -464,15 +464,9 @@ void DebayerEGL::setShaderVariableValues(const DebayerParams ¶ms)\n> \t\t\t << \" textureUniformProjMatrix_ \" << textureUniformProjMatrix_;\n> \n> \tGLfloat ccm[9] = {\n> -\t\tparams.combinedMatrix[0][0],\n> -\t\tparams.combinedMatrix[0][1],\n> -\t\tparams.combinedMatrix[0][2],\n> -\t\tparams.combinedMatrix[1][0],\n> -\t\tparams.combinedMatrix[1][1],\n> -\t\tparams.combinedMatrix[1][2],\n> -\t\tparams.combinedMatrix[2][0],\n> -\t\tparams.combinedMatrix[2][1],\n> -\t\tparams.combinedMatrix[2][2],\n> +\t\tparams.combinedMatrix[0][0], params.combinedMatrix[1][0], params.combinedMatrix[2][0],\n> +\t\tparams.combinedMatrix[0][1], params.combinedMatrix[1][1], params.combinedMatrix[2][1],\n> +\t\tparams.combinedMatrix[0][2], params.combinedMatrix[1][2], params.combinedMatrix[2][2],\n\nYou're now storing the data in column-major order, ...\n\n> \t};\n> \tglUniformMatrix3fv(ccmUniformDataIn_, 1, GL_FALSE, ccm);\n\n... and the third argument matches that. As the libcamera Matrix class\nstores elements in row-major order, I think it would be clearer to store\nthe data in row-major order in the ccm array as well, and set the third\nargument to GL_TRUE accordingly.\n\nWe could then possibly avoid copying the data to a local array (here or\nas a separate patch).\n\n> \tLOG(Debayer, Debug) << \" ccmUniformDataIn_ \" << ccmUniformDataIn_ << \" data \" << params.combinedMatrix;\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 86901BEFBE\n\tfor ;\n\tWed, 8 Apr 2026 00:17:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8E79B62DA7;\n\tWed, 8 Apr 2026 02:17:05 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D03662CEB\n\tfor ;\n\tWed, 8 Apr 2026 02:17:04 +0200 (CEST)","from killaraus.ideasonboard.com\n\t(2001-14ba-703d-e500--2a1.rev.dnainternet.fi\n\t[IPv6:2001:14ba:703d:e500::2a1])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 62ABE1121; \n\tWed, 8 Apr 2026 02:15:36 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cV/zLL9z\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1775607336;\n\tbh=vGvZCBsjVB1z4e82FQzLBGwd1DRouQ3yKjrKkt0Ia5E=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=cV/zLL9z/nSZKBniN3JnCHtjXou7VVxRhnvL5AnsqhL/Yn0kIEiOaJkcY3StS0EXd\n\tlXDBQfj0vAkQBpJ1FdYu6KdLi8xuSbmsYeOWkEa5e1T39n5bGhcM5eQFvjYnEkpJ22\n\tOHZ9XBeqYREGAJrhr/jOpAmgFUI/agWSN1FuW6FY=","Date":"Wed, 8 Apr 2026 03:17:02 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH 05/13] shaders: bayer: Use native matrix multiplication","Message-ID":"<20260408001702.GN1268443@killaraus.ideasonboard.com>","References":"<20260407-kbingham-awb-split-v1-0-a39af3f4dc20@ideasonboard.com>\n\t<20260407-kbingham-awb-split-v1-5-a39af3f4dc20@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20260407-kbingham-awb-split-v1-5-a39af3f4dc20@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]