From patchwork Sun Aug 24 00:48:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 24211 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8D48BBEFBE for ; Sun, 24 Aug 2025 00:49:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 10EC36931C; Sun, 24 Aug 2025 02:49:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="tS61ZYPl"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C33BA69319 for ; Sun, 24 Aug 2025 02:49:01 +0200 (CEST) Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-3c8b0f1b699so53779f8f.0 for ; Sat, 23 Aug 2025 17:49:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1755996541; x=1756601341; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ifoFUbMkssRsOpI6z9cjwZnmKjTOulEJYtGZzT5bor4=; b=tS61ZYPlTM8fBrMjkJSrVXfP3GPwC6Y4yNq12vnqC/hb3WEFaywfy9JMUhuOpdJV3K wXgpWzTt9wUibmLU/xXrtPMjcAtMhL30eJ4KE13W5YorMmtBqDx+oE5JhRVH++Q/ypkp OX3wzmhm1a7zBLmYAcvc1FmYz1rJu5bYMqCPPD9DbpMcw9k6PNOfVAA6BOWVINdxlaYO wz1dz0OjNr0hk7wBXyF1aqt5caSlhig5u9IG97iVZ1HdknDOPY9NjZPhQG7wKjRSLf11 VQ2qXqqpMGo+HubQcyX9slEm4GfoxXVKNKIkS0Um/uqf1CUUDXJwiWagrPsXUMehEsvA 15/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755996541; x=1756601341; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ifoFUbMkssRsOpI6z9cjwZnmKjTOulEJYtGZzT5bor4=; b=P47TuPLhxLXb6XYEqkmDEmH0NvCXVm3UQz+D5S4vKPEGGmhoptsL+HHcGSOI5lNisv 6tQPB53SRpM7cJcX+Rq0KSeCxJ35Cq+tfLCfj+sAV+ilX/LZqa7jxWe9mTDBZzn43kyk ysnOhfWS/oEWtg64r1vVKlpx424XG4D7DECXF29n6DGGsengJWTcu/9CnCJFq49zQeOY f4uJbYzWnCPBxnvqjx54QA0naxB4/E39JdTRmOzd02wtPwsX+ebeumVdVwd6EtZRWPTu szGMb3FWymu3UCcXtfTUMIqGx0hAgVMVRIOoc7BtE3vtJ/paG7zwqCKJ4SgrW68+AUev r0qQ== X-Gm-Message-State: AOJu0YznK3HFMpC+LVU4w5liEwVAimhApaKnypxdJE4SB4RaptIgW/Um XpJwjNDsrj7Hx+Vpuy4nSoIMHEOLlC0CEK3gY5/Sz++N0rNQRy/VMerE+fnZMJgLMif6oKhwFEK f48Wdf4E= X-Gm-Gg: ASbGnctZPNA71Eb7wQud5PipcJ0k61qIUT6gp313j1ny3v4v8zOnNwGFYRAsGhvJaZa HRoQbyDp70i/TnGOJ1qaWFH81blCwYCVfASrRhd8Edh/l/0RbsS86lf2tE4twEbxnSvI3eUikGf oL/Bx4hJWp7yK6ks9MmjpTyN4FebhrNs4nxwPP3SvobIC2g1BV2nAIEmcR2Zrx3p22XWGhbgN/w xSCt4Hikxcc6ny4FVQvoIAvpdBzFjPXZzpUPyWXCsYBOYpSjBjOoDGskF9k9Q8fd+Rf5R4KZzol BoZyDJDFpqwaHbOZW362E33yyucjDbXHKVteKr9noBj3WnHu6IGDAeqZUc2Av+gPypTcEHgw7TD 0MNXXNpZlrarzKBLEdeFbnzYOy+r6xYiNNpZdbeXQFWdYcoqFYFWyR6hxvTJVTLkg419TjBJK44 SS3/KBM2CO/IcF6qUy9qhtbhkNBgxmiZA= X-Google-Smtp-Source: AGHT+IGxDUQDr4z767fW5JA6YAXtWJp0dgbTAZjuzydKBP+XNgYFAOYSB7iZqn7hK5kXzHL9q5bILg== X-Received: by 2002:a5d:5d0a:0:b0:3b9:14f2:7eea with SMTP id ffacd0b85a97d-3c5daefc2a3mr5291965f8f.18.1755996541133; Sat, 23 Aug 2025 17:49:01 -0700 (PDT) Received: from [192.168.0.13] (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b4e1d530esm69347225e9.0.2025.08.23.17.49.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Aug 2025 17:49:00 -0700 (PDT) From: Bryan O'Donoghue Date: Sun, 24 Aug 2025 01:48:40 +0100 Subject: [PATCH v2 28/37] libcamera: shaders: Extend debayer shaders to apply RGB gain values on output MIME-Version: 1.0 Message-Id: <20250824-b4-v0-5-2-gpuisp-v2-a-v2-28-96f4576c814e@linaro.org> References: <20250824-b4-v0-5-2-gpuisp-v2-a-v2-0-96f4576c814e@linaro.org> In-Reply-To: <20250824-b4-v0-5-2-gpuisp-v2-a-v2-0-96f4576c814e@linaro.org> To: libcamera-devel@lists.libcamera.org Cc: Bryan O'Donoghue X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6534; i=bryan.odonoghue@linaro.org; h=from:subject:message-id; bh=lmx/VP/ABJfrY2JJzolQMppLP6ALvW2gpux/pMgOaa8=; b=owEBbQKS/ZANAwAKASJxO7Ohjcg6AcsmYgBoqmFh30aFF5nA+PO59kP1VAPh0aeS/uNxVvVVx x23u2+RqISJAjMEAAEKAB0WIQTmk/sqq6Nt4Rerb7QicTuzoY3IOgUCaKphYQAKCRAicTuzoY3I OhhnD/wPl+G04cCfWRvy7BJeGqkzxWRAEehCfq22RsUCAOn2aVSjmTi7YwciEAluk0352Z9ZfCB EpNFYhLKoBHL1YzPK048sI3QTuH7bVKP68MiYLgsxT3LLLLcFwuzZSJN4ZiuvJrg0HK2mORWmDo /jDba+EpGYVgpDYpGvcyqntx4PtrY9b+HuZmfe5UxC98206BVFNYokJR+1ApZjxjJgKN2X5jE1B aLdoZ+lWR5EURIpJ0XOnefIG/8oXj7LlMJ/9+sz70zaI9LAuhhmKwKFM8ZS4dPp5yKaWp0ccttb CZHw92qQ8mZVGQ1capeepvsZ6R3NQ6bI4MpBp1/SMI+eZqjCbwGp+RTq3VB75swhiBQD4ZHfJUO CRb4lYXqzPQgclVwk4EzzwtHhd949fGfETxEoCMGGCOjdx3FXG8KzsjNyHxen/qpyTzrmdqamtJ ESuOXYPM18ErQtpSlOWCnVO02RIr8SK+9tZMKRIgtXb9xcegkdbM5fJf7UvmWjX1Eb6wyAx4Uki 3aHC2DngWT7C2LduXr6kDSinPC2CtUgzStTo2SjJrlcr8tN2vhxot7P1O3znf15W18eaHztLQ9j tYs9e+sLwetk4bjAxW0FXdcz6YRcwbl26Tvkv4fConWzUaWQRe0Y6l1rdMX9jVHUoMkNlx/bKnB YfN4jwFPPeVd3+Q== X-Developer-Key: i=bryan.odonoghue@linaro.org; a=openpgp; fpr=E693FB2AABA36DE117AB6FB422713BB3A18DC83A X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend out the bayer fragment shaders to take 3 x 256 byte inputs as textures from the CPU. We then use an index to the table to recover the colour-corrected values provided by the SoftIPA thread. Signed-off-by: Bryan O'Donoghue --- .../internal/shaders/bayer_1x_packed.frag | 56 +++++++++++++++++++ include/libcamera/internal/shaders/bayer_8.frag | 62 +++++++++++++++++++++- 2 files changed, 117 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/shaders/bayer_1x_packed.frag b/include/libcamera/internal/shaders/bayer_1x_packed.frag index 19b13ad08b3a71e408774dbb8e75ca4e22d9ba8e..90bd645709e02d8cf9bee112b25cbb26c681db0a 100644 --- a/include/libcamera/internal/shaders/bayer_1x_packed.frag +++ b/include/libcamera/internal/shaders/bayer_1x_packed.frag @@ -65,6 +65,10 @@ uniform vec2 tex_step; uniform vec2 tex_bayer_first_red; uniform sampler2D tex_y; +uniform sampler2D red_param; +uniform sampler2D green_param; +uniform sampler2D blue_param; +uniform mat3 ccm; void main(void) { @@ -212,5 +216,57 @@ void main(void) vec3(patterns.y, C, patterns.x) : vec3(patterns.wz, C)); +#if defined(APPLY_CCM_PARAMETERS) + /* + * CCM is a 3x3 in the format + * + * +--------------+----------------+---------------+ + * | RedRedGain | RedGreenGain | RedBlueGain | + * +--------------+----------------+---------------+ + * | GreenRedGain | GreenGreenGain | GreenBlueGain | + * +--------------+----------------+---------------+ + * | BlueRedGain | BlueGreenGain | BlueBlueGain | + * +--------------+----------------+---------------+ + * + * Rout = RedRedGain * Rin + RedGreenGain * Gin + RedBlueGain * Bin + * Gout = GreenRedGain * Rin + GreenGreenGain * Gin + GreenBlueGain * Bin + * Bout = BlueRedGain * Rin + BlueGreenGain * Gin + BlueBlueGain * Bin + * + * We upload to the GPU without transposition glUniformMatrix3f(.., .., GL_FALSE, ccm); + * + * CPU + * float ccm [] = { + * RedRedGain, RedGreenGain, RedBlueGain, + * GreenRedGain, GreenGreenGain, GreenBlueGain, + * BlueRedGain, BlueGreenGain, BlueBlueGain, + * }; + * + * GPU + * ccm = { + * RedRedGain, GreenRedGain, BlueRedGain, + * RedGreenGain, GreenGreenGain, BlueGreenGain, + * RedBlueGain, GreenBlueGain, BlueBlueGain, + * } + * + * However the indexing for the mat data-type is column major hence + * ccm[0][0] = RedRedGain, ccm[0][1] = RedGreenGain, ccm[0][2] = RedBlueGain + * + */ + float rin, gin, bin; + rin = rgb.r; + gin = rgb.g; + bin = rgb.b; + + rgb.r = (rin * ccm[0][0]) + (gin * ccm[0][1]) + (bin * ccm[0][2]); + rgb.g = (rin * ccm[1][0]) + (gin * ccm[1][1]) + (bin * ccm[1][2]); + rgb.b = (rin * ccm[2][0]) + (gin * ccm[2][1]) + (bin * ccm[2][2]); + +#elif defined(APPLY_RGB_PARAMETERS) + /* Apply bayer params */ + rgb.r = texture2D(red_param, vec2(rgb.r, 0.5)).r; + rgb.g = texture2D(green_param, vec2(rgb.g, 0.5)).g; + rgb.b = texture2D(blue_param, vec2(rgb.b, 0.5)).b; +#endif + gl_FragColor = vec4(rgb, 1.0); } diff --git a/include/libcamera/internal/shaders/bayer_8.frag b/include/libcamera/internal/shaders/bayer_8.frag index aa7a1b00436ae15d2ed6af7d666441d20db6bb0b..5955c2eafbe03678158b240740e18c7f00e58057 100644 --- a/include/libcamera/internal/shaders/bayer_8.frag +++ b/include/libcamera/internal/shaders/bayer_8.frag @@ -21,11 +21,17 @@ precision highp float; /** Monochrome RGBA or GL_LUMINANCE Bayer encoded texture.*/ uniform sampler2D tex_y; +uniform sampler2D red_param; +uniform sampler2D green_param; +uniform sampler2D blue_param; varying vec4 center; varying vec4 yCoord; varying vec4 xCoord; +uniform mat3 ccm; void main(void) { + vec3 rgb; + #define fetch(x, y) texture2D(tex_y, vec2(x, y)).r float C = texture2D(tex_y, center.xy).r; // ( 0, 0) @@ -97,11 +103,65 @@ void main(void) { PATTERN.xw += kB.xw * B; PATTERN.xz += kF.xz * F; - gl_FragColor.rgb = (alternate.y == 0.0) ? + rgb = (alternate.y == 0.0) ? ((alternate.x == 0.0) ? vec3(C, PATTERN.xy) : vec3(PATTERN.z, C, PATTERN.w)) : ((alternate.x == 0.0) ? vec3(PATTERN.w, C, PATTERN.z) : vec3(PATTERN.yx, C)); + +#if defined(APPLY_CCM_PARAMETERS) + /* + * CCM is a 3x3 in the format + * + * +--------------+----------------+---------------+ + * | RedRedGain | RedGreenGain | RedBlueGain | + * +--------------+----------------+---------------+ + * | GreenRedGain | GreenGreenGain | GreenBlueGain | + * +--------------+----------------+---------------+ + * | BlueRedGain | BlueGreenGain | BlueBlueGain | + * +--------------+----------------+---------------+ + * + * Rout = RedRedGain * Rin + RedGreenGain * Gin + RedBlueGain * Bin + * Gout = GreenRedGain * Rin + GreenGreenGain * Gin + GreenBlueGain * Bin + * Bout = BlueRedGain * Rin + BlueGreenGain * Gin + BlueBlueGain * Bin + * + * We upload to the GPU without transposition glUniformMatrix3f(.., .., GL_FALSE, ccm); + * + * CPU + * float ccm [] = { + * RedRedGain, RedGreenGain, RedBlueGain, + * GreenRedGain, GreenGreenGain, GreenBlueGain, + * BlueRedGain, BlueGreenGain, BlueBlueGain, + * }; + * + * GPU + * ccm = { + * RedRedGain, GreenRedGain, BlueRedGain, + * RedGreenGain, GreenGreenGain, BlueGreenGain, + * RedBlueGain, GreenBlueGain, BlueBlueGain, + * } + * + * However the indexing for the mat data-type is column major hence + * ccm[0][0] = RedRedGain, ccm[0][1] = RedGreenGain, ccm[0][2] = RedBlueGain + * + */ + float rin, gin, bin; + rin = rgb.r; + gin = rgb.g; + bin = rgb.b; + + rgb.r = (rin * ccm[0][0]) + (gin * ccm[0][1]) + (bin * ccm[0][2]); + rgb.g = (rin * ccm[1][0]) + (gin * ccm[1][1]) + (bin * ccm[1][2]); + rgb.b = (rin * ccm[2][0]) + (gin * ccm[2][1]) + (bin * ccm[2][2]); + +#elif defined(APPLY_RGB_PARAMETERS) + /* Apply bayer params */ + rgb.r = texture2D(red_param, vec2(rgb.r, 0.5)).r; + rgb.g = texture2D(red_param, vec2(rgb.g, 0.5)).g; + rgb.b = texture2D(red_param, vec2(rgb.b, 0.5)).b; +#endif + + gl_FragColor.rgb = rgb; }