From patchwork Wed Jun 11 01:32:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 23539 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5024FC3329 for ; Wed, 11 Jun 2025 01:34:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7EDAD68DFE; Wed, 11 Jun 2025 03:34:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="oXancp/e"; dkim-atps=neutral Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6DFE68DFB for ; Wed, 11 Jun 2025 03:34:36 +0200 (CEST) Received: by mail-wm1-x32d.google.com with SMTP id 5b1f17b1804b1-441ab63a415so63502735e9.3 for ; Tue, 10 Jun 2025 18:34:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1749605676; x=1750210476; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IhTu7mdgMev/e/IpYBpbMZAEkHJmFS1pXA/lABpS35g=; b=oXancp/eIq3LVsk85GUQUNwhGeg6A+eTwAn/3BhwD9a8QvuCZZ6C83Q6BPgcAfELyb VHBVA6vtU8MXP92Nw/0DNC3bbIpNVwn5Dn7u+XSd6ruhUKmTdtspuXA1aN3fzL/gRand bsF5XhsH4V13WgMHld/fDBjtaazMp1axZAylPKVkH03kh4Q/dja6/StD1g3bppPFwldM LW19Y/tsA9ClY79abFH/cSmQK50QRyEuqOVa40ithixEPY+H4lg5FNDqO35mQhXgW4QE W5u6nyHYVyOJ/Be0d0ZlTSSbOipPei/Mbgf+bhN6zq2NWwOH3RiQZ/3htdUXTfTCod/A TZbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749605676; x=1750210476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IhTu7mdgMev/e/IpYBpbMZAEkHJmFS1pXA/lABpS35g=; b=HBLCkbZrZNPwu8xJqVwaQMuVmOuduCmUqYxHqbXE/ZPNBdKH3e5bxaILBpt4GP3w4a 73wexi3Ebrkn+OLRdqrKAmWzid5Ci/bdfRwGDCxepJUYINc+iDZYYjLRun6ZdurCydOw FSHAwr8LHK/hySxwjKGh6JGkCwJTQQARFJp05Q0A/ejGmqrkBortt7rwfsVCUd9257sq 7k9X3idm4NJ+E6OtFY4Ofmp32kvwtXejG94U/5tkdTUCbiLDBcLLPtLDCAQ4KmP0DTbY 9Jh/zDmOz5pTGBZRRz8rxT824hdloCru2bXhk2r2a3b6MSTZKbeKcibHtXgDtRuBToI4 yfEQ== X-Gm-Message-State: AOJu0YymvOE1TSHCKUiiy8cms4e/AhMi90+5L5To0EAc27NktA7r1irL XGOYww55dHZAXWSKoxzVodurMS1yyofD5OVxXzd4CZ7iBmO8Cmud2fBbL04Ih0rYvXOHwp0wB8W JsBJOhQs= X-Gm-Gg: ASbGncvLafIx7nnEJyG90qNxS9lCS6MUgcG3QToflQl6f8KUS6kmX82e9ceD3lskMDS TN2x+W4X5IgjCSY4X8Zr9R6O/sFCxnlBDbg1SY7U879klcaBrCZnQfe5cn2ADW1vv61/z/1Jfwi uDczqQCkpX3fkt9AcUI7MCUjJCtGBJ2P8Lf6H4p2pny0LWcU83a0Pe6BEeGSM5hgPdXT12mf/2u teMX6c9DS6rc2549kKw36g7ZmfT6g0OxjVWWq0QEUVrpDUg5y5+QcmcKq1oclg9j2yXBHRm5Yyc 6Tr4FL9HHyDhigqNhMk094sDiQYZW3CuUXiMYFRWj5wjBkyTelFxBKRJx7BnCnsiLur6GDNWFUq Rcolht1nkbo3orLbY/XmHK2hBJxwFU1fCfwDgkF+gaA== X-Google-Smtp-Source: AGHT+IHlnsui8rdDPd5HazGwud6/iw05I3aHfIw4O7HRFeHrsxjaBJDVHt8KdXT8ycyFSdNq/dG4nw== X-Received: by 2002:a05:600c:64c5:b0:450:cff7:62f9 with SMTP id 5b1f17b1804b1-453248c3bdamr7902115e9.22.1749605676007; Tue, 10 Jun 2025 18:34:36 -0700 (PDT) Received: from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4532514138asm5680625e9.3.2025.06.10.18.34.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Jun 2025 18:34:34 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Bryan O'Donoghue Subject: [PATCH 33/35] libcamera: software_isp: GPU support for unpacked 10/12-bit formats Date: Wed, 11 Jun 2025 02:32:43 +0100 Message-ID: <20250611013245.133785-34-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250611013245.133785-1-bryan.odonoghue@linaro.org> References: <20250611013245.133785-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Milan Zamazal The GPU processing supports 8-bit sensor formats and 10/12-bit packed formats. Support for 10/12-bit unpacked formats is missing, let's add it. 10/12-bit unpacked formats use two adjacent bytes to store the value. This means the 8-bit shaders can be used if we can modify them for additional support of 16-bit addressing. This requires the following modifications: - Using GL_RG (two bytes per pixel) instead of GL_LUMINANCE (one byte per pixel) as the texture format for the given input formats. - Setting the texture width to the number of pixels rather than the number of bytes. - Making the definition of `fetch' macro variable, according to the pixel format. - Using only `fetch' for accessing the texture. Signed-off-by: Milan Zamazal Signed-off-by: Bryan O'Donoghue Reviewed-by: Bryan O'Donoghue --- .../libcamera/internal/shaders/bayer_8.frag | 10 +++- src/libcamera/software_isp/debayer_egl.cpp | 46 +++++++++++++------ src/libcamera/software_isp/debayer_egl.h | 2 + 3 files changed, 44 insertions(+), 14 deletions(-) diff --git a/include/libcamera/internal/shaders/bayer_8.frag b/include/libcamera/internal/shaders/bayer_8.frag index 74ce1509..78c2609c 100644 --- a/include/libcamera/internal/shaders/bayer_8.frag +++ b/include/libcamera/internal/shaders/bayer_8.frag @@ -32,9 +32,17 @@ uniform mat3 ccm; void main(void) { vec3 rgb; + #if defined(RAW10P) + #define pixel(p) p.r / 4.0 + p.g * 64.0 + #define fetch(x, y) pixel(texture2D(tex_y, vec2(x, y))) + #elif defined(RAW12P) + #define pixel(p) p.r / 16.0 + p.g * 16.0 + #define fetch(x, y) pixel(texture2D(tex_y, vec2(x, y))) + #else #define fetch(x, y) texture2D(tex_y, vec2(x, y)).r + #endif - float C = texture2D(tex_y, center.xy).r; // ( 0, 0) + float C = fetch(center.x, center.y); // ( 0, 0) const vec4 kC = vec4( 4.0, 6.0, 5.0, 5.0) / 8.0; // Determine which of four types of pixels we are on. diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index b30d2107..71742d84 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -151,6 +151,8 @@ int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputForm } // Pixel location parameters + glFormat_ = GL_LUMINANCE; + bytesPerPixel_ = 1; switch (inputFormat) { case libcamera::formats::SBGGR8: case libcamera::formats::SBGGR10_CSI2P: @@ -197,20 +199,38 @@ int DebayerEGL::initBayerShaders(PixelFormat inputFormat, PixelFormat outputForm case libcamera::formats::SGRBG10_CSI2P: case libcamera::formats::SRGGB10_CSI2P: egl_.pushEnv(shaderEnv, "#define RAW10P"); - fragmentShaderData = bayer_1x_packed_frag; - fragmentShaderDataLen = bayer_1x_packed_frag_len; - vertexShaderData = identity_vert; - vertexShaderDataLen = identity_vert_len; + if (BayerFormat::fromPixelFormat(inputFormat).packing == BayerFormat::Packing::None) { + fragmentShaderData = bayer_8_frag; + fragmentShaderDataLen = bayer_8_frag_len; + vertexShaderData = bayer_8_vert; + vertexShaderDataLen = bayer_8_vert_len; + glFormat_ = GL_RG; + bytesPerPixel_ = 2; + } else { + fragmentShaderData = bayer_1x_packed_frag; + fragmentShaderDataLen = bayer_1x_packed_frag_len; + vertexShaderData = identity_vert; + vertexShaderDataLen = identity_vert_len; + } break; case libcamera::formats::SBGGR12_CSI2P: case libcamera::formats::SGBRG12_CSI2P: case libcamera::formats::SGRBG12_CSI2P: case libcamera::formats::SRGGB12_CSI2P: egl_.pushEnv(shaderEnv, "#define RAW12P"); - fragmentShaderData = bayer_1x_packed_frag; - fragmentShaderDataLen = bayer_1x_packed_frag_len; - vertexShaderData = identity_vert; - vertexShaderDataLen = identity_vert_len; + if (BayerFormat::fromPixelFormat(inputFormat).packing == BayerFormat::Packing::None) { + fragmentShaderData = bayer_8_frag; + fragmentShaderDataLen = bayer_8_frag_len; + vertexShaderData = bayer_8_vert; + vertexShaderDataLen = bayer_8_vert_len; + glFormat_ = GL_RG; + bytesPerPixel_ = 2; + } else { + fragmentShaderData = bayer_1x_packed_frag; + fragmentShaderDataLen = bayer_1x_packed_frag_len; + vertexShaderData = identity_vert; + vertexShaderDataLen = identity_vert_len; + } break; default: goto invalid_fmt; @@ -430,7 +450,7 @@ void DebayerEGL::setShaderVariableValues(void) GLfloat firstRed[] = { firstRed_x_, firstRed_y_ }; GLfloat imgSize[] = { (GLfloat)width_, (GLfloat)height_ }; - GLfloat Step[] = { 1.0f / (inputConfig_.stride - 1), + GLfloat Step[] = { static_cast(bytesPerPixel_) / (inputConfig_.stride - 1), 1.0f / (height_ - 1) }; GLfloat Stride = 1.0f; GLfloat projIdentityMatrix[] = { @@ -507,7 +527,7 @@ void DebayerEGL::debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out, Debay // Greate a standard texture // we will replace this with the DMA version at some point - egl_.createTexture2D(eglImageBayerIn_, inputConfig_.stride, height_, in.planes()[0].data()); + egl_.createTexture2D(eglImageBayerIn_, glFormat_, inputConfig_.stride / bytesPerPixel_, height_, in.planes()[0].data()); // Populate bayer parameters if (ccmEnabled_) { @@ -518,9 +538,9 @@ void DebayerEGL::debayerGPU(MappedFrameBuffer &in, MappedFrameBuffer &out, Debay }; glUniformMatrix3fv(ccmUniformDataIn_, 1, GL_FALSE, ccm); } else { - egl_.createTexture2D(eglImageRedLookup_, DebayerParams::kRGBLookupSize, 1, ¶ms.red); - egl_.createTexture2D(eglImageGreenLookup_, DebayerParams::kRGBLookupSize, 1, ¶ms.green); - egl_.createTexture2D(eglImageBlueLookup_, DebayerParams::kRGBLookupSize, 1, ¶ms.blue); + egl_.createTexture2D(eglImageRedLookup_, GL_LUMINANCE, DebayerParams::kRGBLookupSize, 1, ¶ms.red); + egl_.createTexture2D(eglImageGreenLookup_, GL_LUMINANCE, DebayerParams::kRGBLookupSize, 1, ¶ms.green); + egl_.createTexture2D(eglImageBlueLookup_, GL_LUMINANCE, DebayerParams::kRGBLookupSize, 1, ¶ms.blue); } // Setup the scene diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index 94bc6fc4..56f5434a 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -150,6 +150,8 @@ private: GBM gbmSurface_; uint32_t width_; uint32_t height_; + GLint glFormat_; + unsigned int bytesPerPixel_; GLfloat vcoordinates[DEBAYER_OPENGL_COORDS][2] = { { -1.0f, -1.0f },