From patchwork Wed Oct 15 01:22:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 24684 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 96ACDC3331 for ; Wed, 15 Oct 2025 01:23:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0ECDD60693; Wed, 15 Oct 2025 03:23:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="xrsXp1TH"; dkim-atps=neutral Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40EE460677 for ; Wed, 15 Oct 2025 03:23:35 +0200 (CEST) Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-b48d8deafaeso1242439466b.1 for ; Tue, 14 Oct 2025 18:23:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1760491414; x=1761096214; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vWBqVaFe6+Z55hicVVGDke3oD9foA7KiH/lYfdfC2lw=; b=xrsXp1THbMSb4H/MpIJLQS4sRmGDNXANlYFOpQYDl19MEKi0WqwaBavljdD8Ru9rd7 LoJGCPsyldlf2oDpc3b15QTUrd69rPoYNMO37hOyU9iTs3SFKhHJlH3rFER3Sa1DF/PE M+eX0vvC7kYR6re1capm/uWY9djhHoEpfeC5V2dtxYBlCafCYcc//r4OKdI2xyyvvItX SpJO/1RQjDRZifWB4sx+uQ+6WHAjfPt9e6mxaMRgIJbY1aknN/OI22kqG+S1i+0aGLZo KZGtfCNFzyAAi7AQTqFqy0kXawJDx3wtesrZy2iMguwd62Z8lU5D8GoOpLcD+aUn0uFw NPGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1760491414; x=1761096214; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vWBqVaFe6+Z55hicVVGDke3oD9foA7KiH/lYfdfC2lw=; b=QFtW5hl7fTMQuaXe3zYGBFVjjf62YHApzVRqfmniSp+heLXBIhqjqpu/8dt2WXA6+D XYhns+1CkOV8zhGv/zjWyI6jLpQPokcP2c47IxEWdj/bTa6AHwVl3qIDPJkvoekCXm51 2qyhEbqTcYK20y8hXlrbwvxseO0ITxbQBS6eK98FfRKQsqRE8snGkikV7rqz3UfNQzD+ x/UWSpY4cSR7Cw24eyUz5pkxFPXimc1Ic7mICa1QEbbpNTyXjtQYXRWg1NOzv/GUp+Wn DU70g0ZSb73gqeJ9JIf4Oq/uz2beKqB7sYLEjVa9RltefMEWj0uTWThAhPdK4olJtfuq 2fTw== X-Gm-Message-State: AOJu0YyMgYWTu3VNpxzR2gSbvQ6AM8OABqnwBRLkK33LAMx4LM8fwp65 IW1j85sV754VJJ9a6pXLl6rk5yRbQol7duFvM2eaG0Bm/4vxKjgPIiAsPOEgwPDpZIyWXQaieNs PE8oo X-Gm-Gg: ASbGnctL8AArHkZDLp9SKRW0yGl8A17BffvchQmWgHwuM9tx0EHgWOBR7E+5pyslVmi eks4+U7Q0LcZU1ojHOd6ahJm1bqgYwynI4Y4sCjv6bOTtOLnaiDfTB3/Q+8fVKLlaU6UPqjsuyk zkFAdiKrdyEW6kbA01gpA/8ACxz2AYy1SdvOIrs/ZkGtobqymQXiYzhZL/OTQncC1Q1sjKZksqJ klviaKnlTEIvSgvtlpZ2LyVCFhP7ISNv/YzFKDJm5aKk1WDCkpQrhxLsPrIVox2uiAsz6WYPlas m/9oQUnBr1DZnsoTl7+ZD4Vj/hvwawIx6/pyAJJBbRb7GtNE/tJMyTqmheTwHf1UDgIGwcQJNCj 96PazAPy3RKBIKBA28vJLQ1bEAs1FHRtLMyH9ywE/SsYTI7ZcSGTYHwglRL+tOyk2ropR2Q2E87 cwestSTaHiRtCY1UnM3W/YpxmcH/YajtbDpCsJu230 X-Google-Smtp-Source: AGHT+IEQU46biLgCtkvKLNH2VkF99O8caZLeOx8kDAz3Q/h7/6hShQeungPCg8sMoyi9Hr//2LpDxA== X-Received: by 2002:a17:906:7949:b0:b04:25e6:2dbe with SMTP id a640c23a62f3a-b50acb0bdc5mr2844157566b.63.1760491414429; Tue, 14 Oct 2025 18:23:34 -0700 (PDT) Received: from inspiron14p-linux.nxsw.local (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b5ccd7b202dsm98348466b.82.2025.10.14.18.23.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Oct 2025 18:23:33 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: hdegoede@redhat.com, mzamazal@redhat.com, bryan.odonoghue@linaro.org, bod.linux@nxsw.ie Subject: [PATCH v3 39/39] libcamera: software_isp: Add a gpuisp todo list Date: Wed, 15 Oct 2025 02:22:51 +0100 Message-ID: <20251015012251.17508-40-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251015012251.17508-1-bryan.odonoghue@linaro.org> References: <20251015012251.17508-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" List the series of things to do in GPU ISP in perceived order of difficulty. Signed-off-by: Bryan O'Donoghue Acked-by: Kieran Bingham --- src/libcamera/software_isp/gpuisp-todo.txt | 83 ++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 src/libcamera/software_isp/gpuisp-todo.txt diff --git a/src/libcamera/software_isp/gpuisp-todo.txt b/src/libcamera/software_isp/gpuisp-todo.txt new file mode 100644 index 00000000..768dcc32 --- /dev/null +++ b/src/libcamera/software_isp/gpuisp-todo.txt @@ -0,0 +1,83 @@ +List the TODOs in perceived order of ease. + +Version 3: +Use dma-buf handle to generate upload texture: + - eglCreateImageKHR can be used to generate the upload texture i.e. + to feed the bayer data into the GPU. + +Denoising: + - As below still TBD + +Dead pixel correction: + - As below still TBD + +Lense shading correction: + - This is WIP but as yet still TBD + +Lense flare correction: + - Not WIP still TBD + +processFrame() to run in its own thread: + - processFrame() runs in the context of the Debayer::process() + thread. Robert Mader suggested and it seems like a good + suggestion too to run processFrame() in its own thread. + +Version 2: +Make GPUISP default: + - Right now the environment variable allows over-riding to swtich + from CPU to GPU. + - Once we support 24 BPP output on GPUISP we will have the same + pixel format support as CPU and can set the default to GPU without + regressing functionality + +glTexture1D: + - Initial code was developed for < GLES 2.O but since we have fixed + on GLES >= 2.0 this means we can use glTexture1D + - Provided this is so amend the shaders to do val = texture(x, y, 0); + not texture(x, y, 0.5) the 0.5 is because of using glTexture2D + +Denoising: + - Run a denoise algorithm in the shaders + - Supply a control to influence the noise-floor ? + +Dead pixel correction: + - Add logic to correct dead pixels in the fragment shaders + +Version 1: +24 bit output support: + - Take the BPP we already capture and get a 24 bit GBM surface + - Pass a compile-time parameter to the shaders to tell them to do + gl_FragColor = rgb not gl_FragColor = rgba + - Version 2: + This is not possible. + gl_FragColor expects vec4 not vec3 on the output. + If you really want RGB888 run cpuisp. + +Surfaceless GBM: + - We get a GBM surface and then have to swap buffers + If we rework for surfaceless GBM and EGL then the swap buffer can + be dropped. + - Version 2: + Complete GBM surface removed, memcpy() phase removed also + +dma-buf texture upload: + - Currently we pass the input buffer to glCreateTexture2D. + We should be able to make the upload of the input buffer go faster + by using eglCreateImageKHR and enumerated the dma-buf contents. + - Version 2: + Complete sm8250 test platform shows 20x performance increase + with CCM. + +Render-to-texture: + - Right now we render to the GBM provided surface framebuffer + and then memcpy from that buffer to the target output buffer. + This necessitates flushing the cache on the target buffer in + addition to the memcpy(). + - Render-to-texture where we generate the target framebuffer + directly from a dma-buf handle will mitigate the memcpy() phase. + - It should be the case then that the consumer of the output buffer + i.e. the thing that's not libcamera is responsible to flush the cache + if-and-only-if that user writes to the buffer. + - We need to flush the cache on the buffer because we are memcpying() to it. + - Version 2: + Done in version 2