From patchwork Thu Nov 27 02:37:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 25254 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E7AA3C0F2A for ; Thu, 27 Nov 2025 02:38:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9171260B0F; Thu, 27 Nov 2025 03:38:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="sSua98qs"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1034D60AB8 for ; Thu, 27 Nov 2025 03:38:21 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-4779cb0a33fso2936345e9.0 for ; Wed, 26 Nov 2025 18:38:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1764211100; x=1764815900; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wNqHKfrdqEnCd6e0vklefM1kTOpuhhmzodWah3Zo3gc=; b=sSua98qsWzVhcyY4U35J8XFhp1oyUgGdo2EdxbLAGFXC2p2HN9IKM6lJyuT2VeOfIE zkbLa2IOYotQ4R+D1KzrJ7iPFjFlJufKsQfo+AIx93BzzoNISUIIeJyV2pm0Z7RHqV4P 2ogH4PBfiJQhdsy5/IQHH9+lovuj6f8W402/73j54W6qIXu/5TZy+7dDIiDA3gwnLO8K Em/wU8XPbW6ve1dX5+DMs06o5YL3oo17oxJG+FCyK5bE0KtfGd3OLnpzrPyMtjjn0dpe i2qY8aGoWimEuDRxKJjpGpuc/xo1elMWZPOkBUo9Cx10iXwqXgUZc00IL0JcG0N8OtNV 2Fvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764211100; x=1764815900; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=wNqHKfrdqEnCd6e0vklefM1kTOpuhhmzodWah3Zo3gc=; b=dlVUiwiYLxlyMst4pXvX6nm7fobaQn/y3uDXrm+g0Ox2/T4O7/n9Lxiioxlv7WE8JE +2dZ2eY2W9p8m2lHR3HJ9NkMVd6hgrpqefLZV6hd372UyjA6RgdJYy9u6WL2QEyUMYOP YXo2iWOnKF3lxm6I0/v2EjN1wS81AjSspm8TYDFhT0rLDzJERvoSFXSEGpTlR3pYKuKn NLEJX1/Q/LuMDn2pEeTE8SloPqJM9mgnZzFNfAwsWnwYZwEHICxNIUeUq3nNLcMjDkbA EbBUEoOQme0NQVYucsExmdRvX0EKZvdUCPNJgkhEANE63VIz+lK3YvXY1XKBDKJF5Pvu mTVA== X-Gm-Message-State: AOJu0Yx3U44UZgOOgU3KCnte6c2hip7p83blwLF0k5nwg2UXbSe9EEa9 uuMLohV5zqWoNt9xyizexK1aO1Lh/AKFEg42BB5vITEgqNJs3H9Tlsmm1ttxBrOrZaCRF10a0T+ 0bs7ywA4= X-Gm-Gg: ASbGncv99VWr1yT7QnEifropehnwf7rO85D/aqPe7a5mth6+1fWBs/t5dpRUeowEqoW tVZNjsn9MOjSqcSVXN5unjgjqWpba2yezzuAI4tvbh74oOFpJ5by4ctmH4dpj+rcPsU/a3CphPq 0VH7qr+Mk/Cu7lvJtPxdSCianQ371kenXus8Am+GA07k8ZJV+h/OaBKqg6qkSQmfpz1XU9M/cIY khtF/rkPAyA+o4panEJYOwgontwqN2TvLHBJWa9lqJM9gVP1Dippol5AQ/1w4+r5NBr/kpjkhK+ Te8yPDQk/ojYnioS1a7JtMGRsTaLRY5VjREPvG9rtfnHseo0K/xai/IqJ6PvB6JBP6f92GVD43D BK/Kn7lm1oIXVlc1P0wljHidzxmRKFsikmUAZwE7MwApc0RyRkwJugUXj9GvoIPJOfIIhPf0m7v oaZ89j7zm5+fAhTw9TK5MAU0h2eDeROu7IuOXNjbPoctfj46RQP+fu/cENPBHrdnj0FbM= X-Google-Smtp-Source: AGHT+IFqX200VboC3AtNvqi0QDpyUB95dLnWrjnYxzzpv17YHNRnvGnvLpVa1g3pGaLZSUTP2lHZ0Q== X-Received: by 2002:a05:600c:470d:b0:471:1717:411 with SMTP id 5b1f17b1804b1-477c01edab1mr272526585e9.24.1764211100395; Wed, 26 Nov 2025 18:38:20 -0800 (PST) Received: from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4790552c3c9sm58186345e9.0.2025.11.26.18.38.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Nov 2025 18:38:19 -0800 (PST) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: pavel@ucw.cz, Bryan O'Donoghue , Kieran Bingham Subject: [PATCH v5 17/24] libcamera: software_isp: Add a gpuisp todo list Date: Thu, 27 Nov 2025 02:37:32 +0000 Message-ID: <20251127023739.179652-18-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251127023739.179652-1-bryan.odonoghue@linaro.org> References: <20251127023739.179652-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" List the series of things to do in GPU ISP in perceived order of difficulty. Acked-by: Kieran Bingham Signed-off-by: Bryan O'Donoghue --- src/libcamera/software_isp/gpuisp-todo.txt | 83 ++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 src/libcamera/software_isp/gpuisp-todo.txt diff --git a/src/libcamera/software_isp/gpuisp-todo.txt b/src/libcamera/software_isp/gpuisp-todo.txt new file mode 100644 index 000000000..768dcc320 --- /dev/null +++ b/src/libcamera/software_isp/gpuisp-todo.txt @@ -0,0 +1,83 @@ +List the TODOs in perceived order of ease. + +Version 3: +Use dma-buf handle to generate upload texture: + - eglCreateImageKHR can be used to generate the upload texture i.e. + to feed the bayer data into the GPU. + +Denoising: + - As below still TBD + +Dead pixel correction: + - As below still TBD + +Lense shading correction: + - This is WIP but as yet still TBD + +Lense flare correction: + - Not WIP still TBD + +processFrame() to run in its own thread: + - processFrame() runs in the context of the Debayer::process() + thread. Robert Mader suggested and it seems like a good + suggestion too to run processFrame() in its own thread. + +Version 2: +Make GPUISP default: + - Right now the environment variable allows over-riding to swtich + from CPU to GPU. + - Once we support 24 BPP output on GPUISP we will have the same + pixel format support as CPU and can set the default to GPU without + regressing functionality + +glTexture1D: + - Initial code was developed for < GLES 2.O but since we have fixed + on GLES >= 2.0 this means we can use glTexture1D + - Provided this is so amend the shaders to do val = texture(x, y, 0); + not texture(x, y, 0.5) the 0.5 is because of using glTexture2D + +Denoising: + - Run a denoise algorithm in the shaders + - Supply a control to influence the noise-floor ? + +Dead pixel correction: + - Add logic to correct dead pixels in the fragment shaders + +Version 1: +24 bit output support: + - Take the BPP we already capture and get a 24 bit GBM surface + - Pass a compile-time parameter to the shaders to tell them to do + gl_FragColor = rgb not gl_FragColor = rgba + - Version 2: + This is not possible. + gl_FragColor expects vec4 not vec3 on the output. + If you really want RGB888 run cpuisp. + +Surfaceless GBM: + - We get a GBM surface and then have to swap buffers + If we rework for surfaceless GBM and EGL then the swap buffer can + be dropped. + - Version 2: + Complete GBM surface removed, memcpy() phase removed also + +dma-buf texture upload: + - Currently we pass the input buffer to glCreateTexture2D. + We should be able to make the upload of the input buffer go faster + by using eglCreateImageKHR and enumerated the dma-buf contents. + - Version 2: + Complete sm8250 test platform shows 20x performance increase + with CCM. + +Render-to-texture: + - Right now we render to the GBM provided surface framebuffer + and then memcpy from that buffer to the target output buffer. + This necessitates flushing the cache on the target buffer in + addition to the memcpy(). + - Render-to-texture where we generate the target framebuffer + directly from a dma-buf handle will mitigate the memcpy() phase. + - It should be the case then that the consumer of the output buffer + i.e. the thing that's not libcamera is responsible to flush the cache + if-and-only-if that user writes to the buffer. + - We need to flush the cache on the buffer because we are memcpying() to it. + - Version 2: + Done in version 2