From patchwork Thu Nov 20 23:33:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 25140 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9C110BD80A for ; Thu, 20 Nov 2025 23:34:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 52CA260ABD; Fri, 21 Nov 2025 00:34:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="CqnKy3nW"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3B18E60AC8 for ; Fri, 21 Nov 2025 00:34:23 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-42b32900c8bso864615f8f.0 for ; Thu, 20 Nov 2025 15:34:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1763681662; x=1764286462; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wNqHKfrdqEnCd6e0vklefM1kTOpuhhmzodWah3Zo3gc=; b=CqnKy3nWwYwaUhj3zsRDEw6nyIp5iJ6dDwxpN47JKFSsCcXzuyqdfWj2FYJpAA5m1m RmQQ4HYmdrF4dhPeyOwq3Ff1oWCH53AXHLlba4A0FfLXqb9JLyPCYq/zkk0lwW4Gc8L3 gpvkrlqkuwFUPtn7llyvKC5lbD6oeBscAx3l9jgeVXBkKOp4ucqXHVGqYMshvTLVwmWD DWr/osD4/5NSUsUTYJYgJB0+pHFd5N3rYzTIgdM4Yf3jJX+YunW3r6NwxZ+z9YiVqL4u qv+clywr1AtMPTNGoJ1lMYvQ8YgE4M5S4LV5J5CI7wj25UUk2jBuhSWhl3k/JDJ+TKuU IQLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763681662; x=1764286462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=wNqHKfrdqEnCd6e0vklefM1kTOpuhhmzodWah3Zo3gc=; b=MsnnOiIjJlRma1J8p75yqusquIhNYu9J0UrVqdzb5F0Frg5VOqekjdpcDAUMkHYPBL 4nESSWoCTcc4vKu8rqtgmzIs6c3CupfFhbVuS/cQpcV96OmXiOtzkQ72smR0U1Sf3QUK Wlw+ji8LpktDRKbBR8QL+lIhs40Oip4EvO4SuYHJFatkNVfLGfxIBRFPokWadtv0ROIk dkOAZ42DmIMsN7KvDewEcrinu2BKWUoVYNCz4HmcGlcLfmcLen6d9L1xzsAjD766kxVN HsT8KZqtBtgj4nANr9BN5Zb/UoDuTUeVVahyvmUk6QfDcXmx/F/Vfgg4qu1gEK47Y5ml hiaw== X-Gm-Message-State: AOJu0YxLYs5LSCs1FVagKol/+LIuRbEvubibWnEestWntVBeCGNQ1vzh AdVnW5CEaZURcRCj4oPpxgR6RBJ+u9hghKrfkbKBz6N/mP9n9IaCP/Iauq0H4F7sMH7Ke2+sspx g4wm8V5k= X-Gm-Gg: ASbGncsgn5ZxQIWYxg01k53Aqk7+MmkWLtev81BXfvUeKj51iE7O5fEMfNjgdm8Av9E vuSdgM+DzuQQ+eVGmDmB8C19jTs4QxJtJ8iAKeNfPWwRzyqdALJwI9WJbhTf1UFoPbyxLglfpui XDpAXzOKjgXxrurrLHGP1YLK4QAoVP6SyTw5+C36bsysN6pvi+fKklPIoEJn5JGKofurhMzA8Ey Z2gKAln83r2Xqeh/+QhtSZZyTjDxqX9XW34rH7jmCgYWFBi+zVHAPsNUk4kbC/1lCY8BG02kZo5 vBq+aZmTVyGm/1I/t27NXmWXdiUeebyngvdinQx7ZkD4NhiTJhZH8/wOQkbBLrc/7pItqJ1cjza 3HK6j0WeEkg0oLADiDQbvNZF+cNIJUWF15ulCPIBK5LHQSCNsgg9iZrtiexlSnEa+3dtTqjVg2I InvQb31ANoXI8GyAg7McysLh1m9b7h+eSG7qQbujFQkbKbHvcSJTT6C6RmNcWqquyC1ng= X-Google-Smtp-Source: AGHT+IEmPtujz+inUvV9REl0YbdLSmwuUj/KiK1njySFp2i+qVnTqISaODHpTwM+UouG0/IA6Uk51w== X-Received: by 2002:a05:6000:400f:b0:42b:3c3e:5d53 with SMTP id ffacd0b85a97d-42cc1cedbf7mr78922f8f.16.1763681662475; Thu, 20 Nov 2025 15:34:22 -0800 (PST) Received: from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-42cb7fa3a81sm7984139f8f.26.2025.11.20.15.34.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Nov 2025 15:34:21 -0800 (PST) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: pavel@ucw.cz, Bryan O'Donoghue , Kieran Bingham Subject: [PATCH v4 16/23] libcamera: software_isp: Add a gpuisp todo list Date: Thu, 20 Nov 2025 23:33:40 +0000 Message-ID: <20251120233347.5046-17-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251120233347.5046-1-bryan.odonoghue@linaro.org> References: <20251120233347.5046-1-bryan.odonoghue@linaro.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" List the series of things to do in GPU ISP in perceived order of difficulty. Acked-by: Kieran Bingham Signed-off-by: Bryan O'Donoghue --- src/libcamera/software_isp/gpuisp-todo.txt | 83 ++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 src/libcamera/software_isp/gpuisp-todo.txt diff --git a/src/libcamera/software_isp/gpuisp-todo.txt b/src/libcamera/software_isp/gpuisp-todo.txt new file mode 100644 index 000000000..768dcc320 --- /dev/null +++ b/src/libcamera/software_isp/gpuisp-todo.txt @@ -0,0 +1,83 @@ +List the TODOs in perceived order of ease. + +Version 3: +Use dma-buf handle to generate upload texture: + - eglCreateImageKHR can be used to generate the upload texture i.e. + to feed the bayer data into the GPU. + +Denoising: + - As below still TBD + +Dead pixel correction: + - As below still TBD + +Lense shading correction: + - This is WIP but as yet still TBD + +Lense flare correction: + - Not WIP still TBD + +processFrame() to run in its own thread: + - processFrame() runs in the context of the Debayer::process() + thread. Robert Mader suggested and it seems like a good + suggestion too to run processFrame() in its own thread. + +Version 2: +Make GPUISP default: + - Right now the environment variable allows over-riding to swtich + from CPU to GPU. + - Once we support 24 BPP output on GPUISP we will have the same + pixel format support as CPU and can set the default to GPU without + regressing functionality + +glTexture1D: + - Initial code was developed for < GLES 2.O but since we have fixed + on GLES >= 2.0 this means we can use glTexture1D + - Provided this is so amend the shaders to do val = texture(x, y, 0); + not texture(x, y, 0.5) the 0.5 is because of using glTexture2D + +Denoising: + - Run a denoise algorithm in the shaders + - Supply a control to influence the noise-floor ? + +Dead pixel correction: + - Add logic to correct dead pixels in the fragment shaders + +Version 1: +24 bit output support: + - Take the BPP we already capture and get a 24 bit GBM surface + - Pass a compile-time parameter to the shaders to tell them to do + gl_FragColor = rgb not gl_FragColor = rgba + - Version 2: + This is not possible. + gl_FragColor expects vec4 not vec3 on the output. + If you really want RGB888 run cpuisp. + +Surfaceless GBM: + - We get a GBM surface and then have to swap buffers + If we rework for surfaceless GBM and EGL then the swap buffer can + be dropped. + - Version 2: + Complete GBM surface removed, memcpy() phase removed also + +dma-buf texture upload: + - Currently we pass the input buffer to glCreateTexture2D. + We should be able to make the upload of the input buffer go faster + by using eglCreateImageKHR and enumerated the dma-buf contents. + - Version 2: + Complete sm8250 test platform shows 20x performance increase + with CCM. + +Render-to-texture: + - Right now we render to the GBM provided surface framebuffer + and then memcpy from that buffer to the target output buffer. + This necessitates flushing the cache on the target buffer in + addition to the memcpy(). + - Render-to-texture where we generate the target framebuffer + directly from a dma-buf handle will mitigate the memcpy() phase. + - It should be the case then that the consumer of the output buffer + i.e. the thing that's not libcamera is responsible to flush the cache + if-and-only-if that user writes to the buffer. + - We need to flush the cache on the buffer because we are memcpying() to it. + - Version 2: + Done in version 2