[{"id":37209,"web_url":"https://patchwork.libcamera.org/comment/37209/","msgid":"<85h5u7z2i6.fsf@mzamazal-thinkpadp1gen7.tpbc.csb>","date":"2025-12-03T21:20:17","subject":"Re: [PATCH v6 24/24] libcamera: software_isp: Add a gpuisp todo list","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"content":"https://lists.libcamera.org/pipermail/libcamera-devel/2025-October/053893.html\nstill not addressed.\n\nBryan O'Donoghue writes:\n\n> List the series of things to do in GPU ISP in perceived order of\n> difficulty.\n>\n> Acked-by: Kieran Bingham \n> Signed-off-by: Bryan O'Donoghue \n> ---\n> src/libcamera/software_isp/gpuisp-todo.txt | 83 ++++++++++++++++++++++\n> 1 file changed, 83 insertions(+)\n> create mode 100644 src/libcamera/software_isp/gpuisp-todo.txt\n>\n> diff --git a/src/libcamera/software_isp/gpuisp-todo.txt b/src/libcamera/software_isp/gpuisp-todo.txt\n> new file mode 100644\n> index 000000000..768dcc320\n> --- /dev/null\n> +++ b/src/libcamera/software_isp/gpuisp-todo.txt\n> @@ -0,0 +1,83 @@\n> +List the TODOs in perceived order of ease.\n> +\n> +Version 3:\n> +Use dma-buf handle to generate upload texture:\n> +\t- eglCreateImageKHR can be used to generate the upload texture i.e.\n> +\t to feed the bayer data into the GPU.\n> +\n> +Denoising:\n> +\t- As below still TBD\n> +\n> +Dead pixel correction:\n> +\t- As below still TBD\n> +\n> +Lense shading correction:\n> +\t- This is WIP but as yet still TBD\n> +\n> +Lense flare correction:\n> +\t- Not WIP still TBD\n> +\n> +processFrame() to run in its own thread:\n> +\t- processFrame() runs in the context of the Debayer::process()\n> +\t thread. Robert Mader suggested and it seems like a good\n> +\t suggestion too to run processFrame() in its own thread.\n> +\n> +Version 2:\n> +Make GPUISP default:\n> +\t- Right now the environment variable allows over-riding to swtich\n> +\t from CPU to GPU.\n> +\t- Once we support 24 BPP output on GPUISP we will have the same\n> +\t pixel format support as CPU and can set the default to GPU without\n> +\t regressing functionality\n> +\n> +glTexture1D:\n> +\t- Initial code was developed for < GLES 2.O but since we have fixed\n> +\t on GLES >= 2.0 this means we can use glTexture1D\n> +\t- Provided this is so amend the shaders to do val = texture(x, y, 0);\n> +\t not texture(x, y, 0.5) the 0.5 is because of using glTexture2D\n> +\n> +Denoising:\n> +\t- Run a denoise algorithm in the shaders\n> +\t- Supply a control to influence the noise-floor ?\n> +\n> +Dead pixel correction:\n> +\t- Add logic to correct dead pixels in the fragment shaders\n> +\n> +Version 1:\n> +24 bit output support:\n> +\t- Take the BPP we already capture and get a 24 bit GBM surface\n> +\t- Pass a compile-time parameter to the shaders to tell them to do\n> +\t gl_FragColor = rgb not gl_FragColor = rgba\n> +\t- Version 2:\n> +\t This is not possible.\n> +\t gl_FragColor expects vec4 not vec3 on the output.\n> +\t If you really want RGB888 run cpuisp.\n> +\n> +Surfaceless GBM:\n> +\t- We get a GBM surface and then have to swap buffers\n> +\t If we rework for surfaceless GBM and EGL then the swap buffer can\n> +\t be dropped.\n> +\t- Version 2:\n> +\t Complete GBM surface removed, memcpy() phase removed also\n> +\n> +dma-buf texture upload:\n> +\t- Currently we pass the input buffer to glCreateTexture2D.\n> +\t We should be able to make the upload of the input buffer go faster\n> +\t by using eglCreateImageKHR and enumerated the dma-buf contents.\n> +\t- Version 2:\n> +\t Complete sm8250 test platform shows 20x performance increase\n> +\t with CCM.\n> +\n> +Render-to-texture:\n> +\t- Right now we render to the GBM provided surface framebuffer\n> +\t and then memcpy from that buffer to the target output buffer.\n> +\t This necessitates flushing the cache on the target buffer in\n> +\t addition to the memcpy().\n> +\t- Render-to-texture where we generate the target framebuffer\n> +\t directly from a dma-buf handle will mitigate the memcpy() phase.\n> +\t- It should be the case then that the consumer of the output buffer\n> +\t i.e. the thing that's not libcamera is responsible to flush the cache\n> +\t if-and-only-if that user writes to the buffer.\n> +\t- We need to flush the cache on the buffer because we are memcpying() to it.\n> +\t- Version 2:\n> +\t Done in version 2","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5D729C3257\n\tfor ;\n\tWed, 3 Dec 2025 21:20:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 97ED4610A1;\n\tWed, 3 Dec 2025 22:20:24 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 22EE0609D8\n\tfor ;\n\tWed, 3 Dec 2025 22:20:23 +0100 (CET)","from mail-wm1-f69.google.com (mail-wm1-f69.google.com\n\t[209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-41-Fj6RCekhPdW3hMJWA9jWww-1; Wed, 03 Dec 2025 16:20:21 -0500","by mail-wm1-f69.google.com with SMTP id\n\t5b1f17b1804b1-477cf25ceccso1433845e9.0\n\tfor ;\n\tWed, 03 Dec 2025 13:20:20 -0800 (PST)","from mzamazal-thinkpadp1gen7.tpbc.csb\n\t(ip-77-48-47-2.net.vodafone.cz. [77.48.47.2])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-4792b14ad9dsm24803445e9.12.2025.12.03.13.20.18\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 03 Dec 2025 13:20:18 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"Ng1867TA\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1764796822;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=x7XVgoy2kHByrRUIPw1UZcgGcgYr9ujJDJ9qHidGmQ8=;\n\tb=Ng1867TAFkSvK7shaPeNHLHlJ+zRldDOzlQ0JqStEx+1WncvMD2273OEOXE8e6yfg/698L\n\tIvWeeoRKZd9aPp5upcDyIfo1rctH5CNE4HAl5Mj0ewdBtcHJfPGSY+HOvCDw6kAIPunhTj\n\tnoUrf3vsdZ9E8ScJGWLEpNFnVR8USyI=","X-MC-Unique":"Fj6RCekhPdW3hMJWA9jWww-1","X-Mimecast-MFC-AGG-ID":"Fj6RCekhPdW3hMJWA9jWww_1764796820","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1764796819; x=1765401619;\n\th=mime-version:user-agent:message-id:date:references:in-reply-to\n\t:subject:cc:to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject\n\t:date:message-id:reply-to;\n\tbh=x7XVgoy2kHByrRUIPw1UZcgGcgYr9ujJDJ9qHidGmQ8=;\n\tb=fEL3ry1+i+SwW8DocJCoo2p6Pea1BnjYOcHWxaucCfLfmDMGazVaBBEyvcKlM88jdJ\n\tZosPsSMRwF3nyGUxeFV1CVlvtAklqu7feFOgwjUpkD7HpJ3eTYJIv7y7BUWNi5q5yhoa\n\tlkh/nA6v0W9F2iPB7FBNiKsd2r1UW7dhCJmvwEhvqjZWmuRpx0gQB7YgwIh3lkrsUjRu\n\tDLX+ydDzieQZkqPe0i7iX72UUv0nK5bD6Sa6jZ2unZ4lVYhv5IbiIjpHEwII+heI6TOm\n\t8X5OXyi1weCRG4FFtpprVqgvq8EfVoL/G8OvuHrIGBhGkeRc9jCeJ7qRjMu69P0O3tzV\n\t+IxQ==","X-Gm-Message-State":"AOJu0Yzi/okO+L584Q9DflB+XcUJ1SOGJIiEU6sAdfXDK/yFsS1zj7kx\n\tTYbfML9ff0VikOmmJfryXBDhrLrGOF7fdJJB++4SOIFB6zgqK40VbTFbNLKG/TXrDMdvUmyyCka\n\tOFbWO7/AMGdZpaZAibTCbJceYMJ3XSxHysLEd2GkLr44zhmciTGInsGvn0XQv4uv37BipR28+wk\n\tjpUuh1qaU=","X-Gm-Gg":"ASbGncvPBPrkwiKAODChoK4FV+sjXMVPDF4fdC7wab6EQY+tPvy7ngB9HGuB5++wxFE\n\tTKY34ja/j85Gjwf/vNJOPmW0cykoz92N+RXv2EkmfUuHTvl/yYrowpPQu58Z3DpLA1pJwCCj3RB\n\tPtTcXP4Tvv7KNwmGEPBwHQz4sNpPQeBjr4246CFv8SHq9s1wJcp/P8GllOuz7jLflh7RA8WNVQi\n\t3SQZ5S+twobsSF3WxSCOxkxF5rLaO5ASDVRn1Eq+x7bN3QOZdnw639EyH5LPiTok/1PFSpLAm/G\n\t614be7e+pcP0deJkPAXq6Evu+It1wsGKTWgn3kBKHxtmReU3HCap+IaGKui1SVRIv2kU8LgMtAO\n\twxipQ59YBaGDIpTOV1MSEpeSljmIyA6j6p0Zdqq9JO5q7OiPF3mJpdLJGrGWGkGY=","X-Received":["by 2002:a05:600c:1d0e:b0:477:fcb:2256 with SMTP id\n\t5b1f17b1804b1-4792f386c42mr4440135e9.17.1764796819485; \n\tWed, 03 Dec 2025 13:20:19 -0800 (PST)","by 2002:a05:600c:1d0e:b0:477:fcb:2256 with SMTP id\n\t5b1f17b1804b1-4792f386c42mr4439965e9.17.1764796819034; \n\tWed, 03 Dec 2025 13:20:19 -0800 (PST)"],"X-Google-Smtp-Source":"AGHT+IGl99ZoF9llXpb/TEPnun8+ztxvl6jdbrmhTZOxb20FEyre8cVYbXfl5IXCO8pI1kcJwIihPA==","From":"Milan Zamazal ","To":"Bryan O'Donoghue ","Cc":"libcamera-devel@lists.libcamera.org, pavel@ucw.cz, Kieran Bingham\n\t","Subject":"Re: [PATCH v6 24/24] libcamera: software_isp: Add a gpuisp todo list","In-Reply-To":"<20251202134544.662446-25-bryan.odonoghue@linaro.org> (Bryan\n\tO'Donoghue's message of \"Tue, 2 Dec 2025 13:45:44 +0000\")","References":"<20251202134544.662446-1-bryan.odonoghue@linaro.org>\n\t<20251202134544.662446-25-bryan.odonoghue@linaro.org>","Date":"Wed, 03 Dec 2025 22:20:17 +0100","Message-ID":"<85h5u7z2i6.fsf@mzamazal-thinkpadp1gen7.tpbc.csb>","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"rPV4vEuyzLFnFk0PvuxGyikV8dSZy9imitx23DYdEsQ_1764796820","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]