[{"id":34443,"web_url":"https://patchwork.libcamera.org/comment/34443/","msgid":"<74bf48fb-7aed-4dff-a358-872555f513da@nxsw.ie>","date":"2025-06-11T01:41:29","subject":"Re: [PATCH 00/35] Add GLES 2.0 GPUISP to libcamera","submitter":{"id":226,"url":"https://patchwork.libcamera.org/api/people/226/","name":"Bryan O'Donoghue","email":"bod.linux@nxsw.ie"},"content":"On 11/06/2025 02:32, Bryan O'Donoghue wrote:\n> This series introduces a GLES 2.0 GPU ISP to libcamera.\n> \n> We have had extensive discussions, meetings and collaborative discussions\n> about this topic over the last year or so.\n> \n> As an overview we want to start to move as much processing of software_isp\n> into the GPU as possible. This is especially advantageous when we are\n> talking about processing a framebuffer's worth of pixels as quickly as\n> possible.\n> \n> The decision to use GLES 2.0 instead of say Vulcan stems from a desire to\n> support as much in the way of older hardware as possible and the fact we\n> already have upstream GLES 2.0 fragment shaders to do debayer.\n> \n> Generally the approach is\n> \n> - Move the fragment shaders out of qcam and into a common location\n> - Update the existing SoftwareISP Debayer/DebayerCPU pair to facilitate\n> addition of a new class DebayerEGL.\n> - Introduce that class\n> - Then do progressive change of the shaders and DebayerEGL class to make\n> the modifications as transparent as possible in the git log.\n> - Reuse as much of the SoftIPA data-structures and logic as possible.\n> - Consume the data from SoftIPA in the Debayer Shaders so that CPUISP and\n> GPUISP give similar - hopefully the same results but with GPUISP going\n> faster.\n> \n> In order to get untiled and uncompressed pixel data out of the GPU\n> framebuffer we need to tell the GPU how to store the data it is writing to\n> that framebuffer. GPUs can store their framebuffer data in tiled or even\n> compressed formats which is why the naive approach of running your fragment\n> shader and then using glReadPixels(GL_RGBA); will be horrendously slow as\n> glReadPixels must convert from the internal GPU format to the requested\n> output format - an operation that for me takes ~ 10 milliseconds per frame.\n> \n> Instead we get the GPU to store its data as ARGB8888 swap buffers and\n> memcpy() from the swapped buffer to our output frame. Right now this series\n> supports 32 bit output formats only.\n> \n> The memcpy() also entails flushing the cache of the target buffer as per\n> the terms of the dma-buf software contract.\n> \n> This leads us onto the main outstanding TODOs\n> \n> - 24 bit GBM buffer support leading\n> - 24 bit output framebuffer support\n> - Surfaceless GBM and eGL context with no swapbuffer\n> - Render to texture\n> If we render directly to a buffer provided to the GPU the output\n> buffer we will not need to memcpy() to the output buffer\n> nor will we need to invalidate the output buffer cache.\n> - eglCreateImageKHR for the texture upload.\n> \n> This list is of the colour \"make it go faster\" not \"make it work\" which is\n> why we are moving to start to submit a v1 for discussion in the full\n> realisation it will have to go through several cycles of review giving us\n> the opportunity to fix:\n> \n> - Doxygen is missing for new classes and methods\n> - Some of the pipelines don't complete in gitlab\n> - 24 bit output seems doable before merge\n> - Render to texture perhaps even too\n> \n> For me on my Qualcomm hardware GPUISP works very well I get 30fps in qcam\n> with about 75% CPU usage versus > 100% - cam goes faster which to me\n> implies a good bit of time is being consumed in qcam itself.\n> \n> The series starts out with fixes and updates from Hans and finishes it out\n> with shader modifications from Milan both of whom along with Kieran,\n> Laurent and Maxime I'd like to thank for being some helpful and patient.\nOh I forgot to mention, you need an environment variable to run with \nGPUISP until we make it the default.\n\n% LIBCAMERA_SOFTISP_MODE=gpu ./builds/build.master.dbg/src/apps/qcam/qcam\n\n---\nbod","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 19E1FC3237\n\tfor ;\n\tWed, 11 Jun 2025 01:41:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 997CD68DBF;\n\tWed, 11 Jun 2025 03:41:35 +0200 (CEST)","from mail-24421.protonmail.ch (mail-24421.protonmail.ch\n\t[109.224.244.21])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 76FDD68DB2\n\tfor ;\n\tWed, 11 Jun 2025 03:41:34 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=nxsw.ie header.i=@nxsw.ie header.b=\"hnBuXZ8e\";\n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxsw.ie;\n\ts=protonmail2; t=1749606093; x=1749865293;\n\tbh=aKHykk9RM0Mb46Sg4osV8v/8UwMM+ppjCN35vMcFGCg=;\n\th=Date:To:From:Subject:Message-ID:In-Reply-To:References:\n\tFeedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:\n\tMessage-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post;\n\tb=hnBuXZ8eSq+YGc+uBYkan4HfUvm5D6t0aZe6IUR10foSFcjupGC4eZWMDgZtlkPQr\n\t/ACdutoFDbRN6NIBu/I6YDIh4Tnbj5qCs1Xs3ZLM6c8ms8+q2A6sjVTHcqksDI5vRO\n\tVDh5TZvQeeEJ+1SfCD2TGkJakLGtoLmrlOEXuvQpuVNpuPbUWLnA1ZvNWh5f+G62DW\n\t5kumYmULDLaoz7xKfmb8ObW5j1e7o6DWNMaexY3InK+Otsbypzegcwy99jEZIRX6mV\n\t+yZ8tN4zaOtSQSMOawPxXMrfNLgBiqZ3AQqYbnHvztY7pCSRqZEk0Uf8W6Q9/l2TIg\n\tp932OIQVigiOA==","Date":"Wed, 11 Jun 2025 01:41:29 +0000","To":"Bryan O'Donoghue ,\n\tlibcamera-devel@lists.libcamera.org","From":"Bryan O'Donoghue ","Subject":"Re: [PATCH 00/35] Add GLES 2.0 GPUISP to libcamera","Message-ID":"<74bf48fb-7aed-4dff-a358-872555f513da@nxsw.ie>","In-Reply-To":"<20250611013245.133785-1-bryan.odonoghue@linaro.org>","References":"<20250611013245.133785-1-bryan.odonoghue@linaro.org>","Feedback-ID":"136405006:user:proton","X-Pm-Message-ID":"84d874bcc130dd72d659068d22a83cf855fc0ffe","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]