Cover Letter Detail
Show a cover letter.
GET /api/covers/25124/?format=api
{ "id": 25124, "url": "https://patchwork.libcamera.org/api/covers/25124/?format=api", "web_url": "https://patchwork.libcamera.org/cover/25124/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20251120233347.5046-1-bryan.odonoghue@linaro.org>", "date": "2025-11-20T23:33:24", "name": "[v4,00/23] Add GLES 2.0 GPUISP to libcamera", "submitter": { "id": 175, "url": "https://patchwork.libcamera.org/api/people/175/?format=api", "name": "Bryan O'Donoghue", "email": "bryan.odonoghue@linaro.org" }, "mbox": "https://patchwork.libcamera.org/cover/25124/mbox/", "series": [ { "id": 5600, "url": "https://patchwork.libcamera.org/api/series/5600/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=5600", "date": "2025-11-20T23:33:24", "name": "Add GLES 2.0 GPUISP to libcamera", "version": 4, "mbox": "https://patchwork.libcamera.org/series/5600/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/covers/25124/comments/", "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7109DBD80A\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu, 20 Nov 2025 23:33:55 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8F7BF60A80;\n\tFri, 21 Nov 2025 00:33:54 +0100 (CET)", "from mail-wm1-x332.google.com (mail-wm1-x332.google.com\n\t[IPv6:2a00:1450:4864:20::332])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 761E5606A0\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 21 Nov 2025 00:33:52 +0100 (CET)", "by mail-wm1-x332.google.com with SMTP id\n\t5b1f17b1804b1-4779a637712so9621365e9.1\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 20 Nov 2025 15:33:52 -0800 (PST)", "from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie.\n\t[188.141.3.146]) by smtp.gmail.com with ESMTPSA id\n\tffacd0b85a97d-42cb7fa3a81sm7984139f8f.26.2025.11.20.15.33.50\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 20 Nov 2025 15:33:50 -0800 (PST)" ], "Authentication-Results": "lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"ikF7t+uU\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=linaro.org; s=google; t=1763681632; x=1764286432;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:from:to:cc:subject:date:message-id:reply-to;\n\tbh=EJLYDsJ38GKbPIQSXX+3rEPVGumkIgDBgAEv9Cd79hM=;\n\tb=ikF7t+uUKK5ZNmrxt2phOWYthS59ClyFoVd0GcYYSS7d+zrXWxPJhm/0k3KG9Yy/pg\n\trwLWQDhYxfIEpnUZXqviB8FBvsS3c+tKpobxjLFi4tWSTLr1bR3DedQGxKr8tTUp/qLi\n\tIcNYcvdveSN4r6vgGqqSx3NJvPg32JewDoBJ6HYRznokr+A5qxd0Rk08tAcu62RSsJGj\n\tfm2KX/+LQq70VMvLOZTUtYqeB5z/eHjuGB1Ndl5TuFajHAGFyPRBeyrnQSWEq0gdk1RR\n\tc0q5HZMRM35ii+d9NyD+WuBe9MmbxEJFaqQBwTFwp2IOTbMr/tL2vZXKSIAQG4lOYNIh\n\tHW4g==", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1763681632; x=1764286432;\n\th=content-transfer-encoding:mime-version:message-id:date:subject:cc\n\t:to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=EJLYDsJ38GKbPIQSXX+3rEPVGumkIgDBgAEv9Cd79hM=;\n\tb=UyjKA72PJK8We/aOMpfYcwznaEaiCPqhtVP7wTaf8fWn1iwErim4fZNcFEgY8E2ToZ\n\tEjpr+3vAyuRzHWktp0TUum3xKCZ6gOhorQgnsLa0q0JfhwAbWFfSIN+eynh30KbyTNnh\n\tJMj4soBEB08Rwf1cMdmlfUQ4+gV11xht1NZFu2eZP/wmFXlceTFe8rD0baKQfDenUD2H\n\tKuRfoUlACuAoxSPFK8BYmtU9SloGce533ubI7imyHMaeKLEGjcjWRUez7O0j4zNpYZMw\n\tZgPb50JAz+xk2SBGVNhCNVQwoCxMtoiHeJC6RJzrWbOtPyaxWC8jlxoZRUJYTuna24ui\n\tE2PA==", "X-Gm-Message-State": "AOJu0Yw1ddvMIM3pUMDO8soHFoZSyKoQAYLjfTMtn+WRdE0WhkihPKng\n\tg9RaUuVDaDsd/8EM44P2FvpbqWAWJaollhFg5SerW4yyTv82a65t1bXELTge2S/efGUVgkNqmqa\n\tci0PuOtI=", "X-Gm-Gg": "ASbGnctqI3kUMVa9U1xkejbwVUoESe8QgU4pvMYftr1RG3ADiI1sNQkXPE7hRSn+OLM\n\tlQgHSRbQ7SwMVtRmFxPxUK67arRpvLu8ey1F1h31Q8Ohq/MJbNrR57SxZ/lQh6MzaOKbX48NKj0\n\tLhwZv59GZr0uSVWrBdB8Ef4Kf6e6EvEhI3ZI/1HRcLLqn/yYru5x+JLmA3kUhz1XMktS/3BU3ID\n\t67FEeafs9LPRuNu8EOVn+JQxeXuLzzBMNFHtwtOWqbGl7/1gL8SqBuzUcdry2Ku05GTkXjcKr3n\n\tAZ6dqfaaxTE2vFXceJ7oRSx+W0KPMqlC9vgDl012ab7UXRzH0VEENqp6zq5aexTX5g20Pmftl3a\n\taXYNKh5IvBMA2OPIiOumvA0vo/5Llec4sGk26sl1evwboi2YZt4KwXU9eodomJqk5vESVKJGJoU\n\tNbvf3Vj8Awp6sQUJmlndUQe/2/YSdgYmnv39MOQUH4xTA086azLNF8oXuVDH6S8gAJAUc=", "X-Google-Smtp-Source": "AGHT+IFGDezd3cRKzFrJCaNSZosITv9odEt7D1S6MpngvjccXkjWJs5wy5GFIZEwsop8SGF4HEKrkg==", "X-Received": "by 2002:a05:600c:1ca0:b0:477:75eb:a643 with SMTP id\n\t5b1f17b1804b1-477c0165b4emr4377795e9.4.1763681631533; \n\tThu, 20 Nov 2025 15:33:51 -0800 (PST)", "From": "Bryan O'Donoghue <bryan.odonoghue@linaro.org>", "To": "libcamera-devel@lists.libcamera.org", "Cc": "pavel@ucw.cz,\n\tBryan O'Donoghue <bryan.odonoghue@linaro.org>", "Subject": "[PATCH v4 00/23] Add GLES 2.0 GPUISP to libcamera", "Date": "Thu, 20 Nov 2025 23:33:24 +0000", "Message-ID": "<20251120233347.5046-1-bryan.odonoghue@linaro.org>", "X-Mailer": "git-send-email 2.51.2", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "This version 4:\n\n- Drops AWB since the CCM contains it already\n- Includes Gamma\n- Includes Contrast - testable via camshark\n- Includes Saturation - testable via camshark\n- Includes a scaler from Robert\n- Includes synch changes from Robert\n- Includes all feedback incorporated from Pavel\n- Generates a default 65k CCM if none is supplied\n- Various Doxygen torments fixed along the way\n- And is the \"top half\" of the precursor series as the GPUISP\n series becomes 44 patches long this is an unreasonable number\n to merge in one go.\n\n- Full testable branch\nLink: https://gitlab.freedesktop.org/camera/libcamera-softisp/-/tree/v0.5.2-gpuisp-v4e?ref_type=heads\n\n- The first part of the series is in the precurso here\nLink: https://gitlab.freedesktop.org/camera/libcamera-softisp/-/commits/v0.5.2-gpuisp-v4e-split\n\nThat precursor is just a tag about half way through the integrated series.\n\nThis version 3:\n\n- Adds AWB to the debayer routine as calculated by the IPA thread\n\n- Implements ~ all of the feedback from Barnabas quicker to mention\n what hasn't been done.\n a) A comment about member initialisation in eGL.cpp\n code I wrote to make constructor init common seemed to negate\n that ask.\n b) meson dependency checks for egl.\n I remember struggling with this earlier on in development.\n I will certainly try to do this for a v4 so its more\n pending a try as opposed to not indended to be done.\n\n- Incorporates various fixes from Robert Mader\n When to sync removing tearing for Milan\n Some error checking that although Robert didn't mention in his\n feedback were in his patches so I stole that code. Thanks.\n\n- Also worth mentioning Robert identified a permissions fix\n that pipewire would need for eGL to work in libcamera with pipewire\n published that fix and got it merged too.\n\n Owe you a beer for that one.\n\n- Is rebased on tip-of-tree\n\n- Currently the documentation checks for the various classes\n don't pass but that is easy enough to fix in a V4.\n\n- In line with our discussions gpuisp is now the default instead of cpuisp.\n\n- Since its only the documentation checks that are pending I thought\n rather than delay further it was time to publish the series without\n and see if anything major gets snagged.\n\nv2:\n\nThis version 2 is an incomplete update with-respect-to previous comment\nfeedback, which ordinarily I would not publish however, given OSSEU is\nstarting on Monday and we have talk about this topic, in addition to some\npretty good progress in the interregnum I thought a v2 would be\nappropriate.\n\n- V2 drops use of GBM surface in favour of generating a framebuffer from\n the dma-buf handle, called render-to-texture.\n\n The conversion from GBM surface + memcpy() including the associated cache\n invalidate has a dramatic effect on GPUISP performance.\n\n Some rough stats for a Qualcomm sm8250 \"kona\" device with an imx517\n sensor @ 4048 x 3040 ABRG8888 - debug builds\n\n CPUISP + CCM:\n 2 FPS CPU usage > 100% single core pulls about 9 watts\n\n GPUISP v1 + CCM:\n 14 FPS - power not measured\n\n GPUISP v2 + CCM:\n 30 FPS - sensor linerate - CPU usage ~ 70 % pulling 8 Watts.\n\n Milan Zamal has reported a TI AM69 + imx219 - unknown resolution\n\n CPUISP 4 FPS\n GPUISP v2 - 2 or 3 FPS\n GPUISP v2 - 15 FPS =3D=3D sensor linerate\n\n In other words for these boards we can hit linerate with GPUISP + 3A +\n CCM.\n\n- Drop GBM surface rendering\n- Drop swapbuffers\n- Use eglCreateImageKHR to directly render into the output dma-buf buffer\n eglCreateImageKHR lets you specify the FOURCC of the texture which means\n we can create the texture in the uncompressed target output pixel format\n we want.\n- Fix stride calculation to 256 bytes\n Laurent and Maxime explained to me about GPU stride alignments being\n tribal wisdom and that 256 bytes is a good cross-platform value.\n This helped to get the render-to-texture command right.\n- A synchronous blocking wait is used to ensure GPU operations have\n completed. Laurent wants this to be made async.\n At the moment its not clear to me the eglWaitSyncKHR is really required\n and in any case doesn't seem to have any performance impact.\n But this part is still TBD - I've included the sync wait for simplicity\n and safety.\n- A Debayer::stop() method has been introduced to ensure we call\n eglDestroySyncKHR when the eGL context is valid, as opposed to in the\n callchain of destructors triggering eGL::~eGL();\n- stats move constructor call chain dropped - Branabas\n- Incorporates Milan's area-of-interest constraint for Bayer stats\n i.e. squashes his v3 update into debayer_egl.cpp directly\n- Moves ALIGN_TO into a common area to facilitate its reuse in\n egl.cpp\n- Rebases on 0.5.2\n\n- There are a number of known checks failing on the CI loop right now\n\nLink to v1: https://lists.libcamera.org/pipermail/libcamera-devel/2025-June=\n/050692.html\n\nv1:\nThis series introduces a GLES 2.0 GPU ISP to libcamera.\n\nWe have had extensive discussions, meetings and collaborative discussions\nabout this topic over the last year or so.\n\nAs an overview we want to start to move as much processing of software_isp\ninto the GPU as possible. This is especially advantageous when we are\ntalking about processing a framebuffer's worth of pixels as quickly as\npossible.\n\nThe decision to use GLES 2.0 instead of say Vulcan stems from a desire to\nsupport as much in the way of older hardware as possible and the fact we\nalready have upstream GLES 2.0 fragment shaders to do debayer.\n\nGenerally the approach is\n\n- Move the fragment shaders out of qcam and into a common location\n- Update the existing SoftwareISP Debayer/DebayerCPU pair to facilitate\n addition of a new class DebayerEGL.\n- Introduce that class\n- Then do progressive change of the shaders and DebayerEGL class to make\n the modifications as transparent as possible in the git log.\n- Reuse as much of the SoftIPA data-structures and logic as possible.\n- Consume the data from SoftIPA in the Debayer Shaders so that CPUISP and\n GPUISP give similar - hopefully the same results but with GPUISP going\n faster.\n\nIn order to get untiled and uncompressed pixel data out of the GPU\nframebuffer we need to tell the GPU how to store the data it is writing to\nthat framebuffer. GPUs can store their framebuffer data in tiled or even\ncompressed formats which is why the naive approach of running your fragment\nshader and then using glReadPixels(GL_RGBA); will be horrendously slow as\nglReadPixels must convert from the internal GPU format to the requested\noutput format - an operation that for me takes ~ 10 milliseconds per frame.\n\nInstead we get the GPU to store its data as ARGB8888 swap buffers and\nmemcpy() from the swapped buffer to our output frame. Right now this series\nsupports 32 bit output formats only.\n\nThe memcpy() also entails flushing the cache of the target buffer as per\nthe terms of the dma-buf software contract.\n\nThis leads us onto the main outstanding TODOs\n\n- 24 bit GBM buffer support leading\n- 24 bit output framebuffer support\n- Surfaceless GBM and eGL context with no swapbuffer\n- Render to texture\n If we render directly to a buffer provided to the GPU the output\n buffer we will not need to memcpy() to the output buffer\n nor will we need to invalidate the output buffer cache.\n- eglCreateImageKHR for the texture upload.\n\nThis list is of the colour \"make it go faster\" not \"make it work\" which is\nwhy we are moving to start to submit a v1 for discussion in the full\nrealisation it will have to go through several cycles of review giving us\nthe opportunity to fix:\n\n- Doxygen is missing for new classes and methods\n- Some of the pipelines don't complete in gitlab\n- 24 bit output seems doable before merge\n- Render to texture perhaps even too\n\nFor me on my Qualcomm hardware GPUISP works very well I get 30fps in qcam\nwith about 75% CPU usage versus > 100% - cam goes faster which to me\nimplies a good bit of time is being consumed in qcam itself.\n\nThe series starts out with fixes and updates from Hans and finishes it out\nwith shader modifications from Milan both of whom along with Kieran,\nLaurent and Maxime I'd like to thank for being some helpful and patient.\n\nBryan O'Donoghue (21):\n libcamera: software_isp: gbm: Add a GBM helper class for GPU surface\n access\n libcamera: software_isp: Make isStandardBayerOrder static\n libcamera: software_isp: egl: Add a eGL base helper class\n libcamera: shaders: Use highp not mediump for float precision\n libcamera: shaders: Extend debayer shaders to apply RGB gain values on\n output\n libcamera: shaders: Extend bayer shaders to support swapping R and B\n on output\n libcamera: shaders: Add support for black level compenstation\n libcamera: shaders: Add support for Gamma\n libcamera: shaders: Add support for contrast\n libcamera: software_isp: debayer_egl: Add an eGL debayer class\n libcamera: software_isp: debayer_egl: Make DebayerEGL an environment\n option\n libcamera: software_isp: debayer_egl: Make gpuisp default softisp mode\n libcamera: software_isp: debayer_cpu: Make getInputConfig and\n getOutputConfig static\n libcamera: software_isp: Add a gpuisp todo list\n libcamera: software_isp: lut: Change default Gamma to 1.0/2.2\n ipa: Add a new Algorithm::init() to support self-initalising\n algorithms\n libcamera: software_isp: Implement a static init() routine\n ipa: simple: Add a flag to indicate gpuIspEnabled\n ipa: libipa: module: Add createSelfEnumeratingAlgorithm\n ipa: software_isp: Call createSelfEnumeratingAlgorithm() to statically\n instantiate CCM algo\n libcamera: software_isp: lut: Skip calculation lookup tables if\n gpuIspEnabled is true.\n\nMilan Zamazal (2):\n libcamera: shaders: Rename bayer_8 to bayer_unpacked\n libcamera: software_isp: GPU support for unpacked 10/12-bit formats\n\n include/libcamera/internal/egl.h | 412 +++++++++++\n include/libcamera/internal/gbm.h | 84 +++\n include/libcamera/internal/meson.build | 1 +\n include/libcamera/internal/shaders/RGB.frag | 2 +-\n .../internal/shaders/YUV_2_planes.frag | 2 +-\n .../internal/shaders/YUV_3_planes.frag | 2 +-\n .../internal/shaders/YUV_packed.frag | 2 +-\n .../internal/shaders/bayer_1x_packed.frag | 89 ++-\n .../{bayer_8.frag => bayer_unpacked.frag} | 105 ++-\n .../{bayer_8.vert => bayer_unpacked.vert} | 0\n .../libcamera/internal/shaders/meson.build | 4 +-\n include/libcamera/ipa/soft.mojom | 2 +-\n src/apps/qcam/assets/shader/shaders.qrc | 4 +-\n src/apps/qcam/viewfinder_gl.cpp | 16 +-\n src/ipa/libipa/algorithm.cpp | 13 +-\n src/ipa/libipa/algorithm.h | 5 +\n src/ipa/libipa/module.h | 41 ++\n src/ipa/simple/algorithms/ccm.cpp | 18 +\n src/ipa/simple/algorithms/ccm.h | 1 +\n src/ipa/simple/algorithms/lut.cpp | 72 +-\n src/ipa/simple/ipa_context.h | 1 +\n src/ipa/simple/soft_simple.cpp | 13 +-\n src/libcamera/egl.cpp | 436 ++++++++++++\n src/libcamera/gbm.cpp | 61 ++\n src/libcamera/meson.build | 34 +\n src/libcamera/software_isp/debayer.h | 2 +-\n src/libcamera/software_isp/debayer_cpu.h | 4 +-\n src/libcamera/software_isp/debayer_egl.cpp | 668 ++++++++++++++++++\n src/libcamera/software_isp/debayer_egl.h | 177 +++++\n src/libcamera/software_isp/gpuisp-todo.txt | 83 +++\n src/libcamera/software_isp/meson.build | 8 +\n src/libcamera/software_isp/software_isp.cpp | 35 +-\n 32 files changed, 2334 insertions(+), 63 deletions(-)\n create mode 100644 include/libcamera/internal/egl.h\n create mode 100644 include/libcamera/internal/gbm.h\n rename include/libcamera/internal/shaders/{bayer_8.frag => bayer_unpacked.frag} (50%)\n rename include/libcamera/internal/shaders/{bayer_8.vert => bayer_unpacked.vert} (100%)\n create mode 100644 src/libcamera/egl.cpp\n create mode 100644 src/libcamera/gbm.cpp\n create mode 100644 src/libcamera/software_isp/debayer_egl.cpp\n create mode 100644 src/libcamera/software_isp/debayer_egl.h\n create mode 100644 src/libcamera/software_isp/gpuisp-todo.txt" }