| Message ID | 20261118095735.170587-1-bryan.odonoghue@linaro.org |
|---|---|
| Headers | show
Return-Path: <libcamera-devel-bounces@lists.libcamera.org> X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 13F98BDE17 for <parsemail@patchwork.libcamera.org>; Wed, 18 Nov 2026 09:57:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3F3EB6A8AA; Wed, 18 Nov 2026 10:57:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="cP1suX2P"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3FD336A7EA for <libcamera-devel@lists.libcamera.org>; Wed, 18 Nov 2026 10:57:46 +0100 (CET) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-4a42b34e930so525475f8f.2 for <libcamera-devel@lists.libcamera.org>; Wed, 18 Nov 2026 01:57:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1794995866; x=1795600666; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=ySYFuTvV6EAojSTlT14dZwpgxrbpdOjynDRpkTBkboM=; b=cP1suX2PJzfcYn0csvFs7k9vqeNvA64kGqGX2DdUQMKmx/uCoYmNUHt0OcW4qRouuw rAxWoqi9sr3Gbn07D/cX/EIuAL5IKulP67DABDzCm793W6qVrWP/ZbUtenLSjRZl/kiO 95s1MLAOhRxO4cDr7q+HDf5AW+z3fuJ+aSeGqQTwLDQDmxryeGQYaUcRQWUfs0Wt1KdS VhCJfnxOjzETko/tzqIv3w6bmHUnE1X3MlYpXq3EdVkXup7C27JDNeFqY2S1TI3ToMwd LuWhrjJD+bEqY+LJkszT6Rx9+t5HDW1xn1X2Q6UAiJAdjktxiQrcG4T+UxokO5cXE8Mm QUzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794995866; x=1795600666; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ySYFuTvV6EAojSTlT14dZwpgxrbpdOjynDRpkTBkboM=; b=meQSn2lkmHyWkwhir8pk+fz6l4cTA3pAFYFCmBRBbumYSQHWkOmjeK5Wov1hPzT0dz oto3LsNALZ229tbw6AvCKNqsEvLRtv/UOG/ayE7yT2CH/COSHDZrqiaSdz/xzXyal5f6 zswPKgjucvt5t9S5BYTRdpRH3D5pW8Q6WZXFwci5Sm6oNK1g+dymX0NGs3L+63G4GxP3 dDXqjKXixI7B7vS1wxDpNJwroPpjROAk/qHUW5T099EjAQ22HkqH3g61gIe5V5XCJie9 6h2u7+Iz4JekeSDHeMTe31Puy0jUAH8dgimXQlEWf7zjwv5Uno1zYuy9X+yPKSLxI3LA ofSw== X-Gm-Message-State: APuU4Z+zgTMOsO6OITTBXuSr7wMmVuRIEInk+1GGc08LfaMe5GwDwvH4 294gMaWUkRsC0Z9MMhBuyaZSN/bWoEvxKM5f+B7MbyyzwfcfrG9bQGPVS0R9pkMs29ZnTTSXAsa iw0QxAvs= X-Gm-Gg: AZFdTZeyHD1cgxrfV/hh1kYhR5czVG+6+B5okt4LRuhne2KUjBhiF/YTaaerBr5lhCY 4WhLRzRmcjH5m3Y0F5wZ4lrDYaGrr+NaLTQPKKhHjZnvZoyTHqKmwotRtgRUJipk9vmXUXiQf3v NU3T/4LS+dlUOxV5BZ8KH9RKnL7wVN7cwRIQsuWXwgLPMNGvDnTf3hOZiKnv+5Qse1VSG5YHJ36 kXiNRpPHmVOMLhicpFXMvswL1DiF2qAWQ7xd2yaIVCqpT65JQHoNNytPt0sNOGAW1sPnZm0IxLI B5lZ+JFcwNt4h+j/nbo6ouFleLOKQybw+g6qmnx8Nc710qvbIHErZvR6DSvNnXoSWfDfIlQIFQh tqy1wkP4RgFRdtgHjRIzQ+5mk0m0aOBlT4UGqD3xUUJ79zu6jZZzl6XCgRWV8LvjvC5yhcRhpaJ UYxwja0Isg4yhyPhIoR/qnQzg8NAe/a8fuXQVrbKoVvxges6iLWtRok4iw1x5nxmXf/RjacyQP1 4j2y69KcxVdoY6/Yr0uKZQUPstwd/dZOiQTifYRIn+JhWPJHikg9jkuJFA6Es/CARE6ZqeG1KD+ vK0sZ14C/k85kFgTu/VSGWiypUnf2KGemIkckt3SUuDZTQ== X-Received: by 2002:a05:6000:46c7:b0:49c:192f:cf1e with SMTP id ffacd0b85a97d-4a63b13855amr3164674f8f.32.1794995865525; Wed, 18 Nov 2026 01:57:45 -0800 (PST) Received: from inspiron14p-linux ([109.76.187.112]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4a655f9f4a7sm3940871f8f.34.2026.11.18.01.57.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Nov 2026 01:57:45 -0800 (PST) From: Bryan O'Donoghue <bryan.odonoghue@linaro.org> To: libcamera-devel@lists.libcamera.org Cc: rstrode@redhat.com, robert.mader@collabora.com, barnabas.pocze@ideasonboard.com, Bryan O'Donoghue <bryan.odonoghue@linaro.org> Subject: [RFC PATCH v2 00/26] gpuisp multipass + fixes Date: Wed, 18 Nov 2026 10:57:07 +0100 Message-ID: <20261118095735.170587-1-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.55.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: <libcamera-devel.lists.libcamera.org> List-Unsubscribe: <https://lists.libcamera.org/options/libcamera-devel>, <mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe> List-Archive: <https://lists.libcamera.org/pipermail/libcamera-devel/> List-Post: <mailto:libcamera-devel@lists.libcamera.org> List-Help: <mailto:libcamera-devel-request@lists.libcamera.org?subject=help> List-Subscribe: <https://lists.libcamera.org/listinfo/libcamera-devel>, <mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe> Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" <libcamera-devel-bounces@lists.libcamera.org> |
| Series |
|
| Related |
show
|
This series is a bit more advanced now just rebased on 0.8 it tackles a number of open issues. - First we have an ongoing problem of having individual shaders for the packed and unpacked debayer path. There is no real reason for this except "that is how it was done" and "changing the shaders is difficult" Or at least that's been my excuse. Unification of the two shader paths is a no-brainer. You don't maintain parallel almost identical functions in a single file, so you shouldn't maintain parallel virtually identical shaders either. - The second problem tackled is "we don't understand the logic of the shaders well". Fixes end up being posted for the demosiac failures which the authors can't fully explain ostensibly because of the LLM input but really IMO because what is going on with the shaders was already complex and not well understood. So we have a cognition bug. - And we have a general desire to extend out the gpuisp to make it more composable - possible to have real hardware ISP pipelines invoke the GPU to do additional work. This last problem isn't really addressed in this series but the series takes a step in that direction. A series of classes get renamed to reflect the fact we are moving, have moved beyond simple debayering - thus the namespace of the software should reflect the logic and intent of what is going on. We aren't debayering only so lets stop calling it "debayer_egl.cpp" etc. A number of demosiac bugs are fixed in this series in a deliberate order. A) The fact that we have two parallel shaders doing almost exactly the same job. Since we are doing two passes this means our first pass vertex/fragment shader pair outputs a standard frame buffer for the second pass to consume. B) The second pass is a unified demosiac pass for the packed and unpacked path. C) The cognition bug - the readability of the code is addressed. D) Selecting the correct center byte is fixed. It is not apparent on real hardware that we readily select the wrong center byte but, testing on Google Chrome's swift shader which is a pure software implementation - we can quickly see that the sampled center-byte is incorrect. This is because of conversion from pixel co-ordinates and glsl texture2D() co-ordinates. texture2D() wants co-ordinates in the range 0.0f..1.0f. The details are in patch # 23. E) Confusion between two different ways of specifying the texel to sample using both the line+texture and a passed xCoord/YCoord The simple solution when you strip all of the technical analysis away is to not do that. Have a single source of truth. So D and E answer the question "why do multi-pass gpuisp just fix bugs first". Replicating the fixes to two different shaders is just proliferation of our technical debt. Once this series lands we are in pretty good shape to tackle GPU stats but, before adding new functionality I really wanted to drill down on bugs. I'm doing my part: https://www.youtube.com/watch?v=81M62lI4mhc Bryan O'Donoghue (25): libcamera: software_isp: Rename Bayer classes to SoftwareIspInstance libcamera: software_isp: gpu: rename debayerGPU to processGPU libcamera: software_isp: gpu_pipeline_shader_pass: Add base class GpuPipelineShaderPass libcamera: software_isp: gpu_pipeline_shader_pass: Add GpuPipelineShaderPassDemosiac libcamera: software_isp: gpu: Switch to using GpuIspShaderPassDemosiac libcamera: software_isp: gpu: Make Rectangle window_ a local variable in configure() libcamera: software_isp: gpu_pipeline_shader_pass: Move common attribute and uniform variables to base shader class libcamera: software_isp: gpu_pipeline_shader_pass: Move common shader selection logic into base class in new method initShaders() libcamera: shaders: Split packed and unpacked demosiac up libcamera: shaders: bayer_glr16_to_rgba.frag: Use bilinear filtering libcamera: shaders: bayer_{1x_packed|unpacked}_to_glr16f.frag: Add blacklevel correction to bayer domain shaders libcamera: software_isp: gpu: Add GpuIspShaderPassBlcNormalise libcamera: software_isp: egl: Extend eGL::createTexture2D to understand floats libcamera: software_isp: egl: Use Texture Unit 3 for final output texture libcamera: software_isp: egl: Add Ping/Pong buffers with start/stop bindings only libcamera: software_isp: gpu: Include GpuIspShaderPassBlcNormalise in init sequence libcamera: software_isp: gpu: Swtich to two pass logic libcamera: shaders: bayer_glr16_to_rgba.frag: Make fetch() into a function libcamera: software_isp: gpu_pipeline_shader_pass_demosiac: Switch to unified fragment shader libcamera: shaders: bayer_glr16_to_rgba.frag: Sample pixels into a structure libcamera: shaders: bayer_glr16_to_rgba.frag: Move debayer logic to a routine libcamera: software_isp: egl: Move scaling to the second stage libcamera: shaders: bayer_1x_packed_to_blc_glr16f.frag: Sample inputs at the texel centers libcamera: shaders: bayer_glr16_to_rgba.frag: Use center.zw to derive 3x3 texel center libcamera: software_isp: gpu_pipeline_shader_pass_demosiac: Remove unnecessary tex_step input Ray Strode (1): libcamera: software_isp: gpu_pipeline_shader_pass_blc_normalise: Fix texture sampling steps .../internal/software_isp/software_isp.h | 4 +- .../internal/software_isp/swstats_gpu.h | 30 + include/linux/v4l2-isp.h | 102 +++ src/libcamera/egl.cpp | 21 +- .../shaders/bayer_1x_packed_to_glr16f.frag | 97 +++ .../shaders/bayer_glr16f_to_rgba.frag | 221 +++++ src/libcamera/shaders/bayer_unpacked.vert | 14 - .../shaders/bayer_unpacked_to_glr16f.frag | 50 ++ src/libcamera/shaders/meson.build | 3 + src/libcamera/software_isp/debayer_egl.cpp | 752 ------------------ .../software_isp/gpu_pipeline_shader_pass.cpp | 179 +++++ .../software_isp/gpu_pipeline_shader_pass.h | 105 +++ ...gpu_pipeline_shader_pass_blc_normalise.cpp | 209 +++++ .../gpu_pipeline_shader_pass_blc_normalise.h | 52 ++ .../gpu_pipeline_shader_pass_demosiac.cpp | 253 ++++++ .../gpu_pipeline_shader_pass_demosiac.h | 68 ++ src/libcamera/software_isp/meson.build | 9 +- src/libcamera/software_isp/software_isp.cpp | 58 +- ...{debayer.cpp => software_isp_instance.cpp} | 88 +- .../{debayer.h => software_isp_instance.h} | 8 +- ..._cpu.cpp => software_isp_instance_cpu.cpp} | 200 ++--- ...ayer_cpu.h => software_isp_instance_cpu.h} | 16 +- .../software_isp_instance_gpu.cpp | 465 +++++++++++ ...ayer_egl.h => software_isp_instance_gpu.h} | 63 +- src/libcamera/software_isp/swstats_gpu.cpp | 69 ++ 25 files changed, 2128 insertions(+), 1008 deletions(-) create mode 100644 include/libcamera/internal/software_isp/swstats_gpu.h create mode 100644 include/linux/v4l2-isp.h create mode 100644 src/libcamera/shaders/bayer_1x_packed_to_glr16f.frag create mode 100644 src/libcamera/shaders/bayer_glr16f_to_rgba.frag create mode 100644 src/libcamera/shaders/bayer_unpacked_to_glr16f.frag delete mode 100644 src/libcamera/software_isp/debayer_egl.cpp create mode 100644 src/libcamera/software_isp/gpu_pipeline_shader_pass.cpp create mode 100644 src/libcamera/software_isp/gpu_pipeline_shader_pass.h create mode 100644 src/libcamera/software_isp/gpu_pipeline_shader_pass_blc_normalise.cpp create mode 100644 src/libcamera/software_isp/gpu_pipeline_shader_pass_blc_normalise.h create mode 100644 src/libcamera/software_isp/gpu_pipeline_shader_pass_demosiac.cpp create mode 100644 src/libcamera/software_isp/gpu_pipeline_shader_pass_demosiac.h rename src/libcamera/software_isp/{debayer.cpp => software_isp_instance.cpp} (72%) rename src/libcamera/software_isp/{debayer.h => software_isp_instance.h} (93%) rename src/libcamera/software_isp/{debayer_cpu.cpp => software_isp_instance_cpu.cpp} (81%) rename src/libcamera/software_isp/{debayer_cpu.h => software_isp_instance_cpu.h} (93%) create mode 100644 src/libcamera/software_isp/software_isp_instance_gpu.cpp rename src/libcamera/software_isp/{debayer_egl.h => software_isp_instance_gpu.h} (66%) create mode 100644 src/libcamera/software_isp/swstats_gpu.cpp