From patchwork Tue Apr 22 21:58:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bryan O'Donoghue X-Patchwork-Id: 23213 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AAB63C327D for ; Tue, 22 Apr 2025 21:59:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9814668ACE; Tue, 22 Apr 2025 23:59:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="gxCmqW2p"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A3309617E3 for ; Tue, 22 Apr 2025 23:59:23 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-43cf034d4abso52043885e9.3 for ; Tue, 22 Apr 2025 14:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745359163; x=1745963963; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=J1eu+VFkRk+vqNPfv+gMqeAgR9bS2BFJuAcxD0sE2oI=; b=gxCmqW2pEEsFIf0+jCSWhlb2RA3ZNDXT9dRmZeKR4d+PmHjKastbiT2w7VlPTWQec4 PDjN0w4dRlFM0zD5Fqtcw70JTfKG6aLwOl9JO2eWi4kPGWAvnxZyCrASyoYjB5+xOyZa GzhyTfl0rH2HsGLqFIp4kb/hjXyOGZuri4Hbh/ZY2N5eo2fcRf52SfOY4gCUB4VDzEcg J2oDpaHKoE9HR0xrXyljdEf3Qk7vGek3YXVufCJLg5AAraid+XYji8kgUSV/zddAjdHU UhUWfmpcrKZUgmhd0bHpCksP8+0bKavz1APFpVSeWPftEFPpZi0MejkXKM+fdsWNl9Mo Giwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745359163; x=1745963963; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=J1eu+VFkRk+vqNPfv+gMqeAgR9bS2BFJuAcxD0sE2oI=; b=bHHaaP42LwRzJurY2Q0vX7LUo2beA3c7s5tLDXGcCtnEOC8mKEqfW4BllKAqva754l hQQZ03mAy/lXJ7yNOEXKmsPwl3Q0cKDy4BwW2TmpsRcBckf9HgOXSyOJh1M7zZV2re4w DC4EYktvkBT0bf+oAe/P/hDQXwEJyJ7cCquUJDEvrVbiMv1jcrf1ZRtTKqAb17r4iIug T2nIDW1RLpRVDUX6rOSpGwzMRs8OYKBJY5wz8XVKeiPPDHC4RSe9a6cerjbED+HnNNxM Ivk64nho0yd5XVq7Gzny0COmnsDFZMv/1dfTjFBXou9LgyCM86SOlsVV/pzCSKIItYVn o6qQ== X-Gm-Message-State: AOJu0YxQxUjuZex59z3LEOtyq9XC21p1mvPv7ywugkxUPmDazEw2iBVc nOzah2SeniZ1Tp4PuoJ0/0So36kDKUwAGstPr6L4RWyZ6Dg8C5D5xkNuo46D1XjKlxBMfgKzfUm 3eHI= X-Gm-Gg: ASbGncvgBlrAnsnguLDkIMUOQSn5YMo2dobwthrrotxWOpsl8K8awOZLYEcruIYbMNz SDT7NRU9VxRaYP1liH/kDX8nYNOMCbO8aNcq/jWT+lbpwstce5TXCfLMZY98jjgWmkUbjsU2EPJ rLMX4lNifIEq8hdg0wEPE6aHFNX7FAmEZejoaO6KmO2DM/b4umelicDxvkKEhrVlP0uEPw3/eOX 0VhQbneqYV+H7s/nzWuZUiIPAaNnY/Bh+32opWXO9+BuwUiFFgB+sOjXkRrla71Qdofh13ODJ4z R441wt3a9aX7zSP+Aq6bbvURMMBjPgcEL3ER1VC85+c/rWVCPTimS4959EvCi8/+4/QXZeE4Aba tWEO3EjIQH0m09EIhWREahq6fNG10Gs4= X-Google-Smtp-Source: AGHT+IHB1kycjXHKEYDn8wll0gaIm9cn1WVXxipCuzeokx2bIqALYNO9VVHMFDR3OZ5SsFJ1aa87Qg== X-Received: by 2002:a05:600c:3482:b0:43c:ec0a:ddfd with SMTP id 5b1f17b1804b1-4406ab7fe91mr143086035e9.6.1745359162716; Tue, 22 Apr 2025 14:59:22 -0700 (PDT) Received: from inspiron14p-linux.ht.home (188-141-3-146.dynamic.upc.ie. [188.141.3.146]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44092d2eccesm2726615e9.20.2025.04.22.14.59.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Apr 2025 14:59:22 -0700 (PDT) From: Bryan O'Donoghue To: libcamera-devel@lists.libcamera.org Cc: hdegoede@redhat.com, mzamazal@redhat.com, bryan.odonoghue@linaro.org, bod.linux@nxsw.ie Subject: [PATCH 00/27] RFC: Add in a eGL based GPUISP in libcamera Date: Tue, 22 Apr 2025 22:58:53 +0100 Message-ID: <20250422215920.4297-1-bryan.odonoghue@linaro.org> X-Mailer: git-send-email 2.49.0 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This series takes the existing debayer shaders we have for qcam and repurposes those to operate inside of libcamera/softisp. EGL based to support the widest range of hardware. This code integrates with the stastics and parameters code in softisp with CCM an additional WIP on top. We've been doing this project on the back burner for ~ 12 months with alot of fun and learning on my end in the process. I've tidied up the code somewhat and gotten it to the stage of producing a pretty OK picture. Still todo here: - CCM - DMA for input and output - Adding something on the algorithms side that cannot be done in CPU. This code isn't ready for merge but, its certainly ready for comment and test. Bryan O'Donoghue (21): libcamera: MappedFrameBuffer: Add tracking of mmap file descriptors to MappedFrameBuffer libcamera: MappedFrameBuffer: Add accessor for MappedFrameBuffer plane maps libcamera: dma_buf_allocator: Favour udmabuf over cma heap allocations libcamera: software_isp: Start the ISP thread in configure libcamera: software_isp: Move configure to worker thread libcamera: software_isp: Virtual base class reference in invokeMethod libcamera: software_isp: gbm: Add in a GBM helper class for GPU surface access libcamera: software_isp: egl: Introduce an eGL base helper class libcamera: software_isp: Move useful items from DebayerCpu to Debayer base class libcamera: software_isp: Move param select code to Debayer base class libcamera: software_isp: Move DMA sync code to common place in Debayer base libcamera: software_isp: Make the debayer_ object of type class Debayer not DebayerCpu libcamera: software_isp: debayer_egl: Add an eGL debayer class libcamera: software_isp: debayer_egl: Extend logic to enable application of softISP RGB debayer params libcamera: software_isp: debayer_egl: Make DebayerEGL the default debayer method libcamera: software_isp: egl: Bind each eglImage to its own texture unit libcamera: shaders: Use highp not mediump for float precision libcamera: shaders: Move GL shader programs to src/libcamera/assets/shader utils: gen-shader-headers: Add a utility to generate headers from shaders meson: Automatically generate glsl_shaders.h from specified shader programs shaders: Extend debayer shaders to apply RGB gain values on output Hans de Goede (6): libcamera: swstats_cpu: Update statsProcessFn() / processLine0() documentation libcamera: swstats_cpu: Drop patternSize_ documentation libcamera: software_isp: Move benchmark code to its own class libcamera: swstats_cpu: Add processFrame() method libcamera: swstats_cpu: Add support for YUV420 libcamera: swstats_cpu: Move header to libcamera/internal/software_isp .../libcamera/internal/mapped_framebuffer.h | 3 + include/libcamera/internal/meson.build | 10 + .../libcamera/internal/shaders}/RGB.frag | 2 +- .../internal/shaders}/YUV_2_planes.frag | 2 +- .../internal/shaders}/YUV_3_planes.frag | 2 +- .../internal/shaders}/YUV_packed.frag | 2 +- .../internal/shaders}/bayer_1x_packed.frag | 12 +- .../libcamera/internal/shaders}/bayer_8.frag | 18 +- .../libcamera/internal/shaders}/bayer_8.vert | 0 .../libcamera/internal/shaders}/identity.vert | 0 .../libcamera/internal/shaders/meson.build | 10 + .../internal/software_isp/benchmark.h | 36 ++ .../internal/software_isp/meson.build | 2 + .../internal/software_isp/software_isp.h | 5 +- .../internal}/software_isp/swstats_cpu.h | 18 + src/apps/qcam/assets/shader/shaders.qrc | 16 +- src/apps/qcam/viewfinder_gl.cpp | 70 +-- src/libcamera/dma_buf_allocator.cpp | 2 +- src/libcamera/mapped_framebuffer.cpp | 1 + src/libcamera/software_isp/benchmark.cpp | 93 +++ src/libcamera/software_isp/debayer.cpp | 40 ++ src/libcamera/software_isp/debayer.h | 41 +- src/libcamera/software_isp/debayer_cpu.cpp | 65 +- src/libcamera/software_isp/debayer_cpu.h | 39 +- src/libcamera/software_isp/debayer_egl.cpp | 571 ++++++++++++++++++ src/libcamera/software_isp/debayer_egl.h | 175 ++++++ src/libcamera/software_isp/egl.cpp | 347 +++++++++++ src/libcamera/software_isp/egl.h | 110 ++++ src/libcamera/software_isp/gbm.cpp | 136 +++++ src/libcamera/software_isp/gbm.h | 52 ++ src/libcamera/software_isp/meson.build | 31 + src/libcamera/software_isp/software_isp.cpp | 25 +- src/libcamera/software_isp/swstats_cpu.cpp | 178 +++++- utils/gen-shader-header.py | 38 ++ utils/gen-shader-headers.sh | 41 ++ utils/meson.build | 2 + 36 files changed, 2016 insertions(+), 179 deletions(-) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/RGB.frag (93%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/YUV_2_planes.frag (97%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/YUV_3_planes.frag (96%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/YUV_packed.frag (99%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/bayer_1x_packed.frag (95%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/bayer_8.frag (89%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/bayer_8.vert (100%) rename {src/apps/qcam/assets/shader => include/libcamera/internal/shaders}/identity.vert (100%) create mode 100644 include/libcamera/internal/shaders/meson.build create mode 100644 include/libcamera/internal/software_isp/benchmark.h rename {src/libcamera => include/libcamera/internal}/software_isp/swstats_cpu.h (79%) create mode 100644 src/libcamera/software_isp/benchmark.cpp create mode 100644 src/libcamera/software_isp/debayer_egl.cpp create mode 100644 src/libcamera/software_isp/debayer_egl.h create mode 100644 src/libcamera/software_isp/egl.cpp create mode 100644 src/libcamera/software_isp/egl.h create mode 100644 src/libcamera/software_isp/gbm.cpp create mode 100644 src/libcamera/software_isp/gbm.h create mode 100755 utils/gen-shader-header.py create mode 100755 utils/gen-shader-headers.sh