[v8,00/26] Add GLES 2.0 GPUISP to libcamera

25515 mbox series

v8:

A small number of updates. I'll copy and paste my change notes directly.

- gbm patch
  - \todo, nullptr comments addressed - Milan
  - Left fail - meant fail
  - Changed cleanup flow
  - // namespace libcamera -> /* namespace libcamera */
  - Dropped HAVE_GBM

- Contrast patch
  Passed contrastExp as argument to apply_contrast() - Milan

- Dropped "invalid." for "invalid"
  Left "it makes sense" not sure of "let's try" was a suggested
  commit log comment but - "it makes sense" to me :) - Milan

- Changes CCM init to Matrix<float, 3, 3> identityMatrix = Matrix<float, 3, 3>::identity();
  Amends commit log  - Milan

- Fixes the identity CCM commit log
  Old commit log not updated on review changes - Milan

- Changed selfEnumerating to use common code
  Milan left a comment on not having copied code so I created
  a helper function and moved a much as I thought I could - Milan

- todo
  - rewriging -> rewiring.
    onomatopoeic error !
  - manging -> managing

v7:

- Takes Laurent's question on doing egl init in start() instead of
  configure() and implements it. Tested with repeated start()/stop() from
  qcam.

- A few supporting patches to get data we require in configure() into
  start() has been added as a result.

- Bayer object now has a start() and a stop() - introduced in the
  precursor series.

- Dropped overloaded algorith init() function as Milan seemed to
  prefer a flag in context_ over an overloaded init().

- Small tidy up on the use of TEXTURE1 instead of TEXTURE4 now
  that the RGB lookup tables have been dropped too.

- Updates the todo list

- The one thing not done is passing the complex contrast
  into the fragment shader per frame, instead of having it
  calculated per-pixel. I'd rather get the current series
  out/merged than add this extra bit right now.

v6:

- Rebases on 0.6.0

- Removes conditional compilation for lookup-table v CCM.
  The conditional compile to switch the ordering of blue is kept. - Laurent

- Removes Bayer lookup table code from shaders and debayer_egl logic
  Disussed with Laurent and agree it can be dropped now

- Adds a patch to pass no-op values to the debayer shaders when
  running those shaders in qcam. Confirmed this works by switching
  the true/false flag in simple.cpp and recompiling. Once the raw-streams
  stuff gets merged changing simple.cpp shouldn't be required.

  This lets us continue to support the original behaviour of the debayer
  shaders in Qcam while also reusing those shaders with our extensions
  in softisp.

- Re-orders the patches a bit to group changes a bit more logically
  especially on more recently added patches. CI is still happy with building
  as progressive changes.

- Drops the patch to make isStandardBayerOrder static as this logic
  has been squashed into the precursor series. Barnabas

v5:

- Takes in a fix from Hans for the unpacked center bytes.
  I opted to have separate stride values passed to the shaders
  to maintain consistency with qcam
- Implements the unique_ptr change Barnabas suggested
- Changes to glFinish instead of eglWaitSyncKHR - Milan/Robert
- Fixes interface to pass gpuIspEnabled as a parameter - Barnabas
- Makes it an error if you request GPUISP but object instantiation fails - Barnabas
- Fixes indentation splat in unpacked shader - I love whitespace :( - Laurent
- Leaves the upload tables in place - I thought we noted to do this in weekly meeting
  I'm open/happy to drop this code though its not really necessary now.
- Leaves conditional compilation in shaders.
  The reason being with conditional compilation we can still run the original
  unmodifed shaders - which gives a nice base to mess with the demosiac phase
  in isolation.

v4:

- Drops AWB since the CCM contains it already
- Includes Gamma
- Includes Contrast - testable via camshark
- Includes Saturation - testable via camshark
- Includes a scaler from Robert
- Includes synch changes from Robert
- Includes all feedback incorporated from Pavel
- Generates a default 65k CCM if none is supplied
- Various Doxygen torments fixed along the way
- And is the "top half" of the precursor series as the GPUISP
  series becomes 44 patches long this is an unreasonable number
  to merge in one go.

- Full testable branch
Link: https://gitlab.freedesktop.org/camera/libcamera-softisp/-/tree/v0.5.2-gpuisp-v4e?ref_type=heads

- The first part of the series is in the precurso here
Link: https://gitlab.freedesktop.org/camera/libcamera-softisp/-/commits/v0.5.2-gpuisp-v4e-split

That precursor is just a tag about half way through the integrated series.

Bryan O'Donoghue (23):
  libcamera: software_isp: gbm: Add a GBM helper class for GPU surface
    access
  libcamera: software_isp: egl: Add a eGL base helper class
  qcam: viewfinder_gl: Set no-op Bayer shader values
  libcamera: shaders: Use highp not mediump for float precision
  libcamera: shaders: Extend debayer shaders to apply CCM gains
  libcamera: shaders: Extend bayer shaders to support swapping R and B
    on output
  libcamera: shaders: Add support for black level compensation
  libcamera: shaders: Add support for Gamma
  libcamera: shaders: Add support for contrast
  libcamera: software_isp: Add member variables to track selected
    input/output pixelFormat
  libcamera: software_isp: Add a Size_ member variable to pass to eGL
    later
  libcamera: software_isp: debayer_egl: Add an eGL Debayer class
  libcamera: software_isp: debayer_cpu: Make getInputConfig and
    getOutputConfig static
  libcamera: software_isp: debayer_egl: Make DebayerEGL an environment
    option
  libcamera: software_isp: debayer_egl: Make gpuisp default softisp mode
  ipa: software_isp: Add a selfInitialising flag to IPAContext
  libcamera: software_isp: ccm: Add self-initialising identity CCM to
    Ccm::init
  ipa: libipa: module: Add createSelfEnumeratingAlgorithm
  ipa: simple: Add a flag to indicate gpuIspEnabled
  ipa: software_isp: Call createSelfEnumeratingAlgorithm() to statically
    instantiate CCM algo
  libcamera: software_isp: lut: Skip calculation lookup tables if
    gpuIspEnabled is true
  libcamera: software_isp: lut: Change default Gamma to 1.0/2.2
  libcamera: software_isp: Add a gpuisp todo list

Hans de Goede (1):
  libcamera: shaders: Fix input sampling when width != stride

Milan Zamazal (2):
  libcamera: shaders: Rename bayer_8 to bayer_unpacked
  libcamera: software_isp: GPU support for unpacked 10/12-bit formats

 include/libcamera/internal/egl.h              | 186 +++++
 include/libcamera/internal/gbm.h              |  39 ++
 include/libcamera/internal/meson.build        |   1 +
 include/libcamera/ipa/soft.mojom              |   3 +-
 src/apps/qcam/assets/shader/shaders.qrc       |   4 +-
 src/apps/qcam/viewfinder_gl.cpp               |  41 +-
 src/apps/qcam/viewfinder_gl.h                 |  12 +
 src/ipa/libipa/module.cpp                     |  34 +
 src/ipa/libipa/module.h                       |  47 +-
 src/ipa/simple/algorithms/ccm.cpp             |  22 +-
 src/ipa/simple/algorithms/lut.cpp             |  72 +-
 src/ipa/simple/ipa_context.h                  |   2 +
 src/ipa/simple/soft_simple.cpp                |   9 +
 src/libcamera/egl.cpp                         | 619 +++++++++++++++++
 src/libcamera/gbm.cpp                         | 130 ++++
 src/libcamera/meson.build                     |  33 +
 src/libcamera/shaders/RGB.frag                |   2 +-
 src/libcamera/shaders/YUV_2_planes.frag       |   2 +-
 src/libcamera/shaders/YUV_3_planes.frag       |   2 +-
 src/libcamera/shaders/YUV_packed.frag         |   2 +-
 src/libcamera/shaders/bayer_1x_packed.frag    |  79 ++-
 .../{bayer_8.frag => bayer_unpacked.frag}     |  95 ++-
 .../{bayer_8.vert => bayer_unpacked.vert}     |   6 +-
 src/libcamera/shaders/meson.build             |   4 +-
 src/libcamera/software_isp/debayer.cpp        |  15 +
 src/libcamera/software_isp/debayer.h          |   3 +
 src/libcamera/software_isp/debayer_cpu.h      |   4 +-
 src/libcamera/software_isp/debayer_egl.cpp    | 641 ++++++++++++++++++
 src/libcamera/software_isp/debayer_egl.h      | 142 ++++
 src/libcamera/software_isp/gpuisp-todo.txt    |  40 ++
 src/libcamera/software_isp/meson.build        |   8 +
 src/libcamera/software_isp/software_isp.cpp   |  37 +-
 32 files changed, 2258 insertions(+), 78 deletions(-)
 create mode 100644 include/libcamera/internal/egl.h
 create mode 100644 include/libcamera/internal/gbm.h
 create mode 100644 src/libcamera/egl.cpp
 create mode 100644 src/libcamera/gbm.cpp
 rename src/libcamera/shaders/{bayer_8.frag => bayer_unpacked.frag} (51%)
 rename src/libcamera/shaders/{bayer_8.vert => bayer_unpacked.vert} (90%)
 create mode 100644 src/libcamera/software_isp/debayer_egl.cpp
 create mode 100644 src/libcamera/software_isp/debayer_egl.h
 create mode 100644 src/libcamera/software_isp/gpuisp-todo.txt