[00/13] ipa: simple: Convert to libipa AWB implementation

26483 mbox series

This series starts by clearing the unpacked fragment shader to keep the
same indentation in both shader types to ease future patches.

Then there is a quite substantial patch which splits the AWB handling
for the SoftISP out of the combined matrix.

This is important to be able to prevent saturation issues in bright
regions showing up as neon colours. The AWB is applied and then the
result is clamped to the sensor range (at that point with black level
subtracted).

The gains are then fixed in the CPU ISP, with contributions from Milan.

The GPU CCM matrix multiplication is simplified to use the native GPU
multiplication types, and adapts the ordering when the uniforms are
created.

We then refactor the libipa lux algorithm to provide a common data
structure and move the RKISP1 implementation to use this new type.

That allows the Simple IPA to re-use the lux algorithm without
duplicating the data structures.

The same approach is then taken to convert the Awb structures to common
definitions and the simple IPA is refactored to be able to utilise the
libipa AWB implementation.

The AWB implementations for configure, queueReqeust, and prepare are
then refactored into functions that can be directly called in the libipa
implementation to reduce duplicating identical code when adding a new
platform to use AWB.

The last conversion patches might also further refactor common details
from the initialisation functions and process functions, and also the
stats helpers are completely duplicated, so they'll be in the next
version too.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---
Kieran Bingham (11):
      libcamera: shaders: unpacked: Fix indentations
      softisp: Split AWB from Combined Matrix
      shaders: bayer: Use native matrix multiplication
      ipa: libipa: provide lux context structures
      ipa: simple: Add lux algorithm to the SoftIPA
      ipa: libipa: Awb: Provide common context storage
      libcamera: vector: Convertor Constructor
      ipa: simple: Convert awb to libipa implementation
      ipa: libipa: awb: Move configure to common
      ipa: libipa: awb: convert to common queueRequest
      ipa: libipa: awb: factor out common prepare

Milan Zamazal (2):
      libcamera: software_isp: Fix black level handling in CPU ISP
      libcamera: software_isp: Apply gains in CPU ISP

 include/libcamera/internal/vector.h        |   8 +
 src/ipa/libipa/awb.cpp                     | 155 ++++++++++++++++++++
 src/ipa/libipa/awb.h                       |  38 +++++
 src/ipa/libipa/lux.cpp                     |  16 ++
 src/ipa/libipa/lux.h                       |  12 ++
 src/ipa/rkisp1/algorithms/awb.cpp          |  73 +--------
 src/ipa/rkisp1/ipa_context.h               |  41 ++----
 src/ipa/simple/algorithms/awb.cpp          | 171 +++++++++++++++++-----
 src/ipa/simple/algorithms/awb.h            |  12 ++
 src/ipa/simple/algorithms/ccm.cpp          |   2 +-
 src/ipa/simple/algorithms/lux.cpp          |  93 ++++++++++++
 src/ipa/simple/algorithms/lux.h            |  39 +++++
 src/ipa/simple/algorithms/meson.build      |   1 +
 src/ipa/simple/ipa_context.h               |  20 +--
 src/libcamera/shaders/bayer_1x_packed.frag |  47 +-----
 src/libcamera/shaders/bayer_unpacked.frag  | 228 ++++++++++++-----------------
 src/libcamera/software_isp/debayer_cpu.cpp |  71 ++++-----
 src/libcamera/software_isp/debayer_egl.cpp |  17 +--
 src/libcamera/software_isp/debayer_egl.h   |   3 +
 src/libcamera/vector.cpp                   |   7 +
 20 files changed, 693 insertions(+), 361 deletions(-)
---
base-commit: 7528d65aeee57c02ea5b085a3e3d364f23ed4ade
change-id: 20260321-kbingham-awb-split-d130eee76e36

Best regards,

On Tue, Apr 07, 2026 at 11:01:03PM +0100, Kieran Bingham wrote:
> This series starts by clearing the unpacked fragment shader to keep the
> same indentation in both shader types to ease future patches.
> 
> Then there is a quite substantial patch which splits the AWB handling
> for the SoftISP out of the combined matrix.
> 
> This is important to be able to prevent saturation issues in bright
> regions showing up as neon colours. The AWB is applied and then the
> result is clamped to the sensor range (at that point with black level
> subtracted).

But you don't fix the issue in this patch series, right ?

> The gains are then fixed in the CPU ISP, with contributions from Milan.
> 
> The GPU CCM matrix multiplication is simplified to use the native GPU
> multiplication types, and adapts the ordering when the uniforms are
> created.
> 
> We then refactor the libipa lux algorithm to provide a common data
> structure and move the RKISP1 implementation to use this new type.
> 
> That allows the Simple IPA to re-use the lux algorithm without
> duplicating the data structures.
> 
> The same approach is then taken to convert the Awb structures to common
> definitions and the simple IPA is refactored to be able to utilise the
> libipa AWB implementation.
> 
> The AWB implementations for configure, queueReqeust, and prepare are
> then refactored into functions that can be directly called in the libipa
> implementation to reduce duplicating identical code when adding a new
> platform to use AWB.
> 
> The last conversion patches might also further refactor common details
> from the initialisation functions and process functions, and also the
> stats helpers are completely duplicated, so they'll be in the next
> version too.

I was going to ask if this was planned :-)

One point that needs careful attention is if the common implementation
can support inline ISPs, offline ISPs, and offline ISP backends with
statistics computed by the frontend on the same frame.

> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> ---
> Kieran Bingham (11):
>       libcamera: shaders: unpacked: Fix indentations
>       softisp: Split AWB from Combined Matrix
>       shaders: bayer: Use native matrix multiplication
>       ipa: libipa: provide lux context structures
>       ipa: simple: Add lux algorithm to the SoftIPA
>       ipa: libipa: Awb: Provide common context storage
>       libcamera: vector: Convertor Constructor
>       ipa: simple: Convert awb to libipa implementation
>       ipa: libipa: awb: Move configure to common
>       ipa: libipa: awb: convert to common queueRequest
>       ipa: libipa: awb: factor out common prepare
> 
> Milan Zamazal (2):
>       libcamera: software_isp: Fix black level handling in CPU ISP
>       libcamera: software_isp: Apply gains in CPU ISP
> 
>  include/libcamera/internal/vector.h        |   8 +
>  src/ipa/libipa/awb.cpp                     | 155 ++++++++++++++++++++
>  src/ipa/libipa/awb.h                       |  38 +++++
>  src/ipa/libipa/lux.cpp                     |  16 ++
>  src/ipa/libipa/lux.h                       |  12 ++
>  src/ipa/rkisp1/algorithms/awb.cpp          |  73 +--------
>  src/ipa/rkisp1/ipa_context.h               |  41 ++----
>  src/ipa/simple/algorithms/awb.cpp          | 171 +++++++++++++++++-----
>  src/ipa/simple/algorithms/awb.h            |  12 ++
>  src/ipa/simple/algorithms/ccm.cpp          |   2 +-
>  src/ipa/simple/algorithms/lux.cpp          |  93 ++++++++++++
>  src/ipa/simple/algorithms/lux.h            |  39 +++++
>  src/ipa/simple/algorithms/meson.build      |   1 +
>  src/ipa/simple/ipa_context.h               |  20 +--
>  src/libcamera/shaders/bayer_1x_packed.frag |  47 +-----
>  src/libcamera/shaders/bayer_unpacked.frag  | 228 ++++++++++++-----------------
>  src/libcamera/software_isp/debayer_cpu.cpp |  71 ++++-----
>  src/libcamera/software_isp/debayer_egl.cpp |  17 +--
>  src/libcamera/software_isp/debayer_egl.h   |   3 +
>  src/libcamera/vector.cpp                   |   7 +
>  20 files changed, 693 insertions(+), 361 deletions(-)
> ---
> base-commit: 7528d65aeee57c02ea5b085a3e3d364f23ed4ade
> change-id: 20260321-kbingham-awb-split-d130eee76e36