[v5,00/19] Implement WDR algorithm

24409 mbox series

Hi all,

This series implements a global tone mapping algorithm for rkisp1. The
details of the algorithms are explained in patch 15/16, so I won't
repeat them here.

v1 of the series was here:
https://patchwork.libcamera.org/project/libcamera/list/?series=5122

v2 was here:
https://patchwork.libcamera.org/project/libcamera/list/?series=5364

v3 was here:
https://patchwork.libcamera.org/project/libcamera/list/?series=5382

v4 was here:
https://patchwork.libcamera.org/project/libcamera/list/?series=5449

Changes in v5:

The Wdr controls were moved from draft to core namespace and a few small
fixes in patch 1 and 18.  CI is also happy:
https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1513542

So I guess I'll merge that :-)

Changes in v4:

Some smaller fixes from the review. Please see individual patches for
the local changelog.

Changes in v3:

I hope I addressed all points from the last review. Some feedback got me
thinking and that lead to the WDR algorithm being reworked a bit.
Basically I replaced the former regulation loop + WDR-exposure-value
with a additional constraint in the AGC regulation.  That allowed me to
ditch the PID controller (for now). I believe the structure is easier
now. For that to work I needed a way to add constraints to the AGC at
runtime. The current implementation is in patch 16. I'm not entirely
happy with that due to dynamic allocations in the scope of the frame
context. So maybe we have to come up with a neater way there.

Changes in v2:

Compared to v1 of the series there were quite some changes. The old
series contained mitigation for exposure quantization using the AWB
gains. During my work on HDR stitching support it got clear that we need
to use the compression curve from the companding block of the imx8mp.
That curve can serve as a way better gain than the AWB gains, so I added
a compression algorithm (patches 1-2) which then is used for the
quantization correction.  The quantization correction was generalized to
support exposure and gain quantization effects (patches 3-9). Patch 10
spilled over from the WDR preparation series and now has a (possibly)
better commit message.  The rest is the actual WDR implementation that
was mostly there in v1. I fixed a few bugs and included changes from
review (thanks Paul). The details are in the individual patches.

Best regards,
Stefan

Stefan Klug (19):
  ipa: rkisp1: Add basic compression algorithm
  tuning: rksip1: Add a static Compress entry
  libipa: camera_sensor_helper: Add quantizeGain() function
  libipa: exposure_mode_helper: Take exposure/gain quantization into
    account
  libipa: exposure_mode_helper: Remove double calculation of
    lastStageGain
  libipa: exposure_mode_helper: Remove unnecessary clamp calls
  libipa: agc_mean_luminance: Fix constraint logging
  libipa: agc_mean_luminance: Configure the exposure mode helpers
  libipa: exposure_mode_helper: Calculate quantization gain in
    splitExposure()
  ipa: rkisp1: agc: Add correction for exposure quantization
  pipeline: rkisp1: Add error log when parameter queuing fails
  include: linux: Partially update linux headers from
    v6.16-rc1-310-gd968e50b5c26
  ipa: rkisp1: Switch histogram to RGB combined mode
  pipeline: rkisp1: Query kernel for available params blocks
  libipa: agc_mean_luminance: Introduce effectiveYTarget() accessor
  libipa: agc_mean_luminance: Add support for additional constraints
  rkisp1: agc: Agc add yTarget to frame context
  ipa: rkisp1: Add WDR algorithm
  tuning: rksip1: Add a static WideDynamicRange entry

 include/libcamera/ipa/rkisp1.mojom       |   2 +-
 include/linux/rkisp1-config.h            | 108 ++++-
 include/linux/v4l2-controls.h            |   6 +
 src/ipa/ipu3/algorithms/agc.cpp          |   6 +-
 src/ipa/libipa/agc_mean_luminance.cpp    |  59 ++-
 src/ipa/libipa/agc_mean_luminance.h      |   8 +-
 src/ipa/libipa/camera_sensor_helper.cpp  |  23 ++
 src/ipa/libipa/camera_sensor_helper.h    |   1 +
 src/ipa/libipa/exposure_mode_helper.cpp  | 108 +++--
 src/ipa/libipa/exposure_mode_helper.h    |  12 +-
 src/ipa/mali-c55/algorithms/agc.cpp      |   7 +-
 src/ipa/rkisp1/algorithms/agc.cpp        |  85 +++-
 src/ipa/rkisp1/algorithms/blc.cpp        |   4 +-
 src/ipa/rkisp1/algorithms/compress.cpp   | 103 +++++
 src/ipa/rkisp1/algorithms/compress.h     |  30 ++
 src/ipa/rkisp1/algorithms/goc.cpp        |   4 +-
 src/ipa/rkisp1/algorithms/lux.cpp        |   2 +-
 src/ipa/rkisp1/algorithms/meson.build    |   2 +
 src/ipa/rkisp1/algorithms/wdr.cpp        | 493 +++++++++++++++++++++++
 src/ipa/rkisp1/algorithms/wdr.h          |  58 +++
 src/ipa/rkisp1/ipa_context.cpp           |  19 +
 src/ipa/rkisp1/ipa_context.h             |  32 +-
 src/ipa/rkisp1/params.cpp                |   1 +
 src/ipa/rkisp1/params.h                  |   2 +
 src/ipa/rkisp1/rkisp1.cpp                |  12 +-
 src/libcamera/control_ids_core.yaml      |  62 +++
 src/libcamera/pipeline/rkisp1/rkisp1.cpp |  38 +-
 utils/tuning/rkisp1.py                   |   8 +-
 28 files changed, 1211 insertions(+), 84 deletions(-)
 create mode 100644 src/ipa/rkisp1/algorithms/compress.cpp
 create mode 100644 src/ipa/rkisp1/algorithms/compress.h
 create mode 100644 src/ipa/rkisp1/algorithms/wdr.cpp
 create mode 100644 src/ipa/rkisp1/algorithms/wdr.h