[libcamera-devel,v3,00/14] IPA: IPU3: Introduce per-frame controls

14555 mbox series

Hello,

This is v3 of the patch series, but only half of v1 is here, as the
second part, which adds the std::queue, is subject to discussion.

This patch series is not only introducing per-frame controls, using the
sensor controls directly in the EventStatReady, it also improves the AGC
in some difficult cases (saturated scenes, backlight, etc.) thanks to
the possibility to use the effective exposure and gains values for a
given frame.

The first two patches 1/14 and 2/14 are Han-Lin Chan series, splitted to
keep only the sensor controls part. I have modified the first patch to
remove the lens controls references, and another series containing only
the lens controls should be sent separately as it requires a bit more
review.

In v2, I also modified the patch 2/14 to call the reset on
DelayedControl at start and not at configure.

For the next patches, v3 introduces a sensor structure in the
frameContext, to distinguish input and output for the algorithms.

From patches 3/14 to 9/14 we use those sensor controls as input
when a EventStatReady event is received, by updating the frameContext
values. With those values, we can be better at estimating the exposure
value, and we introduce a brightness measure to have a better behaviour
in saturated cases.

Patches 10/14 to 14/14 are here to improve the per-frame control updates
by the IPA, introducing the colour temperature control, and updating the
ExposureTime and AnalogueGain controls.

Han-Lin Chen (2):
  ipa: ipu3: Extend ipu3 ipa interface for sensor controls
  ipu3: ipa: Report effective sensor controls with statistics to IPA

Jean-Michel Hautbois (12):
  ipa: ipu3: Use sensor controls to update frameContext
  ipa: ipu3: agc: Limit the number of saturated cells
  ipa: ipu3: agc: Compute the gain for each frame
  ipa: ipu3: agc: Refactor ev gain calculation and testing
  ipa: ipu3: agc: Improve gain calculation
  ipa: ipu3: agc: Update previous exposure value
  ipa: ipu3: agc: Use exposure in time for storage
  ipa: ipu3: awb: Add support for color temperature
  ipa: ipu3: Send color temperature in the metadata
  ipa: ipu3: Cache line duration at configure call
  ipa: ipu3: Move ExposureTime to IPA
  ipa: ipu3: Pass the AnalogueGain control

 include/libcamera/ipa/ipu3.mojom     |   2 +
 src/ipa/ipu3/algorithms/agc.cpp      | 200 ++++++++++++++++++++-------
 src/ipa/ipu3/algorithms/agc.h        |  12 +-
 src/ipa/ipu3/algorithms/awb.cpp      |   1 +
 src/ipa/ipu3/ipa_context.cpp         |  14 ++
 src/ipa/ipu3/ipa_context.h           |   7 +
 src/ipa/ipu3/ipu3.cpp                |  49 ++++---
 src/libcamera/pipeline/ipu3/frames.h |   4 +
 src/libcamera/pipeline/ipu3/ipu3.cpp |  27 ++--
 9 files changed, 227 insertions(+), 89 deletions(-)