[v3,00/19] libipa: agc: Calculate exposure limits

25052 mbox series

By definition, the exposure time that can be programmed on a sensor is
limited by the frame duration and whenever the frame duration is
updated, the exposure time limits should be updated as well.

This is not the case for IPAs using the AgcMeanLuminance algorithm from
libipa, where the exposure limits are computed at configure() time and
never updated.

This has two implications:
1) The AGC algorithms will always operate with an exposure time bound to
   the frame duration programmed at startup time
2) The Camera::controls() limits are not updated to reflect the new
   constraints

This series addresses issue 1) by changing the AgcMeanLuminance class
and the associated helpers to regulate the exposure time base on the
currently programmed max frame duration using a sensor-specific margin
that is now reported by the CameraHelper class.

In order to get there, a few things worth describing on the design of
the series.

Along the whole series, I tried to separate "sensor" configuration data
from run-time configuration data. The main target here is to facilitate
the work Kieran is doing about centralizing the sensor configuration in
the sensorHelper, so that each IPA doesn't have to populate and handle
the sensor configuration data.

For this reason I tried to:

1) Align all IPAS to use the same IPAContext.configuration.sensor layout
   where they populate the sensor configuration data at configure() time
2) Make the AgcMeanLuminance and ExposureModeHelper class use a similar
   type to:
   - Pass configuration data to AgcMeanLuminance::configure()
   - Store the configuration data separately from the run-time data in
     ExposureModeHelper

   I'm not thrilled by adding more types, one per each class, but
   hopefully this should make it easy to replace all of them by just
   using the camera sensor helper

As the AgcMeanLuminance algorithm tries to push the exposure time up
to the maximum allowed frame duration, initializing the algorithm
with the sensor's maximum capabilities results in a very low frame rate,
unless the user specifies a FrameDurationLimits range in which the
algorithm should operate. To avoid slowing the frame rate down too much
compute a "default" frame rate at algorithm configuration time, were
the canonical 30, 15 and 10 FPS values are tested and selected in order
of preference.

I took the occasion to also rework the AgcMeanLuminance interface which
now looks like:

- AgcMeanLuminance::configure(sensorConfig, sensorHelper)
- AgcMeanLuminance::setExposureLimits(fixedShutter, fixedGain, frameDuration)

Once all IPAs have been moved to compute the exposure time from the
frame duration, the exposure time can be removed from the sensor
configuration data.

Niklas suggested we should remove inspecting V4L2_CID_EXPOSURE
completely. While I don't 100% agree this is any problem, I tried to do
so, but the minExposureTime is effectivelly a sensor parameter and if
the kernel offers an easy way to retrieve it I think we should use it.

The last patch is not for inclusion as Mali support needs to be
finalized upstream first, but shows how is it possible to handle
frame duration limits and exposure with the new model (and I needed that
patch for testing on Mali anyway).

Tested with camshark on Mali-C55 and RkISP1 and with a test scripts that
sets a longer frame duration (66 msecs) and by validating that the
exposure time is increased to match the new limits on RkISP1.

RkISP1Agc agc.cpp:644 Divided up exposure time, analogue gain, quantization gain and digital gain are 33251.27us, 10.6667, 1.00018 and 6.49639
61.545331 (47.58 fps) cam0-stream0 seq: 000050 bytesused: 2073600/1036800
kISP1Agc agc.cpp:644 Divided up exposure time, analogue gain, quantization gain and digital gain are 66578.16us, 10.6667, 1.00018 and 3.24451
61.566241 (47.82 fps) cam0-stream0 seq: 000051 bytesused: 2073600/1036800

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
Changes in v3:
- EDITME: describe what is new in this series revision.
- EDITME: use bulletpoints and terse descriptions.
- Link to v2: https://lore.kernel.org/r/20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com

Changes in v2:
- Rework of v1 that moves the exposure limits computation to the
  AgcMeanLuminance exposure helpers

---
Jacopo Mondi (18):
      ipa: ipu3: Move CameraHelper to context
      ipa: rkisp1: Store FrameDurationLimits as sensor config
      ipa: mali: Move exposure limits to sensor config
      ipa: ipu3: Move exposure limits to sensor config
      ipa: camera_sensor_helper: Introduce maxShutterTime()
      ipa: camera_sensor_helper: Introduce minFrameDuration()
      ipa: libipa: agc: Remove unused accessors
      ipa: libipa: agc: Store sensor configuration parameters
      ipa: libipa: agc: Make Agc::configure() set limits
      ipa: libipa: agc: Pass the frame duration to configure()
      ipa: libipa: agc: Compute max shutter time with frame duration
      ipa: libipa: agc: Calculate exposure limits with frame duration
      ipa: libipa: agc: Rework setLimits() interface
      ipa: libipa: agc: Fold resetFrameCount() in configure()
      ipa: libipa: Remove maxExposureTime from sensor configuration
      ipa: rkisp1: agc: Calculate frame duration using cam helper
      ipa: libipa: agc: Initialize a sensible frame duration
      [DNI] ipa: mali: Handle FrameDurationLimits

Kieran Bingham (1):
      ipa: mali-c55: Move CameraHelper to context

 src/ipa/ipu3/algorithms/agc.cpp         |  29 +++----
 src/ipa/ipu3/algorithms/agc.h           |   4 -
 src/ipa/ipu3/ipa_context.cpp            |  29 +++----
 src/ipa/ipu3/ipa_context.h              |  17 ++--
 src/ipa/ipu3/ipu3.cpp                   |  37 ++++----
 src/ipa/libipa/agc_mean_luminance.cpp   | 127 ++++++++++++++++++++-------
 src/ipa/libipa/agc_mean_luminance.h     |  27 ++++--
 src/ipa/libipa/camera_sensor_helper.cpp | 113 ++++++++++++++++++++++++
 src/ipa/libipa/camera_sensor_helper.h   |   6 ++
 src/ipa/libipa/exposure_mode_helper.cpp | 148 +++++++++++++++++++++-----------
 src/ipa/libipa/exposure_mode_helper.h   |  36 +++++---
 src/ipa/mali-c55/algorithms/agc.cpp     |  73 +++++++++++++---
 src/ipa/mali-c55/ipa_context.cpp        |   6 ++
 src/ipa/mali-c55/ipa_context.h          |  21 ++++-
 src/ipa/mali-c55/mali-c55.cpp           |  55 ++++++------
 src/ipa/rkisp1/algorithms/agc.cpp       |  92 +++++++++-----------
 src/ipa/rkisp1/ipa_context.cpp          |   3 -
 src/ipa/rkisp1/ipa_context.h            |   3 +-
 src/ipa/rkisp1/rkisp1.cpp               |  45 ++++++----
 19 files changed, 598 insertions(+), 273 deletions(-)
---
base-commit: 6d19c813b0a74770165e032e7f527c36b838e456
change-id: 20251017-exposure-limits-5c48252fe548

Best regards,