[libcamera-devel,3/3] libcamera: control_ids: Add high-level documentation for AE

15972 diff mbox series

Add high-level documentation for AE, describing the states and the
transitions and how one would use the AE controls.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
---
 src/libcamera/control_ids.cpp.in | 82 ++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

Hi Paul,

I'm going to hijack this patch to bring up a question to David :-)

David, how do you envision the exposure time and gain controls to be
handled by the IPA when one of the two is in automatic mode and the
other one in manual mode ? Let's take an example. The camera is running,
with the gain set to manual mode and value x1.0. The exposure time is
automatic, and the AGC algorithm has converged and computed a proper
exposure value. At some point, in request N, the application modifies
the gain to x2.0. The frame captured for request N should have that gain
applied. How about the exposure time ? Do you expect the IPA to halve
the exposure time exactly for that frame (neither earlier nor later), or
do you expect that the AGC could need one or a few frames to converge to
the new correct exposure time ?

On Wed, May 18, 2022 at 03:47:28PM +0200, Paul Elder via libcamera-devel wrote:
> Add high-level documentation for AE, describing the states and the
> transitions and how one would use the AE controls.
> 
> Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
> ---
>  src/libcamera/control_ids.cpp.in | 82 ++++++++++++++++++++++++++++++++
>  1 file changed, 82 insertions(+)
> 
> diff --git a/src/libcamera/control_ids.cpp.in b/src/libcamera/control_ids.cpp.in
> index 5fb1c2c3..62451d7f 100644
> --- a/src/libcamera/control_ids.cpp.in
> +++ b/src/libcamera/control_ids.cpp.in
> @@ -10,6 +10,88 @@
>  #include <libcamera/control_ids.h>
>  #include <libcamera/controls.h>
>  
> +/**
> + * \page AE Auto-exposure Support
> + *
> + * libcamera offers the following controls related to exposure and gain:
> + * - AnalogueGain
> + * - AnalogueGainMode
> + * - ExposureTime
> + * - ExposureTimeMode
> + * - AeState
> + *
> + * Auto-exposure and auto-gain can be enabled and disabled separately using the
> + * ExposureTimeMode and AnalogueGainMode controls respectively. There is no
> + * overarching AeEnable control.
> + *
> + * For each of exposure and gain, we can model it with three states: auto,
> + * locked, and manual. Note that AnalogueGainMode and ExposureTimeMode only
> + * have two values, as the locked state is simulated.
> + *
> + *     /---------------------------------\
> + *     |                                 |
> + *     V                                 |
> + * +--------+                        +--------+
> + * |        | ---------------------> |        |
> + * |  Auto  |       +--------+       | Manual |
> + * |        | ----> | Locked | ----> |        |
> + * +--------+       +--------+       +--------+
> + *     ^                |
> + *     |                |
> + *     \----------------/
> + *
> + * Notice from the state diagram that locked to manual is a one-way state
> + * change, as the reverse direction is nonsensical (see the design document for
> + * more details on this topic).
> + *
> + * The exposure/gain is in the Auto state when
> + * ExposureTimeMode/AnalogueGainMode is set to Auto. In this state, the value
> + * that is computed by the AE algorithm is applied to the image sensor. Any
> + * value that is supplied in the ExposureTime/AnalogueGain control is ignored
> + * and is not retained.
> + *
> + * If ExposureTimeMode/AnalogueGainMode is set to Disabled, it can put us in
> + * either the Locked or Manual state. The difference is in if
> + * ExposureTime/AnalogueGain has been supplied. If it has not yet been
> + * supplied, then we are in the Locked state. If it has been supplied, then we
> + * are in the Manual state.
> + *
> + * In both the Locked state and the Manual state the exposure/gain value does
> + * not come from the AE algorithm. In the Locked state the value comes from the
> + * last value computed by the AE algorithm while the state was Auto, or if the
> + * state was never Auto (e.g. we started in Locked, or the camera doesn't
> + * support Auto), then the value should be a best-effort default value. In the
> + * Manual state the value comes from the value supplied in the
> + * ExposureTime/AnalogueGain control.
> + *
> + * To transition from the Locked state to the Manual state, a value needs to be
> + * submitted in ExposureTime/AnalogueGain. Once the state has transitioned to
> + * Manual, then this value will be retained, and doesn't need to be resubmitted
> + * if it doesn't change.
> + *
> + * To transition to the Auto state, simply set
> + * ExposureTimeMode/AnalogueGainMode to Auto.
> + *
> + *
> + * The AeState metadata reports the state of the AE algorithm. As AE cannot
> + * compute exposure and gain separately, the state of the AE component is
> + * unified. There are three states: Inactive, Searching, and Converged.
> + *
> + * The state shall be Inactive if both ExposureTimeMode and AnalogueGainMode
> + * are set to Disabled. If the camera only supports one of the two controls,
> + * then the state shall be Inactive if that one control is set to Disabled. If
> + * the camera does not support Disabled for at least one of the two controls,
> + * then the state will never be Inactive, as AE will always be running.
> + *
> + * The state shall be Searching if at least one of exposure or gain calculated
> + * by the AE algorithm is used (that is, at least one of the two modes is Auto),
> + * *and* the value(s) have not converged yet.
> + *
> + * The state shall be Converged if at least one of exposure or gain calculated
> + * by the AE algorithm is used (that is, at least one of the two modes is Auto),
> + * *and* the value(s) have converged.
> + */
> +
>  /**
>   * \file control_ids.h
>   * \brief Camera control identifiers