[libcamera-devel,v3,1/8] controls: Reorganize the AE-related controls
diff mbox series

Message ID 20211221043610.2512334-2-paul.elder@ideasonboard.com
State New
Delegated to: Paul Elder
Headers show
Series
  • The Great AE Changes
Related show

Commit Message

Paul Elder Dec. 21, 2021, 4:36 a.m. UTC
We have multiple goals:
- we need a lock of some sort, to instruct the AEGC to not update output
  results
- we need manual modes, to override the values computed by the AEGC
- we need to support seamless transitions from auto -> manual, and do so
  without flickering
- we need custom minimum values for the manual controls, that is no
  magic values for enabling/disabling auto
- all of these need to be done with AE sub-controls (exposure time,
  analogue gain)

To achieve these goals, we introduce mode controls for the AE
sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto
state, and a disabled state. The disabled state has an internal one-way
state change from locked to manual, triggered by the presence of the
value-controls (ExposureTime and AnalogueGain).

We then remove the AeEnable control, as it is a redundant control in the
face of these two mode controls.

We also remove AeLocked, as it is insufficient for reporting the AE
state, and we promote AeState to non-draft to fill its role. Notably,
the locked state is removed, since this information can be obtained from
the aforementioned mode controls.

Bug: https://bugs.libcamera.org/show_bug.cgi?id=42
Bug: https://bugs.libcamera.org/show_bug.cgi?id=43
Bug: https://bugs.libcamera.org/show_bug.cgi?id=47
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>

---
Changes in v3:
- improve wording of the control descriptions
  - make more succinct and clear
- add description of how to do a flickerless transition

Changes in v2:
- No changes, just resubmitting at the head of this series so that it's
  together and so that /people will actually see it/

Initial version:
Still RFC as I haven't updated the users of the control yet, and I want
to check that these are the controls and docs that we want.

We've decided that the "master AE control" will be implemented by a
helper... but looking at uvcvideo and the V4L2 controls I'm wondering if
such helper should come earlier than later?
---
 src/libcamera/control_ids.yaml | 239 ++++++++++++++++++++++++---------
 1 file changed, 177 insertions(+), 62 deletions(-)

Patch
diff mbox series

diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml
index 9d4638ae..84679317 100644
--- a/src/libcamera/control_ids.yaml
+++ b/src/libcamera/control_ids.yaml
@@ -7,23 +7,63 @@ 
 # Unless otherwise stated, all controls are bi-directional, i.e. they can be
 # set through Request::controls() and returned out through Request::metadata().
 controls:
-  - AeEnable:
-      type: bool
+  - AeState:
+      type: int32_t
       description: |
-        Enable or disable the AE.
+        Control to report the current AE algorithm state. The camera device can do
+        several state transitions between two results, if it is allowed by the
+        state transition table. For example, AeStateInactive may never
+        actually be seen in a result.
 
-        \sa ExposureTime AnalogueGain
+        The state in the result is the state for this request. If AE state
+        becomes AeStateConverged, then the image data associated with the
+        result should be good to use.
 
-  - AeLocked:
-      type: bool
-      description: |
-        Report the lock status of a running AE algorithm.
+        The state is still reported even if ExposureTimeMode or
+        AnalogueGainMode is set to Disabled.
 
-        If the AE algorithm is locked the value shall be set to true, if it's
-        converging it shall be set to false. If the AE algorithm is not
-        running the control shall not be present in the metadata control list.
+        \sa AnalogueGain
+        \sa AnalogueGainMode
+        \sa ExposureTime
+        \sa ExposureTimeMode
 
-        \sa AeEnable
+      enum:
+        - name: AeStateInactive
+          value: 0
+          description: |
+            The AE algorithm is inactive.
+            If the camera initiates an AE scan, the state shall go to
+            AeStateSearching.
+
+            Unlike the analogous control in Android, enabling or disabling any
+            of the AE-related mode controls (eg. AnalogueGainMode,
+            ExposureTimeMode) does not require the state to be reset to to
+            AeStateInactive.
+        - name: AeStateSearching
+          value: 1
+          description: |
+            The AE algorithm has not converged yet.
+            If the camera finishes an AE scan, the state shall go to
+            AeStateConverged. If the camera finishes an AE scan, but flash is
+            required, the state shall go to AeStateFlashRequired.
+        - name: AeStateConverged
+          value: 2
+          description: |
+            The AE algorithm has converged.
+            If the camera initiates an AE scan, the state shall go to
+            AeStateSearching.
+        - name: AeStateFlashRequired
+          value: 3
+          description: |
+            The AE algorithm would need a flash for good results.
+            If the camera initiates an AE scan, the state shall go to
+            AeStateSearching.
+        - name: AeStatePrecapture
+          value: 4
+          description: |
+            The AE algorithm has started a pre-capture metering session.
+            After the sequence is finished, the state shall go to AeStateConverged if
+            \sa AePrecaptureTrigger
 
   # AeMeteringMode needs further attention:
   # - Auto-generate max enum value.
@@ -93,6 +133,13 @@  controls:
         how the desired total exposure is divided between the shutter time
         and the sensor's analogue gain. The exposure modes are platform
         specific, and not all exposure modes may be supported.
+
+        When one of AnalogueGainMode or ExposureTimeMode is set to Disabled,
+        the fixed values will override any choices made by AeExposureMode.
+
+        \sa AnalogueGainMode
+        \sa ExposureTimeMode
+
       enum:
         - name: ExposureNormal
           value: 0
@@ -111,13 +158,15 @@  controls:
       type: float
       description: |
         Specify an Exposure Value (EV) parameter. The EV parameter will only be
-        applied if the AE algorithm is currently enabled.
+        applied if the AE algorithm is currently enabled, that is, at least one
+        of AnalogueGainMode and ExposureTimeMode are auto.
 
         By convention EV adjusts the exposure as log2. For example
         EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment
         of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x].
 
-        \sa AeEnable
+        \sa AnalogueGainMode
+        \sa ExposureTimeMode
 
   - ExposureTime:
       type: int32_t
@@ -125,17 +174,65 @@  controls:
         Exposure time (shutter speed) for the frame applied in the sensor
         device. This value is specified in micro-seconds.
 
-        Setting this value means that it is now fixed and the AE algorithm may
-        not change it. Setting it back to zero returns it to the control of the
-        AE algorithm.
+        This control will only take effect if ExposureTimeMode is Disabled. Its
+        presence in a request acts as a trigger to switch to the internal
+        manual mode within ExposureTimeModeDisabled.
+
+        When reported in metadata, this control indicates what exposure time
+        was used for the current request, regardless of ExposureTimeMode.
+        ExposureTimeMode will indicate the source of the exposure time value,
+        whether it came from the AE algorithm or not.
+
+        \sa AnalogueGain
+        \sa ExposureTimeMode
 
-        \sa AnalogueGain AeEnable
+  - ExposureTimeMode:
+      type: int32_t
+      description: |
+        Controls how the frame exposure time is computed. When set to Auto, the
+        AE algorithm computes the exposure time of the next frame and
+        configures the image sensor accordingly. When set to Disabled, the AE
+        algorithm stops updating the exposure time.
 
-        \todo Document the interactions between AeEnable and setting a fixed
-        value for this control. Consider interactions with other AE features,
-        such as aperture and aperture/shutter priority mode, and decide if
-        control of which features should be automatically adjusted shouldn't
-        better be handled through a separate AE mode control.
+        As the camera device has a pipeline of in-flight requests, there is a
+        period of time between submitting a request with ExposureTimeMode set
+        to Disabled and the exposure time component of the AE actually being
+        disabled, during which the AE algorithm can still update the exposure
+        time. If an application is switching from automatic and manual control
+        and wishes to eliminate any flicker during the switch, the following
+        procedure is recommended.
+
+        1. Start with ExposureTimeMode set to Auto
+
+        2. Set ExposureTimeMode to Disabled
+
+        3. Wait for the first request to be output that has ExposureTimeMode
+        set to Disabled
+
+        4. Copy the value reported in ExposureTime into a new request, and
+        submit it
+
+        5. Proceed to run manual exposure time
+
+        \sa ExposureTime
+      enum:
+        - name: ExposureTimeModeAuto
+          value: 0
+          description: |
+            The exposure time will be calculated automatically and set by the
+            AE algorithm.
+        - name: ExposureTimeModeDisabled
+          value: 1
+          description: |
+            The exposure time will not be updated by the AE algorithm. It will
+            come from the last calculated value when the mode was Auto, or from
+            the value specified in ExposureTime.
+
+            When transitioning from Auto to Disabled mode the last computed
+            exposure value is used until a new value is specified through the
+            ExposureTime control. If an ExposureTime value is specified in the
+            same request where the ExposureTimeMode is set to Disabled, the
+            provided ExposureTime is applied immediately.
 
   - AnalogueGain:
       type: float
@@ -144,17 +241,65 @@  controls:
         The value of the control specifies the gain multiplier applied to all
         colour channels. This value cannot be lower than 1.0.
 
-        Setting this value means that it is now fixed and the AE algorithm may
-        not change it. Setting it back to zero returns it to the control of the
-        AE algorithm.
+        This control will only take effect if ExposureTimeMode is Disabled. Its
+        presence in a request acts as a trigger to switch to the internal
+        manual mode within ExposureTimeModeDisabled.
+
+        When reported in metadata, this control indicates what exposure time
+        was used for the current request, regardless of ExposureTimeMode.
+        ExposureTimeMode will indicate the source of the exposure time value,
+        whether it came from the AE algorithm or not.
+
+        \sa ExposureTime
+        \sa AnalogueGainMode
+
+  - AnalogueGainMode:
+      type: int32_t
+      description: |
+        Controls how the analogue gain is computed. When set to Auto, the
+        AE algorithm computes the analogue gain of the next frame and
+        configures the image sensor accordingly. When set to Disabled, the AE
+        algorithm stops updating the analogue gain.
+
+        As the camera device has a pipeline of in-flight requests, there is a
+        period of time between submitting a request with AnalogueGainMode set
+        to Disabled and the analogue gain component of the AE actually being
+        disabled, during which the AE algorithm can still update the analogue
+        gain. If an application is switching from automatic and manual control
+        and wishes to eliminate any flicker during the switch, the following
+        procedure is recommended.
+
+        1. Start with AnalogueGainMode set to Auto
 
-        \sa ExposureTime AeEnable
+        2. Set AnalogueGainMode to Disabled
 
-        \todo Document the interactions between AeEnable and setting a fixed
-        value for this control. Consider interactions with other AE features,
-        such as aperture and aperture/shutter priority mode, and decide if
-        control of which features should be automatically adjusted shouldn't
-        better be handled through a separate AE mode control.
+        3. Wait for the first request to be output that has AnalogueGainMode
+        set to Disabled
+
+        4. Copy the value reported in AnalogueGain into a new request, and
+        submit it
+
+        5. Proceed to run manual analogue gain
+
+        \sa AnalogueGain
+      enum:
+        - name: AnalogueGainModeAuto
+          value: 0
+          description: |
+            The analogue gain will be calculated automatically and set by the
+            AE algorithm.
+        - name: AnalogueGainModeDisabled
+          value: 1
+          description: |
+            The analogue gain will not be updated by the AE algorithm. It will
+            come from the last calculated value when the mode was Auto, or from
+            the value specified in AnalogueGain.
+
+            When transitioning from Auto to Disabled mode the last computed
+            gain value is used until a new value is specified through the
+            AnalogueGain control. If an AnalogueGain value is specified in the
+            same request where the AnalogueGainMode is set to Disabled, the
+            provided AnalogueGain is applied immediately.
 
   - Brightness:
       type: float
@@ -477,36 +622,6 @@  controls:
             High quality aberration correction which might reduce the frame
             rate.
 
-  - AeState:
-      type: int32_t
-      draft: true
-      description: |
-       Control to report the current AE algorithm state. Currently identical to
-       ANDROID_CONTROL_AE_STATE.
-
-        Current state of the AE algorithm.
-      enum:
-        - name: AeStateInactive
-          value: 0
-          description: The AE algorithm is inactive.
-        - name: AeStateSearching
-          value: 1
-          description: The AE algorithm has not converged yet.
-        - name: AeStateConverged
-          value: 2
-          description: The AE algorithm has converged.
-        - name: AeStateLocked
-          value: 3
-          description: The AE algorithm is locked.
-        - name: AeStateFlashRequired
-          value: 4
-          description: The AE algorithm would need a flash for good results
-        - name: AeStatePrecapture
-          value: 5
-          description: |
-            The AE algorithm has started a pre-capture metering session.
-            \sa AePrecaptureTrigger
-
   - AfState:
       type: int32_t
       draft: true