From patchwork Fri Jul 1 15:46:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 16511 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CD13CBD808 for ; Fri, 1 Jul 2022 15:47:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1EDCA6564B; Fri, 1 Jul 2022 17:47:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656690429; bh=OsqbNL+ygZCUWbd+OUr+JnjJsV2yynWegpQps1cBmDI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=gTNj4JIUGiywRutm6QmzBKxsE8Ubd/NUqhAP3rbKDhhWEYeaDjmKnC0iU6pPLNyl3 YxYB6jWgDHCoyKvasizDzM4xX2j/M+QXbFFfvOSbC5XinpIp/5XOT6Ri7wa5a+LFAA gw6x6rusOGYZUlRZD2d9ZQ1uvKepKSkcYYcIH9QD6jHr54m+DlenF9grrBe6YCGJeN 91zX3Ho1drL52DpAQkCup/6VD6HgF2oc9zGGn/ew5x4JY4Kd29ot+4CZgmJZ6DoBEc Sf0Y7AV90ygsVxHfFr5qG0dwARsmXehDCZ5w80ocf5yn7iGyA3IU7jC8c5FZ+BT2p/ yqvhRC9MBwkHA== Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::224]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CAB560552 for ; Fri, 1 Jul 2022 17:47:07 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id 012DAE0004; Fri, 1 Jul 2022 15:47:06 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Jul 2022 17:46:53 +0200 Message-Id: <20220701154701.354052-1-jacopo@jmondi.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220518134728.777709-2-paul.elder@ideasonboard.com> References: <20220518134728.777709-2-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/9] controls: Reorganize the AE-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Paul Elder via libcamera-devel We have multiple goals: - we need a lock of some sort, to instruct the AEGC to not update output results - we need manual modes, to override the values computed by the AEGC - we need to support seamless transitions from auto -> manual, and do so without flickering - we need custom minimum values for the manual controls, that is no magic values for enabling/disabling auto - all of these need to be done with AE sub-controls (exposure time, analogue gain) To achieve these goals, we introduce mode controls for the AE sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto state, and a disabled state. The disabled state has an internal one-way state change from locked to manual, triggered by the presence of the value-controls (ExposureTime and AnalogueGain). We then remove the AeEnable control, as it is a redundant control in the face of these two mode controls. We also remove AeLocked, as it is insufficient for reporting the AE state, and we promote AeState to non-draft to fill its role. Notably, the locked state is removed, since this information can be obtained from the aforementioned mode controls. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- src/libcamera/control_ids.yaml | 262 +++++++++++++++++++++++++-------- 1 file changed, 200 insertions(+), 62 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index ecab3ae97260..21275fcb9a83 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -7,23 +7,46 @@ # Unless otherwise stated, all controls are bi-directional, i.e. they can be # set through Request::controls() and returned out through Request::metadata(). controls: - - AeEnable: - type: bool + - AeState: + type: int32_t description: | - Enable or disable the AE. + Control to report the AE algorithm state associated with the capture + result. - \sa ExposureTime AnalogueGain + The state is still reported even if ExposureTimeMode or + AnalogueGainMode is set to Disabled. - - AeLocked: - type: bool - description: | - Report the lock status of a running AE algorithm. + \sa AnalogueGain + \sa AnalogueGainMode + \sa ExposureTime + \sa ExposureTimeMode - If the AE algorithm is locked the value shall be set to true, if it's - converging it shall be set to false. If the AE algorithm is not - running the control shall not be present in the metadata control list. + enum: + - name: AeStateInactive + value: 0 + description: | + The AE algorithm is inactive. - \sa AeEnable + This state should be returned if both AnalogueGainMode and + ExposureTimeMode are set to disabled (or one, if the camera only + supports one of the two controls). + - name: AeStateSearching + value: 1 + description: | + The AE algorithm has not converged yet. + + This state should be returned if at least one of AnalogueGainMode + or ExposureTimeMode is set to auto, and the AE algorithm hasn't + converged yet. If the AE algorithm converges, the state shall go to + AeStateConverged. + - name: AeStateConverged + value: 2 + description: | + The AE algorithm has converged. + + This state should be returned if at least one of AnalogueGainMode + or ExposureTimeMode is set to auto, and the AE algorithm has + converged. # AeMeteringMode needs further attention: # - Auto-generate max enum value. @@ -93,6 +116,13 @@ controls: how the desired total exposure is divided between the shutter time and the sensor's analogue gain. The exposure modes are platform specific, and not all exposure modes may be supported. + + When one of AnalogueGainMode or ExposureTimeMode is set to Disabled, + the fixed values will override any choices made by AeExposureMode. + + \sa AnalogueGainMode + \sa ExposureTimeMode + enum: - name: ExposureNormal value: 0 @@ -111,13 +141,15 @@ controls: type: float description: | Specify an Exposure Value (EV) parameter. The EV parameter will only be - applied if the AE algorithm is currently enabled. + applied if the AE algorithm is currently enabled, that is, at least one + of AnalogueGainMode and ExposureTimeMode are auto. By convention EV adjusts the exposure as log2. For example EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x]. - \sa AeEnable + \sa AnalogueGainMode + \sa ExposureTimeMode - ExposureTime: type: int32_t @@ -125,17 +157,85 @@ controls: Exposure time (shutter speed) for the frame applied in the sensor device. This value is specified in micro-seconds. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. If + this control is set when ExposureTimeMode is Auto, the value will be + ignored and will not be retained. + + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. + + \sa AnalogueGain + \sa ExposureTimeMode + + - ExposureTimeMode: + type: int32_t + description: | + Controls the source of the exposure time that is applied to the image + sensor. When set to Auto, the AE algorithm computes the exposure time + and configures the image sensor accordingly. When set to Disabled, + exposure time specified in ExposureTime is applied to the image sensor. + If ExposureTime is not set, then the value last computed by the AE + algorithm when the mode was Auto will be used. + + If ExposureTime is not set and the mode is ExposureTimeModeDisabled and + AE was never Auto (either because the camera started in Disabled mode, + or Auto is not supported by the camera), the camera should use a + best-effort default value. + + When ExposureTimeMode is set Auto, the value set in ExposureTime is + ignored and is not retained. This means that if ExposureTimeMode is set + to Disabled and ExposureTime is not also set, the exposure time that + was last computed by the AE algorithm while the mode was Auto will be + applied to the sensor. + + If ExposureTimeModeDisabled is supported, the ExposureTime control must + also be supported. + + The set of ExposureTimeMode modes that are supported by the camera must + have an intersection with the supported set of AnalogueGainMode modes. - \sa AnalogueGain AeEnable + As it takes a few frames to apply the exposure time, there is a period of + time between submitting a request with ExposureTimeMode set to Disabled + and the exposure time component of the AE actually being disabled, + during which the AE algorithm can still update the exposure time. If an + application is switching from automatic and manual control and wishes + to eliminate any flicker during the switch, the following procedure is + recommended. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + 1. Start with ExposureTimeMode set to Auto + + 2. Set ExposureTimeMode to Disabled + + 3. Wait for the first request to be output that has ExposureTimeMode + set to Disabled + + 4. Copy the value reported in ExposureTime into a new request, and + submit it + + 5. Proceed to run manual exposure time + + \sa ExposureTime + enum: + - name: ExposureTimeModeAuto + value: 0 + description: | + The exposure time will be calculated automatically and set by the + AE algorithm. If ExposureTime is set while this mode is active, it + will be ignored, and it will also not be retained. + - name: ExposureTimeModeDisabled + value: 1 + description: | + The exposure time will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in ExposureTime. + + When transitioning from Auto to Disabled mode, the last computed + exposure value is used until a new value is specified through the + ExposureTime control. If an ExposureTime value is specified in the + same request where the ExposureTimeMode is changed from Auto to + Disabled, the provided ExposureTime is applied. - AnalogueGain: type: float @@ -144,17 +244,85 @@ controls: The value of the control specifies the gain multiplier applied to all colour channels. This value cannot be lower than 1.0. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if AnalogueGainMode is Disabled. If + this control is set when AnalogueGainMode is Auto, the value will be + ignored and will not be retained. + + When reported in metadata, this control indicates what analogue gain + was used for the current request, regardless of AnalogueGainMode. + AnalogueGainMode will indicate the source of the analogue gain value, + whether it came from the AE algorithm or not. + + \sa ExposureTime + \sa AnalogueGainMode + + - AnalogueGainMode: + type: int32_t + description: | + Controls the source of the analogue gain that is applied to the image + sensor. When set to Auto, the AE algorithm computes the analogue gain + and configures the image sensor accordingly. When set to Disabled, + analogue gain specified in AnalogueGain is applied to the image sensor. + If AnalogueGain is not set, then the value last computed by the AE + algorithm when the mode was Auto will be used. + + If AnalogueGain is not set and the mode is AnalogueGainModeDisabled and + AE was never Auto (either because the camera started in Disabled mode, + or Auto is not supported by the camera), the camera should use a + best-effort default value. + + When AnalogueGainMode is set Auto, the value set in AnalogueGain is + ignored and is not retained. This means that if AnalogueGainMode is set + to Disabled and AnalogueGain is not also set, the analogue gain that + was last computed by the AE algorithm while the mode was Auto will be + applied to the sensor. - \sa ExposureTime AeEnable + If AnalogueGainModeDisabled is supported, the AnalogueGain control must + also be supported. + + The set of AnalogueGainMode modes that are supported by the camera must + have an intersection with the supported set of ExposureTimeMode modes. + + As it takes a few frames to apply the analogue gain, there is a period of + time between submitting a request with AnalogueGainMode set to Disabled + and the analogue gain component of the AE actually being disabled, + during which the AE algorithm can still update the analogue gain. If an + application is switching from automatic and manual control and wishes + to eliminate any flicker during the switch, the following procedure is + recommended. + + 1. Start with AnalogueGainMode set to Auto + + 2. Set AnalogueGainMode to Disabled + + 3. Wait for the first request to be output that has AnalogueGainMode + set to Disabled + + 4. Copy the value reported in AnalogueGain into a new request, and + submit it + + 5. Proceed to run manual analogue gain + + \sa AnalogueGain + enum: + - name: AnalogueGainModeAuto + value: 0 + description: | + The analogue gain will be calculated automatically and set by the + AE algorithm. If AnalogueGain is set while this mode is active, it + will be ignored, and it will also not be retained. + - name: AnalogueGainModeDisabled + value: 1 + description: | + The analogue gain will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in AnalogueGain. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + When transitioning from Auto to Disabled mode the last computed + gain value is used until a new value is specified through the + AnalogueGain control. If an AnalogueGain value is specified in the + same request where the AnalogueGainMode is set to Disabled, the + provided AnalogueGain is applied. - Brightness: type: float @@ -767,36 +935,6 @@ controls: High quality aberration correction which might reduce the frame rate. - - AeState: - type: int32_t - draft: true - description: | - Control to report the current AE algorithm state. Currently identical to - ANDROID_CONTROL_AE_STATE. - - Current state of the AE algorithm. - enum: - - name: AeStateInactive - value: 0 - description: The AE algorithm is inactive. - - name: AeStateSearching - value: 1 - description: The AE algorithm has not converged yet. - - name: AeStateConverged - value: 2 - description: The AE algorithm has converged. - - name: AeStateLocked - value: 3 - description: The AE algorithm is locked. - - name: AeStateFlashRequired - value: 4 - description: The AE algorithm would need a flash for good results - - name: AeStatePrecapture - value: 5 - description: | - The AE algorithm has started a pre-capture metering session. - \sa AePrecaptureTrigger - - AwbState: type: int32_t draft: true