From patchwork Fri Oct 1 10:33:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14016 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1992DBDC71 for ; Fri, 1 Oct 2021 10:33:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D5534691AC; Fri, 1 Oct 2021 12:33:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dlx1rHrH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A40E691AA for ; Fri, 1 Oct 2021 12:33:39 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 97781A16; Fri, 1 Oct 2021 12:33:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084418; bh=KJM5JdQPyPLEELRvHAwJqWh/zTOh9FW5iDIbNvPAehk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dlx1rHrHUcieRlzkujQMvdL9T3W2JzdREzwedqYak5Aig6SGzrelLJrrti6S9MhaG DI5xcZpF8NPYikUOBV9FxuSKMZgt1s5IiX5CIOAGLWzhfSsaXMwFlQTbb8zPZZ4C2I hJsbrPT4hBpNddrOG7D1PGB5U+XJtR/0zG9a2BaI= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:19 +0900 Message-Id: <20211001103325.1077590-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/7] controls: Reorganize the AE-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We have multiple goals: - we need a lock of some sort, to instruct the AEGC to not update output results - we need manual modes, to override the values computed by the AEGC - we need to support seamless transitions from auto -> manual, and do so without flickering - we need custom minimum values for the manual controls, that is no magic values for enabling/disabling auto - all of these need to be done with AE sub-controls (exposure time, analogue gain) To achieve these goals, we introduce mode controls for the AE sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto state, and a disabled state. The disabled state has an internal one-way state change from locked to manual, triggered by the presence of the value-controls (ExposureTime and AnalogueGain). We then remove the AeEnable control, as it is a redundant control in the face of these two mode controls. We also remove AeLocked, as it is insufficient for reporting the AE state, and we promote AeState to non-draft to fill its role. Notably, the locked state is removed, since this information can be obtained from the aforementioned mode controls. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- Changes in v2: - No changes, just resubmitting at the head of this series so that it's together and so that /people will actually see it/ Initial version: Still RFC as I haven't updated the users of the control yet, and I want to check that these are the controls and docs that we want. We've decided that the "master AE control" will be implemented by a helper... but looking at uvcvideo and the V4L2 controls I'm wondering if such helper should come earlier than later? --- src/libcamera/control_ids.yaml | 215 +++++++++++++++++++++++---------- 1 file changed, 150 insertions(+), 65 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 9d4638ae..18b186e8 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -7,23 +7,61 @@ # Unless otherwise stated, all controls are bi-directional, i.e. they can be # set through Request::controls() and returned out through Request::metadata(). controls: - - AeEnable: - type: bool - description: | - Enable or disable the AE. - - \sa ExposureTime AnalogueGain - - - AeLocked: - type: bool + - AeState: + type: int32_t description: | - Report the lock status of a running AE algorithm. - - If the AE algorithm is locked the value shall be set to true, if it's - converging it shall be set to false. If the AE algorithm is not - running the control shall not be present in the metadata control list. + Control to report the current AE algorithm state. Enabling or disabling + any of the AE-related mode controls (eg. AnalogueGainMode, + ExposureTimeMode) is not required to reset the state to + AeStateInactive. The camera device can do several state transitions + between two results, if it is allowed by the state transition table. + For example, AeStateInactive may never actually be seen in a result. + + The state in the result is the state for this image (in sync with this + image). If AE state becomes AeStateConverged, then the image data + associated with the result should be good to use. + + As some AE algorithms may still be running when any of the AE-related + controls are in manual mode, the states are valid even when the mode + controls are set to Disabled. To know if this image image used AE or + manual (or a mixture), check the relevant modes (eg. AnalogueGainMode, + ExposureTimeMode). + + \sa AnalogueGain + \sa AnalogueGainMode + \sa ExposureTime + \sa ExposureTimeMode - \sa AeEnable + enum: + - name: AeStateInactive + value: 0 + description: | + The AE algorithm is inactive. + If the camera initiates an AE scan, the state shall go to Searching. + - name: AeStateSearching + value: 1 + description: | + The AE algorithm has not converged yet. + If the camera finishes an AE scan, the state shall go to Converged. + If the camera finishes an AE scan, but flash is required, the state + shall go to FlashRequired. + - name: AeStateConverged + value: 2 + description: | + The AE algorithm has converged. + If the camera initiates an AE scan, the state shall go to Searching. + - name: AeStateFlashRequired + value: 3 + description: | + The AE algorithm would need a flash for good results + If the camera initiates an AE scan, the state shall go to Searching. + If AeLock is on, the state shall go to Locked. + - name: AeStatePrecapture + value: 4 + description: | + The AE algorithm has started a pre-capture metering session. + After the sequence is finished, the state shall go to Converged if + \sa AePrecaptureTrigger # AeMeteringMode needs further attention: # - Auto-generate max enum value. @@ -93,6 +131,17 @@ controls: how the desired total exposure is divided between the shutter time and the sensor's analogue gain. The exposure modes are platform specific, and not all exposure modes may be supported. + + When one of AnalogueGainMode or ExposureTimeMode is set to Disabled and + a corresponding manual control is provided, AeExposureMode may be + ignored. This is because, for example, if AeExposureMode is set to + ExposureLong but ExposureTimeMode is Disabled and a short ExposureTime + is provided (AnalogueGainMode set to Auto), then the AeExposureMode + doesn't make sense. + + \sa AnalogueGainMode + \sa ExposureTimeMode + enum: - name: ExposureNormal value: 0 @@ -111,13 +160,15 @@ controls: type: float description: | Specify an Exposure Value (EV) parameter. The EV parameter will only be - applied if the AE algorithm is currently enabled. + applied if the AE algorithm is currently enabled, that is, at least one + of AnalogueGainMode and ExposureTimeMode are auto. By convention EV adjusts the exposure as log2. For example EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x]. - \sa AeEnable + \sa AnalogueGainMode + \sa ExposureTimeMode - ExposureTime: type: int32_t @@ -125,17 +176,49 @@ controls: Exposure time (shutter speed) for the frame applied in the sensor device. This value is specified in micro-seconds. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. Its + presence in a request acts as a trigger to switch to the internal + manual mode within ExposureTimeModeDisabled. + + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. - \sa AnalogueGain AeEnable + \sa AnalogueGain + \sa ExposureTimeMode - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + - ExposureTimeMode: + type: int32_t + description: | + Control to set and report the source of the exposure time that will be + applied. + + \sa ExposureTime + enum: + - name: ExposureTimeModeAuto + value: 0 + description: | + The exposure time will be calculated automatically and set by the + AE algorithm. + - name: ExposureTimeModeDisabled + value: 1 + description: | + The exposure time will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in ExposureTime. + + This Disabled state has two internal states; locked and manual. + When the Disabled state is first entered, the internal state will + be locked, and the latest exposure time value set by the AE + algorithm will be used (or the default ExposureTime value, if the + mode started out as Disabled). When an ExposureTime is provided, + then the internal state will go to manual, and the provided value + will be used for the exposure time. Going from manual to locked is + not possible. If an ExposureTime value is provided in the same + request as going into the Disabled state, then the internal locked + state will be skipped, and the internal state will start out as + manual. - AnalogueGain: type: float @@ -144,17 +227,49 @@ controls: The value of the control specifies the gain multiplier applied to all colour channels. This value cannot be lower than 1.0. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. Its + presence in a request acts as a trigger to switch to the internal + manual mode within ExposureTimeModeDisabled. - \sa ExposureTime AeEnable + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. + + \sa ExposureTime + \sa AnalogueGainMode + + - AnalogueGainMode: + type: int32_t + description: | + Control to set and report the source of the analogue gain that will be + applied. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + \sa AnalogueGain + enum: + - name: AnalogueGainModeAuto + value: 0 + description: | + The analogue gain will be calculated automatically and set by the + AE algorithm. + - name: AnalogueGainModeDisabled + value: 1 + description: | + The analogue gain will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in AnalogueGain. + + This Disabled state has two internal states; locked and manual. + When the Disabled state is first entered, the internal state will + be locked, and the latest analogue gain value set by the AE + algorithm will be used (or the default AnalogueGain value, if the + mode started out as Disabled). When an AnalogueGain is provided, + then the internal state will go to manual, and the provided value + will be used for the analogue gain. Going from manual to locked is + not possible. If an AnalogueGain value is provided in the same + request as going into the Disabled state, then the internal locked + state will be skipped, and the internal state will start out as + manual. - Brightness: type: float @@ -477,36 +592,6 @@ controls: High quality aberration correction which might reduce the frame rate. - - AeState: - type: int32_t - draft: true - description: | - Control to report the current AE algorithm state. Currently identical to - ANDROID_CONTROL_AE_STATE. - - Current state of the AE algorithm. - enum: - - name: AeStateInactive - value: 0 - description: The AE algorithm is inactive. - - name: AeStateSearching - value: 1 - description: The AE algorithm has not converged yet. - - name: AeStateConverged - value: 2 - description: The AE algorithm has converged. - - name: AeStateLocked - value: 3 - description: The AE algorithm is locked. - - name: AeStateFlashRequired - value: 4 - description: The AE algorithm would need a flash for good results - - name: AeStatePrecapture - value: 5 - description: | - The AE algorithm has started a pre-capture metering session. - \sa AePrecaptureTrigger - - AfState: type: int32_t draft: true