{"id":16511,"url":"https://patchwork.libcamera.org/api/1.1/patches/16511/?format=json","web_url":"https://patchwork.libcamera.org/patch/16511/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220701154701.354052-1-jacopo@jmondi.org>","date":"2022-07-01T15:46:53","name":"[libcamera-devel,1/9] controls: Reorganize the AE-related controls","commit_ref":null,"pull_url":null,"state":"superseded","archived":true,"hash":"b97c963aa3cb929be9c7cade27571b3bc0b5af9d","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/1.1/people/3/?format=json","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/16511/mbox/","series":[{"id":3243,"url":"https://patchwork.libcamera.org/api/1.1/series/3243/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3243","date":"2022-07-01T15:46:53","name":"[libcamera-devel,1/9] controls: Reorganize the AE-related controls","version":1,"mbox":"https://patchwork.libcamera.org/series/3243/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/16511/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/16511/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CD13CBD808\n\tfor ;\n\tFri, 1 Jul 2022 15:47:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1EDCA6564B;\n\tFri, 1 Jul 2022 17:47:09 +0200 (CEST)","from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net\n\t[IPv6:2001:4b98:dc4:8::224])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CAB560552\n\tfor ;\n\tFri, 1 Jul 2022 17:47:07 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id 012DAE0004;\n\tFri, 1 Jul 2022 15:47:06 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1656690429;\n\tbh=OsqbNL+ygZCUWbd+OUr+JnjJsV2yynWegpQps1cBmDI=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=gTNj4JIUGiywRutm6QmzBKxsE8Ubd/NUqhAP3rbKDhhWEYeaDjmKnC0iU6pPLNyl3\n\tYxYB6jWgDHCoyKvasizDzM4xX2j/M+QXbFFfvOSbC5XinpIp/5XOT6Ri7wa5a+LFAA\n\tgw6x6rusOGYZUlRZD2d9ZQ1uvKepKSkcYYcIH9QD6jHr54m+DlenF9grrBe6YCGJeN\n\t91zX3Ho1drL52DpAQkCup/6VD6HgF2oc9zGGn/ew5x4JY4Kd29ot+4CZgmJZ6DoBEc\n\tSf0Y7AV90ygsVxHfFr5qG0dwARsmXehDCZ5w80ocf5yn7iGyA3IU7jC8c5FZ+BT2p/\n\tyqvhRC9MBwkHA==","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 1 Jul 2022 17:46:53 +0200","Message-Id":"<20220701154701.354052-1-jacopo@jmondi.org>","X-Mailer":"git-send-email 2.36.1","In-Reply-To":"<20220518134728.777709-2-paul.elder@ideasonboard.com>","References":"<20220518134728.777709-2-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 1/9] controls: Reorganize the AE-related\n\tcontrols","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Paul Elder via libcamera-devel \n\nWe have multiple goals:\n- we need a lock of some sort, to instruct the AEGC to not update output\n results\n- we need manual modes, to override the values computed by the AEGC\n- we need to support seamless transitions from auto -> manual, and do so\n without flickering\n- we need custom minimum values for the manual controls, that is no\n magic values for enabling/disabling auto\n- all of these need to be done with AE sub-controls (exposure time,\n analogue gain)\n\nTo achieve these goals, we introduce mode controls for the AE\nsub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto\nstate, and a disabled state. The disabled state has an internal one-way\nstate change from locked to manual, triggered by the presence of the\nvalue-controls (ExposureTime and AnalogueGain).\n\nWe then remove the AeEnable control, as it is a redundant control in the\nface of these two mode controls.\n\nWe also remove AeLocked, as it is insufficient for reporting the AE\nstate, and we promote AeState to non-draft to fill its role. Notably,\nthe locked state is removed, since this information can be obtained from\nthe aforementioned mode controls.\n\nBug: https://bugs.libcamera.org/show_bug.cgi?id=42\nBug: https://bugs.libcamera.org/show_bug.cgi?id=43\nBug: https://bugs.libcamera.org/show_bug.cgi?id=47\nSigned-off-by: Paul Elder \n---\n src/libcamera/control_ids.yaml | 262 +++++++++++++++++++++++++--------\n 1 file changed, 200 insertions(+), 62 deletions(-)","diff":"diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\nindex ecab3ae97260..21275fcb9a83 100644\n--- a/src/libcamera/control_ids.yaml\n+++ b/src/libcamera/control_ids.yaml\n@@ -7,23 +7,46 @@\n # Unless otherwise stated, all controls are bi-directional, i.e. they can be\n # set through Request::controls() and returned out through Request::metadata().\n controls:\n- - AeEnable:\n- type: bool\n+ - AeState:\n+ type: int32_t\n description: |\n- Enable or disable the AE.\n+ Control to report the AE algorithm state associated with the capture\n+ result.\n \n- \\sa ExposureTime AnalogueGain\n+ The state is still reported even if ExposureTimeMode or\n+ AnalogueGainMode is set to Disabled.\n \n- - AeLocked:\n- type: bool\n- description: |\n- Report the lock status of a running AE algorithm.\n+ \\sa AnalogueGain\n+ \\sa AnalogueGainMode\n+ \\sa ExposureTime\n+ \\sa ExposureTimeMode\n \n- If the AE algorithm is locked the value shall be set to true, if it's\n- converging it shall be set to false. If the AE algorithm is not\n- running the control shall not be present in the metadata control list.\n+ enum:\n+ - name: AeStateInactive\n+ value: 0\n+ description: |\n+ The AE algorithm is inactive.\n \n- \\sa AeEnable\n+ This state should be returned if both AnalogueGainMode and\n+ ExposureTimeMode are set to disabled (or one, if the camera only\n+ supports one of the two controls).\n+ - name: AeStateSearching\n+ value: 1\n+ description: |\n+ The AE algorithm has not converged yet.\n+\n+ This state should be returned if at least one of AnalogueGainMode\n+ or ExposureTimeMode is set to auto, and the AE algorithm hasn't\n+ converged yet. If the AE algorithm converges, the state shall go to\n+ AeStateConverged.\n+ - name: AeStateConverged\n+ value: 2\n+ description: |\n+ The AE algorithm has converged.\n+\n+ This state should be returned if at least one of AnalogueGainMode\n+ or ExposureTimeMode is set to auto, and the AE algorithm has\n+ converged.\n \n # AeMeteringMode needs further attention:\n # - Auto-generate max enum value.\n@@ -93,6 +116,13 @@ controls:\n how the desired total exposure is divided between the shutter time\n and the sensor's analogue gain. The exposure modes are platform\n specific, and not all exposure modes may be supported.\n+\n+ When one of AnalogueGainMode or ExposureTimeMode is set to Disabled,\n+ the fixed values will override any choices made by AeExposureMode.\n+\n+ \\sa AnalogueGainMode\n+ \\sa ExposureTimeMode\n+\n enum:\n - name: ExposureNormal\n value: 0\n@@ -111,13 +141,15 @@ controls:\n type: float\n description: |\n Specify an Exposure Value (EV) parameter. The EV parameter will only be\n- applied if the AE algorithm is currently enabled.\n+ applied if the AE algorithm is currently enabled, that is, at least one\n+ of AnalogueGainMode and ExposureTimeMode are auto.\n \n By convention EV adjusts the exposure as log2. For example\n EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment\n of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x].\n \n- \\sa AeEnable\n+ \\sa AnalogueGainMode\n+ \\sa ExposureTimeMode\n \n - ExposureTime:\n type: int32_t\n@@ -125,17 +157,85 @@ controls:\n Exposure time (shutter speed) for the frame applied in the sensor\n device. This value is specified in micro-seconds.\n \n- Setting this value means that it is now fixed and the AE algorithm may\n- not change it. Setting it back to zero returns it to the control of the\n- AE algorithm.\n+ This control will only take effect if ExposureTimeMode is Disabled. If\n+ this control is set when ExposureTimeMode is Auto, the value will be\n+ ignored and will not be retained.\n+\n+ When reported in metadata, this control indicates what exposure time\n+ was used for the current request, regardless of ExposureTimeMode.\n+ ExposureTimeMode will indicate the source of the exposure time value,\n+ whether it came from the AE algorithm or not.\n+\n+ \\sa AnalogueGain\n+ \\sa ExposureTimeMode\n+\n+ - ExposureTimeMode:\n+ type: int32_t\n+ description: |\n+ Controls the source of the exposure time that is applied to the image\n+ sensor. When set to Auto, the AE algorithm computes the exposure time\n+ and configures the image sensor accordingly. When set to Disabled,\n+ exposure time specified in ExposureTime is applied to the image sensor.\n+ If ExposureTime is not set, then the value last computed by the AE\n+ algorithm when the mode was Auto will be used.\n+\n+ If ExposureTime is not set and the mode is ExposureTimeModeDisabled and\n+ AE was never Auto (either because the camera started in Disabled mode,\n+ or Auto is not supported by the camera), the camera should use a\n+ best-effort default value.\n+\n+ When ExposureTimeMode is set Auto, the value set in ExposureTime is\n+ ignored and is not retained. This means that if ExposureTimeMode is set\n+ to Disabled and ExposureTime is not also set, the exposure time that\n+ was last computed by the AE algorithm while the mode was Auto will be\n+ applied to the sensor.\n+\n+ If ExposureTimeModeDisabled is supported, the ExposureTime control must\n+ also be supported.\n+\n+ The set of ExposureTimeMode modes that are supported by the camera must\n+ have an intersection with the supported set of AnalogueGainMode modes.\n \n- \\sa AnalogueGain AeEnable\n+ As it takes a few frames to apply the exposure time, there is a period of\n+ time between submitting a request with ExposureTimeMode set to Disabled\n+ and the exposure time component of the AE actually being disabled,\n+ during which the AE algorithm can still update the exposure time. If an\n+ application is switching from automatic and manual control and wishes\n+ to eliminate any flicker during the switch, the following procedure is\n+ recommended.\n \n- \\todo Document the interactions between AeEnable and setting a fixed\n- value for this control. Consider interactions with other AE features,\n- such as aperture and aperture/shutter priority mode, and decide if\n- control of which features should be automatically adjusted shouldn't\n- better be handled through a separate AE mode control.\n+ 1. Start with ExposureTimeMode set to Auto\n+\n+ 2. Set ExposureTimeMode to Disabled\n+\n+ 3. Wait for the first request to be output that has ExposureTimeMode\n+ set to Disabled\n+\n+ 4. Copy the value reported in ExposureTime into a new request, and\n+ submit it\n+\n+ 5. Proceed to run manual exposure time\n+\n+ \\sa ExposureTime\n+ enum:\n+ - name: ExposureTimeModeAuto\n+ value: 0\n+ description: |\n+ The exposure time will be calculated automatically and set by the\n+ AE algorithm. If ExposureTime is set while this mode is active, it\n+ will be ignored, and it will also not be retained.\n+ - name: ExposureTimeModeDisabled\n+ value: 1\n+ description: |\n+ The exposure time will not be updated by the AE algorithm. It will\n+ come from the last calculated value when the mode was Auto, or from\n+ the value specified in ExposureTime.\n+\n+ When transitioning from Auto to Disabled mode, the last computed\n+ exposure value is used until a new value is specified through the\n+ ExposureTime control. If an ExposureTime value is specified in the\n+ same request where the ExposureTimeMode is changed from Auto to\n+ Disabled, the provided ExposureTime is applied.\n \n - AnalogueGain:\n type: float\n@@ -144,17 +244,85 @@ controls:\n The value of the control specifies the gain multiplier applied to all\n colour channels. This value cannot be lower than 1.0.\n \n- Setting this value means that it is now fixed and the AE algorithm may\n- not change it. Setting it back to zero returns it to the control of the\n- AE algorithm.\n+ This control will only take effect if AnalogueGainMode is Disabled. If\n+ this control is set when AnalogueGainMode is Auto, the value will be\n+ ignored and will not be retained.\n+\n+ When reported in metadata, this control indicates what analogue gain\n+ was used for the current request, regardless of AnalogueGainMode.\n+ AnalogueGainMode will indicate the source of the analogue gain value,\n+ whether it came from the AE algorithm or not.\n+\n+ \\sa ExposureTime\n+ \\sa AnalogueGainMode\n+\n+ - AnalogueGainMode:\n+ type: int32_t\n+ description: |\n+ Controls the source of the analogue gain that is applied to the image\n+ sensor. When set to Auto, the AE algorithm computes the analogue gain\n+ and configures the image sensor accordingly. When set to Disabled,\n+ analogue gain specified in AnalogueGain is applied to the image sensor.\n+ If AnalogueGain is not set, then the value last computed by the AE\n+ algorithm when the mode was Auto will be used.\n+\n+ If AnalogueGain is not set and the mode is AnalogueGainModeDisabled and\n+ AE was never Auto (either because the camera started in Disabled mode,\n+ or Auto is not supported by the camera), the camera should use a\n+ best-effort default value.\n+\n+ When AnalogueGainMode is set Auto, the value set in AnalogueGain is\n+ ignored and is not retained. This means that if AnalogueGainMode is set\n+ to Disabled and AnalogueGain is not also set, the analogue gain that\n+ was last computed by the AE algorithm while the mode was Auto will be\n+ applied to the sensor.\n \n- \\sa ExposureTime AeEnable\n+ If AnalogueGainModeDisabled is supported, the AnalogueGain control must\n+ also be supported.\n+\n+ The set of AnalogueGainMode modes that are supported by the camera must\n+ have an intersection with the supported set of ExposureTimeMode modes.\n+\n+ As it takes a few frames to apply the analogue gain, there is a period of\n+ time between submitting a request with AnalogueGainMode set to Disabled\n+ and the analogue gain component of the AE actually being disabled,\n+ during which the AE algorithm can still update the analogue gain. If an\n+ application is switching from automatic and manual control and wishes\n+ to eliminate any flicker during the switch, the following procedure is\n+ recommended.\n+\n+ 1. Start with AnalogueGainMode set to Auto\n+\n+ 2. Set AnalogueGainMode to Disabled\n+\n+ 3. Wait for the first request to be output that has AnalogueGainMode\n+ set to Disabled\n+\n+ 4. Copy the value reported in AnalogueGain into a new request, and\n+ submit it\n+\n+ 5. Proceed to run manual analogue gain\n+\n+ \\sa AnalogueGain\n+ enum:\n+ - name: AnalogueGainModeAuto\n+ value: 0\n+ description: |\n+ The analogue gain will be calculated automatically and set by the\n+ AE algorithm. If AnalogueGain is set while this mode is active, it\n+ will be ignored, and it will also not be retained.\n+ - name: AnalogueGainModeDisabled\n+ value: 1\n+ description: |\n+ The analogue gain will not be updated by the AE algorithm. It will\n+ come from the last calculated value when the mode was Auto, or from\n+ the value specified in AnalogueGain.\n \n- \\todo Document the interactions between AeEnable and setting a fixed\n- value for this control. Consider interactions with other AE features,\n- such as aperture and aperture/shutter priority mode, and decide if\n- control of which features should be automatically adjusted shouldn't\n- better be handled through a separate AE mode control.\n+ When transitioning from Auto to Disabled mode the last computed\n+ gain value is used until a new value is specified through the\n+ AnalogueGain control. If an AnalogueGain value is specified in the\n+ same request where the AnalogueGainMode is set to Disabled, the\n+ provided AnalogueGain is applied.\n \n - Brightness:\n type: float\n@@ -767,36 +935,6 @@ controls:\n High quality aberration correction which might reduce the frame\n rate.\n \n- - AeState:\n- type: int32_t\n- draft: true\n- description: |\n- Control to report the current AE algorithm state. Currently identical to\n- ANDROID_CONTROL_AE_STATE.\n-\n- Current state of the AE algorithm.\n- enum:\n- - name: AeStateInactive\n- value: 0\n- description: The AE algorithm is inactive.\n- - name: AeStateSearching\n- value: 1\n- description: The AE algorithm has not converged yet.\n- - name: AeStateConverged\n- value: 2\n- description: The AE algorithm has converged.\n- - name: AeStateLocked\n- value: 3\n- description: The AE algorithm is locked.\n- - name: AeStateFlashRequired\n- value: 4\n- description: The AE algorithm would need a flash for good results\n- - name: AeStatePrecapture\n- value: 5\n- description: |\n- The AE algorithm has started a pre-capture metering session.\n- \\sa AePrecaptureTrigger\n-\n - AwbState:\n type: int32_t\n draft: true\n","prefixes":["libcamera-devel","1/9"]}