[{"id":22537,"web_url":"https://patchwork.libcamera.org/comment/22537/","msgid":"<0165a34f-a35c-5eae-6b3c-0e878441c455@ideasonboard.com>","date":"2022-04-01T07:03:31","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi David,\n\nThanks for your patch !\n\nOn 31/03/2022 17:17, David Plowman wrote:\n> This patch describes a series of controls that allow applications to\n> drive AF algorithms:\n> \n> AfMode - manual, auto or continuous\n> AfLensPosition - set lens position to macro/hyperfocal/infinity\n> AfRange - full, macro or normal\n> AfSpeed - fast or slow\n> AfWindows - AF window locations\n> AfTrigger - start (trigger an AF scan) or cancel\n> AfPause - pause continuous AF\n> LensPosition - position of lens from lens driver\n> AfState - reports the mode, whether scanning/success/failure\n> ---\n> src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> 1 file changed, 313 insertions(+), 60 deletions(-)\n> \n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index 9d4638ae..23607368 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -406,27 +406,6 @@ controls:\n> The camera will cancel any active or completed metering sequence.\n> The AE algorithm is reset to its initial state.\n> \n> - - AfTrigger:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control for AF trigger. Currently identical to\n> - ANDROID_CONTROL_AF_TRIGGER.\n> -\n> - Whether the camera device will trigger autofocus for this request.\n> - enum:\n> - - name: AfTriggerIdle\n> - value: 0\n> - description: The trigger is idle.\n> - - name: AfTriggerStart\n> - value: 1\n> - description: The AF routine is started by the camera.\n> - - name: AfTriggerCancel\n> - value: 2\n> - description: |\n> - The camera will cancel any active trigger and the AF routine is\n> - reset to its initial state.\n> -\n> - NoiseReductionMode:\n> type: int32_t\n> draft: true\n> @@ -507,45 +486,6 @@ controls:\n> The AE algorithm has started a pre-capture metering session.\n> \\sa AePrecaptureTrigger\n> \n> - - AfState:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control to report the current AF algorithm state. Currently identical to\n> - ANDROID_CONTROL_AF_STATE.\n> -\n> - Current state of the AF algorithm.\n> - enum:\n> - - name: AfStateInactive\n> - value: 0\n> - description: The AF algorithm is inactive.\n> - - name: AfStatePassiveScan\n> - value: 1\n> - description: |\n> - AF is performing a passive scan of the scene in continuous\n> - auto-focus mode.\n> - - name: AfStatePassiveFocused\n> - value: 2\n> - description: |\n> - AF believes the scene is in focus, but might restart scanning.\n> - - name: AfStateActiveScan\n> - value: 3\n> - description: |\n> - AF is performing a scan triggered by an AF trigger request.\n> - \\sa AfTrigger\n> - - name: AfStateFocusedLock\n> - value: 4\n> - description: |\n> - AF believes has focused correctly and has locked focus.\n> - - name: AfStateNotFocusedLock\n> - value: 5\n> - description: |\n> - AF has not been able to focus and has locked.\n> - - name: AfStatePassiveUnfocused\n> - value: 6\n> - description: |\n> - AF has completed a passive scan without finding focus.\n> -\n> - AwbState:\n> type: int32_t\n> draft: true\n> @@ -690,4 +630,317 @@ controls:\n> value. All of the custom test patterns will be static (that is the\n> raw image must not vary from frame to frame).\n> \n> + - AfMode:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the mode of the AF (autofocus) algorithm. Applications\n> + are allowed to set a new mode, and to send additional controls for\n> + that new mode, in the same request.\n> +\n> + An implementation may choose not to implement all the modes.\n> +\n> + enum:\n> + - name: AfModeManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode. In this mode it will never\n> + perform any action nor move the lens of its own accord, but an\n> + application can set controls to move the lens.\n> +\n> + In this mode the AfState will always report AfStateManual.\n> + - name: AfModeAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode. This means that the algorithm\n> + will never move the lens or change state unless the AfTrigger\n> + control is used. The AfTrigger control can be used to initiate a\n> + focus scan, the results of which will also be reported by AfState.\n> +\n> + If the autofocus algorithm is moved from AfModeAuto to another\n> + mode while a scan is in progress, the scan is cancelled\n> + immediately, without waiting for the scan to finish.\n> +\n> + When first entering this mode the AfState will report\n> + AfStateAuto. When a trigger control is sent, AfState will\n> + report AfStateAutoScanning for a period before spontaneously\n> + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> + the outcome of the scan. It will remain in this state until\n> + another scan is initiated by the AfTrigger control. If a scan is\n> + cancelled (without changing to another mode), AfState will return\n> + to AfStateAuto.\n> + - name: AfModeContinuous\n> + value: 2\n> + description: |\n> + The AF algorithm is in continuous mode. This means that the lens\n> + can re-start a scan spontaneously at any moment, without any user\n> + intervention. The AfState still reports whether the algorithm is\n> + currently scanning or not, though the application has no ability\n> + to initiate or cancel scans (though it can \"pause\" them), nor to\n> + move the lens for itself.\n> +\n> + When set to AfModeContinuous, the system will immediately initiate\n> + a scan so AfState will report AfStateContinuousScanning, and will\n> + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> + depending on the scan result.\n> +\n> + - AfLensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the position of the lens to one of the following\n> + predefined locations. This control only has any effect when the AfMode\n> + is set to AfModeManual.\n> +\n> + This control is distinct from the LensPosition control, which sets the\n> + lens position using the lens driver's units.\n> + enum:\n> + - name: AfLensPositionMacro\n> + value: 0\n> + description: The closest focal position of the lens.\n> + - name: AfLensPositionHyperfocal\n> + value: 1\n> + description: Hyperfocal position.\n> + - name: AfLensPositionInfinity\n> + value: 2\n> + description: The furthest focal position (usually infinity).\n\nThis is a clear call for VCMHelper classes and vcm tuning files :-).\n\n> +\n> + - AfRange:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the range of focus distances that is scanned. An\n> + implementation may choose not to implement all the options here.\n> + enum:\n> + - name: AfRangeNormal\n> + value: 0\n> + description: |\n> + A wide range of focus distances is scanned, all the way from\n> + infinity down to close distances, though depending on the\n> + implementation, possibly not including the very closest macro\n> + positions.\n> + - name: AfRangeMacro\n> + value: 1\n> + description: Only close distances are scanned.\n> + - name: AfRangeFull\n> + value: 2\n> + description: |\n> + The full range of focus distances is scanned just as with\n> + AfRangeNormal but this time including the very closest macro\n> + positions.\n> +\n> + - AfSpeed:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control that determines whether the AF algorithm is to move the lens\n> + as quickly as possible or more steadily. For example, during video\n> + recording it may be desirable not to move the lens too abruptly, but\n> + when in a preview mode (waiting for a still capture) it may be\n> + helpful to move the lens as quickly as is reasonably possible.\n> + enum:\n> + - name: AfSpeedNormal\n> + value: 0\n> + description: Move the lens at its usual speed.\n> + - name: AfSpeedFast\n> + value: 1\n> + description: Move the lens more quickly.\n> +\n> + - AfWindows:\n> + type: Rectangle\n> + draft: true\n> + description: |\n> + Sets the focus windows used by the AF algorithm. The units used express\n> + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> + image), as u0.16 format numbers.\n> +\n> + In order to be activated, a rectangle must be programmed with non-zero\n> + width and height. If no rectangles are programmed in this way, then the\n> + system will choose its own single default window in the centre of the\n> + image.\n> +\n> + The details of how the windows are used are platform dependent. We note\n> + that when there is more than one AF window, a typical implementation\n> + might find the optimal focus position for each one and finally select\n> + the window closest to the camera.\n> +\n> + size: [platform dependent]\n> +\n> + - AfTrigger:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> + and can also be used to terminate a scan early.\n> +\n> + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> +\n> + enum:\n> + - name: AfTriggerStart\n> + value: 0\n> + description: Start an AF scan. Ignored if a scan is in progress.\n> + - name: AfTriggerCancel\n> + value: 1\n> + description: Cancel an AF scan. This does not cause the lens to move\n> + anywhere else. Ignored if no scan is in progress.\n> +\n> + - AfPause:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control has no effect except when in continuous autofocus mode\n> + (AfModeContinuous). It can be used to pause any lens movements while\n> + (for example) images are captured. The algorithm remains inactive\n> + until it is instructed to resume.\n> +\n> + enum:\n> + - name: AfPauseImmediate\n> + value: 0\n> + description: |\n> + Pause the continuous autofocus algorithm immediately, whether or\n> + not any kind of scan is underway. If the AfState was previously\n> + reporting AfStateContinuousScanning it will now change to\n> + AfStateContinuousScanningPaused, and similarly for\n> + AfStateContinuousFocused and AfStateContinuousFailed.\n> + - name AfPauseDeferred\n> + value: 1\n\nI get an error when compiling:\nyaml.scanner.ScannerError: mapping values are not allowed here\n in \"\", line 806, column 16:\n value: 1\n\nAnd indeed, you missed a colon:\ns/name AfPauseDeferred/name: AfPauseDeferred/\n\n> + description: |\n> + Pause the continuous autofocus algorithm as soon as it is no longer\n> + scanning. If the AfState is currently reporting\n> + AfStateContinuousFocused is will change to\n> + AfStateContinuousFocusedPaused, and similarly in the case of\n> + AfStateContinuousFailed.\n> +\n> + If AfState reports AfStateContinuousScanning it will change to\n> + AfStateContinuousScanningPausing, and then move to one of\n> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> + when the scan completes.\n> + - name: AfPauseResume\n> + value: 2\n> + description: |\n> + Resume continous autofocus operation. The algorithm starts again\n\ns/continous/continuous\n\n> + from exactly where it left off, and the AfState will drop the\n> + 'Paused' or 'Pausing' part of the state.\n> +\n> + - LensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Acts as a control to instruct the lens to move to a particular position\n> + and also reports back the position of the lens for each frame.\n> +\n> + The units are determined by the lens driver.\n> +\n> + The LensPosition control is ignored unless the AfMode is set to\n> + AfModeManual.\n> +\n> + Note that this control is distinct from AfLensPosition, which allows\n> + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> + than using lens driver units.\n> +\n> + - AfState:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Reports the current state of the AF algorithm. The possible state changes\n> + are described below, although we add that the following general state\n> + changes are also allowed.\n> +\n> + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> + may be set to AfModeAuto and the algorithm will move to the\n> + AfStateAuto state. If AfTriggerStart is sent at the same time\n> + then the algorithm will move to AfStateAutoScanning state.\n> +\n> + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> + be set to AfModeContinuous and the algorithm will move to\n> + AfStateContinuousScanning.\n> +\n> + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> + be set to AfModeManual and the algorithm will move to\n> + AfStateManual. The lens will not be moved and will be left where\n> + it was at that moment.\n> +\n> + enum:\n> + - name: AfStateManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> + and AfLensPosition controls can be used directly to move the lens.\n> + - name: AfStateAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> + been moved into that state, or a scan that was in progress has been\n> + cancelled.\n> + - name: AfStateAutoScanning\n> + value: 2\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> + started using the AfTrigger control. The scan can be cancelled by\n> + sending AfTriggerCancel at which point the algorithm will either\n> + move back to AfStateAuto or, if the scan actually completes\n> + before the cancel request is processed, to one of\n> + AfStateAutoFocused or AfStateAutoFailed.\n> + - name: AfStateAutoFocused\n> + value: 3\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm believes the image is\n> + now in focus.\n> + - name: AfStateAutoFailed\n> + value: 4\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm did not find a good\n> + focus position.\n> + - name: AfStateContinuousScanning\n> + value: 5\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + is currently scanning for a good focus position. This occurs when\n> + the mode is first set to AfModeContinuous, or may happen\n> + spontaneously when the algorithm believes a re-scan is needed.\n> + - name: AfStateContinuousScanningPausing\n> + value: 6\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseDeferred was sent. The scan will complete\n> + at which point the algorithm moves to the\n> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> + state.\n> + - name: AfStateContinuousScanningPaused\n> + value: 7\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseImmediate was sent. The scan will not\n> + complete and the algorithm will remain in this state. The scan\n> + may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFocused\n> + value: 8\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed with the algorithm believing it has found a\n> + good focus position.\n> + - name: AfStateContinuousFocusedPaused\n> + value: 9\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFocused state and\n> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and the\n> + scan has completed successfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFailed\n> + value: 10\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed without finding a good focus position.\n> + - name: AfStateContinuousFailedPaused\n> + value: 11\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFailed state and\n> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and the\n> + scan has completed unsuccessfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> +\n> ...\n\nI think this is a very important step, and I would like it integrated as \nsoon as possible.\n\nWith the fixes:\nReviewed-by: Jean-Michel Hautbois \n\nThanks !\nJM","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 87922C0F1B\n\tfor ;\n\tFri, 1 Apr 2022 07:03:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EDF4165634;\n\tFri, 1 Apr 2022 09:03:35 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3235B633A5\n\tfor ;\n\tFri, 1 Apr 2022 09:03:35 +0200 (CEST)","from [IPV6:2a01:e0a:169:7140:75af:fd52:2b64:c09] (unknown\n\t[IPv6:2a01:e0a:169:7140:75af:fd52:2b64:c09])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AAECF2F7;\n\tFri, 1 Apr 2022 09:03:34 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1648796616;\n\tbh=YUKg9j3sZcspDtsFqH4qAxl4bhTIt78hX7ltAdDKW+k=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=Muga3P+l3mTy2eiL1ufX5QpQ4BNd5vww8tBVFjRxqx/qOXHI23rNCivgtei5UfSw7\n\tPNOKajD5lYavVNOjnq9NuJy16LJOxgHJmzAfufvM51jm5M+G7MISXH75dr1X9Uhr/g\n\tDcdCGA00gU3unf3n2hqt055IROrAxQl0obvntr8kUyUqMWXnNQhezmf0MDvwlm5LX5\n\txDqpzPIuhP3NJjRVqHTdKtjU53bFqpk8GgXHZZfvxB98ZiPmXZgKFLXWzhese/Daei\n\tXw9ZCB8//+g4R/zFeoGj0KwQjq2nF56pm1kXwc5bNW9olO0D630i858SCkhYRzoNTV\n\tGV/No7v6oVz0w==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1648796614;\n\tbh=YUKg9j3sZcspDtsFqH4qAxl4bhTIt78hX7ltAdDKW+k=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=de/tVMOSN/I3hDdqwH0rpIrmSerFFqCKXPV9O/2aMQ+4YcGVKQ9rIYOUr1IGIlePf\n\tgMqEqXThC41bUjAGz5aCNKM9XQO4XforJN+P4hQ9nIS9RTCxNVRGF3G98SLP6IMxMU\n\tbTuJOxDrjMYaO33f+5hfbW8/b2hwtCvHmalp13f4="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"de/tVMOS\"; dkim-atps=neutral","Message-ID":"<0165a34f-a35c-5eae-6b3c-0e878441c455@ideasonboard.com>","Date":"Fri, 1 Apr 2022 09:03:31 +0200","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.7.0","Content-Language":"en-US","To":"David Plowman ,\n\tlibcamera-devel@lists.libcamera.org, hanlinchen@chromium.org,\n\thpa@redhat.com","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>","In-Reply-To":"<20220331151747.19458-2-david.plowman@raspberrypi.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jean-Michel Hautbois via libcamera-devel\n\t","Reply-To":"Jean-Michel Hautbois ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22544,"web_url":"https://patchwork.libcamera.org/comment/22544/","msgid":"","date":"2022-04-01T09:36:56","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"content":"Hi David, thanks for the patch!\nReviewed-by: Han-Lin Chen \n\nOn Thu, Mar 31, 2022 at 11:17 PM David Plowman <\ndavid.plowman@raspberrypi.com> wrote:\n\n> This patch describes a series of controls that allow applications to\n> drive AF algorithms:\n>\n> AfMode - manual, auto or continuous\n> AfLensPosition - set lens position to macro/hyperfocal/infinity\n> AfRange - full, macro or normal\n> AfSpeed - fast or slow\n> AfWindows - AF window locations\n> AfTrigger - start (trigger an AF scan) or cancel\n> AfPause - pause continuous AF\n> LensPosition - position of lens from lens driver\n> AfState - reports the mode, whether scanning/success/failure\n> ---\n> src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> 1 file changed, 313 insertions(+), 60 deletions(-)\n>\n> diff --git a/src/libcamera/control_ids.yaml\n> b/src/libcamera/control_ids.yaml\n> index 9d4638ae..23607368 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -406,27 +406,6 @@ controls:\n> The camera will cancel any active or completed metering\n> sequence.\n> The AE algorithm is reset to its initial state.\n>\n> - - AfTrigger:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control for AF trigger. Currently identical to\n> - ANDROID_CONTROL_AF_TRIGGER.\n> -\n> - Whether the camera device will trigger autofocus for this request.\n> - enum:\n> - - name: AfTriggerIdle\n> - value: 0\n> - description: The trigger is idle.\n> - - name: AfTriggerStart\n> - value: 1\n> - description: The AF routine is started by the camera.\n> - - name: AfTriggerCancel\n> - value: 2\n> - description: |\n> - The camera will cancel any active trigger and the AF routine\n> is\n> - reset to its initial state.\n> -\n> - NoiseReductionMode:\n> type: int32_t\n> draft: true\n> @@ -507,45 +486,6 @@ controls:\n> The AE algorithm has started a pre-capture metering session.\n> \\sa AePrecaptureTrigger\n>\n> - - AfState:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control to report the current AF algorithm state. Currently\n> identical to\n> - ANDROID_CONTROL_AF_STATE.\n> -\n> - Current state of the AF algorithm.\n> - enum:\n> - - name: AfStateInactive\n> - value: 0\n> - description: The AF algorithm is inactive.\n> - - name: AfStatePassiveScan\n> - value: 1\n> - description: |\n> - AF is performing a passive scan of the scene in continuous\n> - auto-focus mode.\n> - - name: AfStatePassiveFocused\n> - value: 2\n> - description: |\n> - AF believes the scene is in focus, but might restart scanning.\n> - - name: AfStateActiveScan\n> - value: 3\n> - description: |\n> - AF is performing a scan triggered by an AF trigger request.\n> - \\sa AfTrigger\n> - - name: AfStateFocusedLock\n> - value: 4\n> - description: |\n> - AF believes has focused correctly and has locked focus.\n> - - name: AfStateNotFocusedLock\n> - value: 5\n> - description: |\n> - AF has not been able to focus and has locked.\n> - - name: AfStatePassiveUnfocused\n> - value: 6\n> - description: |\n> - AF has completed a passive scan without finding focus.\n> -\n> - AwbState:\n> type: int32_t\n> draft: true\n> @@ -690,4 +630,317 @@ controls:\n> value. All of the custom test patterns will be static (that\n> is the\n> raw image must not vary from frame to frame).\n>\n> + - AfMode:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the mode of the AF (autofocus) algorithm.\n> Applications\n> + are allowed to set a new mode, and to send additional controls for\n> + that new mode, in the same request.\n> +\n> + An implementation may choose not to implement all the modes.\n> +\n> + enum:\n> + - name: AfModeManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode. In this mode it will never\n> + perform any action nor move the lens of its own accord, but an\n> + application can set controls to move the lens.\n> +\n> + In this mode the AfState will always report AfStateManual.\n> + - name: AfModeAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode. This means that the\n> algorithm\n> + will never move the lens or change state unless the AfTrigger\n> + control is used. The AfTrigger control can be used to\n> initiate a\n> + focus scan, the results of which will also be reported by\n> AfState.\n> +\n> + If the autofocus algorithm is moved from AfModeAuto to another\n> + mode while a scan is in progress, the scan is cancelled\n> + immediately, without waiting for the scan to finish.\n> +\n> + When first entering this mode the AfState will report\n> + AfStateAuto. When a trigger control is sent, AfState will\n> + report AfStateAutoScanning for a period before spontaneously\n> + changing to AfStateAutoFocused or AfStateAutoFailed,\n> depending on\n> + the outcome of the scan. It will remain in this state until\n> + another scan is initiated by the AfTrigger control. If a scan\n> is\n> + cancelled (without changing to another mode), AfState will\n> return\n> + to AfStateAuto.\n> + - name: AfModeContinuous\n> + value: 2\n> + description: |\n> + The AF algorithm is in continuous mode. This means that the\n> lens\n> + can re-start a scan spontaneously at any moment, without any\n> user\n> + intervention. The AfState still reports whether the algorithm\n> is\n> + currently scanning or not, though the application has no\n> ability\n> + to initiate or cancel scans (though it can \"pause\" them), nor\n> to\n> + move the lens for itself.\n> +\n> + When set to AfModeContinuous, the system will immediately\n> initiate\n> + a scan so AfState will report AfStateContinuousScanning, and\n> will\n> + settle on one of AfStateContinuousFocused or\n> AfStateContinuousFailed,\n> + depending on the scan result.\n> +\n> + - AfLensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the position of the lens to one of the following\n> + predefined locations. This control only has any effect when the\n> AfMode\n> + is set to AfModeManual.\n> +\n> + This control is distinct from the LensPosition control, which\n> sets the\n> + lens position using the lens driver's units.\n> + enum:\n> + - name: AfLensPositionMacro\n> + value: 0\n> + description: The closest focal position of the lens.\n> + - name: AfLensPositionHyperfocal\n> + value: 1\n> + description: Hyperfocal position.\n> + - name: AfLensPositionInfinity\n> + value: 2\n> + description: The furthest focal position (usually infinity).\n> +\n> + - AfRange:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the range of focus distances that is scanned. An\n> + implementation may choose not to implement all the options here.\n> + enum:\n> + - name: AfRangeNormal\n> + value: 0\n> + description: |\n> + A wide range of focus distances is scanned, all the way from\n> + infinity down to close distances, though depending on the\n> + implementation, possibly not including the very closest macro\n> + positions.\n> + - name: AfRangeMacro\n> + value: 1\n> + description: Only close distances are scanned.\n> + - name: AfRangeFull\n> + value: 2\n> + description: |\n> + The full range of focus distances is scanned just as with\n> + AfRangeNormal but this time including the very closest macro\n> + positions.\n> +\n> + - AfSpeed:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control that determines whether the AF algorithm is to move the\n> lens\n> + as quickly as possible or more steadily. For example, during video\n> + recording it may be desirable not to move the lens too abruptly,\n> but\n> + when in a preview mode (waiting for a still capture) it may be\n> + helpful to move the lens as quickly as is reasonably possible.\n> + enum:\n> + - name: AfSpeedNormal\n> + value: 0\n> + description: Move the lens at its usual speed.\n> + - name: AfSpeedFast\n> + value: 1\n> + description: Move the lens more quickly.\n> +\n> + - AfWindows:\n> + type: Rectangle\n> + draft: true\n> + description: |\n> + Sets the focus windows used by the AF algorithm. The units used\n> express\n> + a proportion of the ScalerCrop control (or if unavailable, of the\n> entire\n> + image), as u0.16 format numbers.\n> +\n> + In order to be activated, a rectangle must be programmed with\n> non-zero\n> + width and height. If no rectangles are programmed in this way,\n> then the\n> + system will choose its own single default window in the centre of\n> the\n> + image.\n> +\n> + The details of how the windows are used are platform dependent.\n> We note\n> + that when there is more than one AF window, a typical\n> implementation\n> + might find the optimal focus position for each one and finally\n> select\n> + the window closest to the camera.\n> +\n> + size: [platform dependent]\n> +\n> + - AfTrigger:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control starts an autofocus scan when AfMode is set to\n> AfModeAuto,\n> + and can also be used to terminate a scan early.\n> +\n> + It is ignored if AfMode is set to AfModeManual or\n> AfModeContinuous.\n> +\n> + enum:\n> + - name: AfTriggerStart\n> + value: 0\n> + description: Start an AF scan. Ignored if a scan is in progress.\n> + - name: AfTriggerCancel\n> + value: 1\n> + description: Cancel an AF scan. This does not cause the lens to\n> move\n> + anywhere else. Ignored if no scan is in progress.\n> +\n> + - AfPause:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control has no effect except when in continuous autofocus\n> mode\n> + (AfModeContinuous). It can be used to pause any lens movements\n> while\n> + (for example) images are captured. The algorithm remains inactive\n> + until it is instructed to resume.\n> +\n> + enum:\n> + - name: AfPauseImmediate\n> + value: 0\n> + description: |\n> + Pause the continuous autofocus algorithm immediately, whether\n> or\n> + not any kind of scan is underway. If the AfState was\n> previously\n> + reporting AfStateContinuousScanning it will now change to\n> + AfStateContinuousScanningPaused, and similarly for\n> + AfStateContinuousFocused and AfStateContinuousFailed.\n> + - name AfPauseDeferred\n> + value: 1\n> + description: |\n> + Pause the continuous autofocus algorithm as soon as it is no\n> longer\n> + scanning. If the AfState is currently reporting\n> + AfStateContinuousFocused is will change to\n> + AfStateContinuousFocusedPaused, and similarly in the case of\n> + AfStateContinuousFailed.\n> +\n> + If AfState reports AfStateContinuousScanning it will change to\n> + AfStateContinuousScanningPausing, and then move to one of\n> + AfStateContinuousFocusedPaused or\n> AfStateContinuousFailedPaused\n> + when the scan completes.\n> + - name: AfPauseResume\n> + value: 2\n> + description: |\n> + Resume continous autofocus operation. The algorithm starts\n> again\n> + from exactly where it left off, and the AfState will drop the\n> + 'Paused' or 'Pausing' part of the state.\n> +\n> + - LensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Acts as a control to instruct the lens to move to a particular\n> position\n> + and also reports back the position of the lens for each frame.\n> +\n> + The units are determined by the lens driver.\n> +\n> + The LensPosition control is ignored unless the AfMode is set to\n> + AfModeManual.\n> +\n> + Note that this control is distinct from AfLensPosition, which\n> allows\n> + the lens to be moved to its macro/hyperfocal/infinity position,\n> rather\n> + than using lens driver units.\n> +\n> + - AfState:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Reports the current state of the AF algorithm. The possible state\n> changes\n> + are described below, although we add that the following general\n> state\n> + changes are also allowed.\n> +\n> + In any of the AfStateManual or AfStateContinuous states, the\n> AfMode\n> + may be set to AfModeAuto and the algorithm will move to the\n> + AfStateAuto state. If AfTriggerStart is sent at the same time\n> + then the algorithm will move to AfStateAutoScanning state.\n> +\n> + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> + be set to AfModeContinuous and the algorithm will move to\n> + AfStateContinuousScanning.\n> +\n> + In any of the AfStateAuto or AfStateContinuous states, the AfMode\n> may\n> + be set to AfModeManual and the algorithm will move to\n> + AfStateManual. The lens will not be moved and will be left where\n> + it was at that moment.\n> +\n> + enum:\n> + - name: AfStateManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode (AfModeManual). The\n> LensPosition\n> + and AfLensPosition controls can be used directly to move the\n> lens.\n> + - name: AfStateAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and has either\n> just\n> + been moved into that state, or a scan that was in progress\n> has been\n> + cancelled.\n> + - name: AfStateAutoScanning\n> + value: 2\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> been\n> + started using the AfTrigger control. The scan can be\n> cancelled by\n> + sending AfTriggerCancel at which point the algorithm will\n> either\n> + move back to AfStateAuto or, if the scan actually completes\n> + before the cancel request is processed, to one of\n> + AfStateAutoFocused or AfStateAutoFailed.\n> + - name: AfStateAutoFocused\n> + value: 3\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm believes the\n> image is\n> + now in focus.\n> + - name: AfStateAutoFailed\n> + value: 4\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm did not find a\n> good\n> + focus position.\n> + - name: AfStateContinuousScanning\n> + value: 5\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + is currently scanning for a good focus position. This occurs\n> when\n> + the mode is first set to AfModeContinuous, or may happen\n> + spontaneously when the algorithm believes a re-scan is needed.\n> + - name: AfStateContinuousScanningPausing\n> + value: 6\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseDeferred was sent. The scan will\n> complete\n> + at which point the algorithm moves to the\n> + AfStateContinuousFocusedPaused or\n> AfStateContinuousFailedPaused\n> + state.\n> + - name: AfStateContinuousScanningPaused\n> + value: 7\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseImmediate was sent. The scan will not\n> + complete and the algorithm will remain in this state. The scan\n> + may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFocused\n> + value: 8\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed with the algorithm believing it has\n> found a\n> + good focus position.\n> + - name: AfStateContinuousFocusedPaused\n> + value: 9\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFocused state and\n> + has been paused (by either AfPauseImmediate or\n> AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and\n> the\n> + scan has completed successfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFailed\n> + value: 10\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed without finding a good focus position.\n> + - name: AfStateContinuousFailedPaused\n> + value: 11\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFailed state and\n> + has been paused (by either AfPauseImmediate or\n> AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and\n> the\n> + scan has completed unsuccessfully. The algorithm will now\n> remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> +\n> ...\n> --\n> 2.30.2\n>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BDE52C3256\n\tfor ;\n\tFri, 1 Apr 2022 09:37:10 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EFE4C65636;\n\tFri, 1 Apr 2022 11:37:09 +0200 (CEST)","from mail-oa1-x35.google.com (mail-oa1-x35.google.com\n\t[IPv6:2001:4860:4864:20::35])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 39047604B9\n\tfor ;\n\tFri, 1 Apr 2022 11:37:08 +0200 (CEST)","by mail-oa1-x35.google.com with SMTP id\n\t586e51a60fabf-dacc470e03so2106125fac.5\n\tfor ;\n\tFri, 01 Apr 2022 02:37:08 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1648805830;\n\tbh=QooXPtf2wT9ROMl6bZZYC3TyJQ4m1ua7wvsviytxjEE=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=GPOfRmKQwNQu6rjw51PYisUskZSoSO2l5qNBAAEuB1uZdvatnY8/QO3UqQmAlyJHX\n\ttXxBqpcsUiDD82izmb9JdZdNs1qRsVXBCGp5ffw9G9p+9/gZhHO82tauHm1+MgJObd\n\tVlH+Os44e0GAeh3I2qbXJpbmmpurHYSOuN+ETq+bZdhLQsm5O+Ek3sYhMULgslo3Si\n\t9qXUj7YubqNZcGgN9gJix1d+uKGhgC0HddOVr33ocaQ20HNL77mmFXMUFq+djjSybe\n\tN5j85/V+g/QhXZklvtOVx5dEfsZGdy/2cTIkYoAQTRdnbM+naX00P3oiB+9JviW85d\n\tNF2w897wokMHg==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=dhwWCJ35sTgL2Q7k6Y7zsxR6V3oH8cPwZnXy7K9ElNE=;\n\tb=XUocwmwlhF/Itw8j4k2dBTysVUTJDrvM3uKZD41oSPvixsiInvHpKIEREPXnZee6yE\n\tKetyE1cITzQ48XWzZs6uhO+YpoLpbsxnuPnYHL9oTTpBHvG+gs8BzQy1oEOCfazseNMa\n\tlMl6UcA1IyuCMAK2cCYjXrPLR8pNLtnoWbvvk="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"XUocwmwl\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=dhwWCJ35sTgL2Q7k6Y7zsxR6V3oH8cPwZnXy7K9ElNE=;\n\tb=4RDarJw+5KnEW7Xqf7QJ87/HyfCSFeOgtghrVQtpeqYWfOcj2ahdzmv1q9iybrcNvo\n\t4MDuQZlZqFInqHwo7XG23pRtY2E6N8nlUiYYnu7IBGIoqLtU3EjxTE/3jkh2pmo+Xqmr\n\tjwMm9fZikWSAkE4Vxo30Mjfa8EKV80uICQvXYlOP0Vmh6rkL06/shkGh1WC0yeRWfAlw\n\tHupOMTA2zXeDGkydLb1ppupsWMQOG+W3ZckyYkIY24qSytg4pwhPwSpr66HTZrb/f4Op\n\tnlwd2PrsZPZUwwPIuCADTtXWCDPHpTJcSl8gepXDSQUO2FWrju/NPV9FX+/L5aXR/oKR\n\tKUCw==","X-Gm-Message-State":"AOAM531EeiqVdA4fPo4YDDDdW3k+zyhINZzJ2EtzRrmmxW8Zx4r0XfBQ\n\t23GShiWCax6yshJsMo8EeiZmr85f8b1E6nnB5wwwYQ==","X-Google-Smtp-Source":"ABdhPJwjWJcHJIzcKjkiu58tfpFb8r1fpP1YEwHz6EWXFN6kXBQQ/7HQggckR0Jg7GP2Rk5d2BLxjYK0LzvsYJzuKTM=","X-Received":"by 2002:a05:6870:207:b0:de:bd65:38a7 with SMTP id\n\tj7-20020a056870020700b000debd6538a7mr4650710oad.62.1648805826832;\n\tFri, 01 Apr 2022 02:37:06 -0700 (PDT)","MIME-Version":"1.0","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>","In-Reply-To":"<20220331151747.19458-2-david.plowman@raspberrypi.com>","Date":"Fri, 1 Apr 2022 17:36:56 +0800","Message-ID":"","To":"David Plowman ","Content-Type":"multipart/alternative; boundary=\"0000000000009cbe0505db948574\"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Hanlin Chen via libcamera-devel ","Reply-To":"Hanlin Chen ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22560,"web_url":"https://patchwork.libcamera.org/comment/22560/","msgid":"","date":"2022-04-04T12:05:50","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi\n\nThanks to those who have had a chance to look at this!\n\nOn Fri, 1 Apr 2022 at 10:37, Hanlin Chen wrote:\n>\n> Hi David, thanks for the patch!\n> Reviewed-by: Han-Lin Chen \n>\n> On Thu, Mar 31, 2022 at 11:17 PM David Plowman wrote:\n>>\n>> This patch describes a series of controls that allow applications to\n>> drive AF algorithms:\n>>\n>> AfMode - manual, auto or continuous\n>> AfLensPosition - set lens position to macro/hyperfocal/infinity\n>> AfRange - full, macro or normal\n>> AfSpeed - fast or slow\n>> AfWindows - AF window locations\n>> AfTrigger - start (trigger an AF scan) or cancel\n>> AfPause - pause continuous AF\n>> LensPosition - position of lens from lens driver\n>> AfState - reports the mode, whether scanning/success/failure\n>> ---\n>> src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n>> 1 file changed, 313 insertions(+), 60 deletions(-)\n>>\n>> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n>> index 9d4638ae..23607368 100644\n>> --- a/src/libcamera/control_ids.yaml\n>> +++ b/src/libcamera/control_ids.yaml\n>> @@ -406,27 +406,6 @@ controls:\n>> The camera will cancel any active or completed metering sequence.\n>> The AE algorithm is reset to its initial state.\n>>\n>> - - AfTrigger:\n>> - type: int32_t\n>> - draft: true\n>> - description: |\n>> - Control for AF trigger. Currently identical to\n>> - ANDROID_CONTROL_AF_TRIGGER.\n>> -\n>> - Whether the camera device will trigger autofocus for this request.\n>> - enum:\n>> - - name: AfTriggerIdle\n>> - value: 0\n>> - description: The trigger is idle.\n>> - - name: AfTriggerStart\n>> - value: 1\n>> - description: The AF routine is started by the camera.\n>> - - name: AfTriggerCancel\n>> - value: 2\n>> - description: |\n>> - The camera will cancel any active trigger and the AF routine is\n>> - reset to its initial state.\n>> -\n>> - NoiseReductionMode:\n>> type: int32_t\n>> draft: true\n>> @@ -507,45 +486,6 @@ controls:\n>> The AE algorithm has started a pre-capture metering session.\n>> \\sa AePrecaptureTrigger\n>>\n>> - - AfState:\n>> - type: int32_t\n>> - draft: true\n>> - description: |\n>> - Control to report the current AF algorithm state. Currently identical to\n>> - ANDROID_CONTROL_AF_STATE.\n>> -\n>> - Current state of the AF algorithm.\n>> - enum:\n>> - - name: AfStateInactive\n>> - value: 0\n>> - description: The AF algorithm is inactive.\n>> - - name: AfStatePassiveScan\n>> - value: 1\n>> - description: |\n>> - AF is performing a passive scan of the scene in continuous\n>> - auto-focus mode.\n>> - - name: AfStatePassiveFocused\n>> - value: 2\n>> - description: |\n>> - AF believes the scene is in focus, but might restart scanning.\n>> - - name: AfStateActiveScan\n>> - value: 3\n>> - description: |\n>> - AF is performing a scan triggered by an AF trigger request.\n>> - \\sa AfTrigger\n>> - - name: AfStateFocusedLock\n>> - value: 4\n>> - description: |\n>> - AF believes has focused correctly and has locked focus.\n>> - - name: AfStateNotFocusedLock\n>> - value: 5\n>> - description: |\n>> - AF has not been able to focus and has locked.\n>> - - name: AfStatePassiveUnfocused\n>> - value: 6\n>> - description: |\n>> - AF has completed a passive scan without finding focus.\n>> -\n>> - AwbState:\n>> type: int32_t\n>> draft: true\n>> @@ -690,4 +630,317 @@ controls:\n>> value. All of the custom test patterns will be static (that is the\n>> raw image must not vary from frame to frame).\n>>\n>> + - AfMode:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + Control to set the mode of the AF (autofocus) algorithm. Applications\n>> + are allowed to set a new mode, and to send additional controls for\n>> + that new mode, in the same request.\n>> +\n>> + An implementation may choose not to implement all the modes.\n>> +\n>> + enum:\n>> + - name: AfModeManual\n>> + value: 0\n>> + description: |\n>> + The AF algorithm is in manual mode. In this mode it will never\n>> + perform any action nor move the lens of its own accord, but an\n>> + application can set controls to move the lens.\n>> +\n>> + In this mode the AfState will always report AfStateManual.\n>> + - name: AfModeAuto\n>> + value: 1\n>> + description: |\n>> + The AF algorithm is in auto mode. This means that the algorithm\n>> + will never move the lens or change state unless the AfTrigger\n>> + control is used. The AfTrigger control can be used to initiate a\n>> + focus scan, the results of which will also be reported by AfState.\n>> +\n>> + If the autofocus algorithm is moved from AfModeAuto to another\n>> + mode while a scan is in progress, the scan is cancelled\n>> + immediately, without waiting for the scan to finish.\n>> +\n>> + When first entering this mode the AfState will report\n>> + AfStateAuto. When a trigger control is sent, AfState will\n>> + report AfStateAutoScanning for a period before spontaneously\n>> + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n>> + the outcome of the scan. It will remain in this state until\n>> + another scan is initiated by the AfTrigger control. If a scan is\n>> + cancelled (without changing to another mode), AfState will return\n>> + to AfStateAuto.\n>> + - name: AfModeContinuous\n>> + value: 2\n>> + description: |\n>> + The AF algorithm is in continuous mode. This means that the lens\n>> + can re-start a scan spontaneously at any moment, without any user\n>> + intervention. The AfState still reports whether the algorithm is\n>> + currently scanning or not, though the application has no ability\n>> + to initiate or cancel scans (though it can \"pause\" them), nor to\n>> + move the lens for itself.\n>> +\n>> + When set to AfModeContinuous, the system will immediately initiate\n>> + a scan so AfState will report AfStateContinuousScanning, and will\n>> + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n>> + depending on the scan result.\n>> +\n>> + - AfLensPosition:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + Control to set the position of the lens to one of the following\n>> + predefined locations. This control only has any effect when the AfMode\n>> + is set to AfModeManual.\n>> +\n>> + This control is distinct from the LensPosition control, which sets the\n>> + lens position using the lens driver's units.\n>> + enum:\n>> + - name: AfLensPositionMacro\n>> + value: 0\n>> + description: The closest focal position of the lens.\n>> + - name: AfLensPositionHyperfocal\n>> + value: 1\n>> + description: Hyperfocal position.\n>> + - name: AfLensPositionInfinity\n>> + value: 2\n>> + description: The furthest focal position (usually infinity).\n>> +\n>> + - AfRange:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + Control to set the range of focus distances that is scanned. An\n>> + implementation may choose not to implement all the options here.\n>> + enum:\n>> + - name: AfRangeNormal\n>> + value: 0\n>> + description: |\n>> + A wide range of focus distances is scanned, all the way from\n>> + infinity down to close distances, though depending on the\n>> + implementation, possibly not including the very closest macro\n>> + positions.\n>> + - name: AfRangeMacro\n>> + value: 1\n>> + description: Only close distances are scanned.\n>> + - name: AfRangeFull\n>> + value: 2\n>> + description: |\n>> + The full range of focus distances is scanned just as with\n>> + AfRangeNormal but this time including the very closest macro\n>> + positions.\n>> +\n>> + - AfSpeed:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + Control that determines whether the AF algorithm is to move the lens\n>> + as quickly as possible or more steadily. For example, during video\n>> + recording it may be desirable not to move the lens too abruptly, but\n>> + when in a preview mode (waiting for a still capture) it may be\n>> + helpful to move the lens as quickly as is reasonably possible.\n>> + enum:\n>> + - name: AfSpeedNormal\n>> + value: 0\n>> + description: Move the lens at its usual speed.\n>> + - name: AfSpeedFast\n>> + value: 1\n>> + description: Move the lens more quickly.\n>> +\n>> + - AfWindows:\n>> + type: Rectangle\n>> + draft: true\n>> + description: |\n>> + Sets the focus windows used by the AF algorithm. The units used express\n>> + a proportion of the ScalerCrop control (or if unavailable, of the entire\n>> + image), as u0.16 format numbers.\n>> +\n>> + In order to be activated, a rectangle must be programmed with non-zero\n>> + width and height. If no rectangles are programmed in this way, then the\n>> + system will choose its own single default window in the centre of the\n>> + image.\n>> +\n>> + The details of how the windows are used are platform dependent. We note\n>> + that when there is more than one AF window, a typical implementation\n>> + might find the optimal focus position for each one and finally select\n>> + the window closest to the camera.\n>> +\n>> + size: [platform dependent]\n>> +\n>> + - AfTrigger:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n>> + and can also be used to terminate a scan early.\n>> +\n>> + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n>> +\n>> + enum:\n>> + - name: AfTriggerStart\n>> + value: 0\n>> + description: Start an AF scan. Ignored if a scan is in progress.\n>> + - name: AfTriggerCancel\n>> + value: 1\n>> + description: Cancel an AF scan. This does not cause the lens to move\n>> + anywhere else. Ignored if no scan is in progress.\n>> +\n>> + - AfPause:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + This control has no effect except when in continuous autofocus mode\n>> + (AfModeContinuous). It can be used to pause any lens movements while\n>> + (for example) images are captured. The algorithm remains inactive\n>> + until it is instructed to resume.\n>> +\n>> + enum:\n>> + - name: AfPauseImmediate\n>> + value: 0\n>> + description: |\n>> + Pause the continuous autofocus algorithm immediately, whether or\n>> + not any kind of scan is underway. If the AfState was previously\n>> + reporting AfStateContinuousScanning it will now change to\n>> + AfStateContinuousScanningPaused, and similarly for\n>> + AfStateContinuousFocused and AfStateContinuousFailed.\n>> + - name AfPauseDeferred\n\nThanks for pointing out the typo here, Jean-Michel. I will fix it in\nthe next revision (I'll just wait a bit and see if any other reviews\ncome in..).\n\n>> + value: 1\n>> + description: |\n>> + Pause the continuous autofocus algorithm as soon as it is no longer\n>> + scanning. If the AfState is currently reporting\n>> + AfStateContinuousFocused is will change to\n>> + AfStateContinuousFocusedPaused, and similarly in the case of\n>> + AfStateContinuousFailed.\n>> +\n>> + If AfState reports AfStateContinuousScanning it will change to\n>> + AfStateContinuousScanningPausing, and then move to one of\n>> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n>> + when the scan completes.\n>> + - name: AfPauseResume\n>> + value: 2\n>> + description: |\n>> + Resume continous autofocus operation. The algorithm starts again\n>> + from exactly where it left off, and the AfState will drop the\n>> + 'Paused' or 'Pausing' part of the state.\n>> +\n>> + - LensPosition:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + Acts as a control to instruct the lens to move to a particular position\n>> + and also reports back the position of the lens for each frame.\n>> +\n>> + The units are determined by the lens driver.\n>> +\n>> + The LensPosition control is ignored unless the AfMode is set to\n>> + AfModeManual.\n>> +\n>> + Note that this control is distinct from AfLensPosition, which allows\n>> + the lens to be moved to its macro/hyperfocal/infinity position, rather\n>> + than using lens driver units.\n>> +\n>> + - AfState:\n>> + type: int32_t\n>> + draft: true\n>> + description: |\n>> + Reports the current state of the AF algorithm. The possible state changes\n>> + are described below, although we add that the following general state\n>> + changes are also allowed.\n>> +\n>> + In any of the AfStateManual or AfStateContinuous states, the AfMode\n>> + may be set to AfModeAuto and the algorithm will move to the\n>> + AfStateAuto state. If AfTriggerStart is sent at the same time\n>> + then the algorithm will move to AfStateAutoScanning state.\n>> +\n>> + In any of the AfStateAuto or AfStateManual states, the AfMode may\n>> + be set to AfModeContinuous and the algorithm will move to\n>> + AfStateContinuousScanning.\n>> +\n>> + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n>> + be set to AfModeManual and the algorithm will move to\n>> + AfStateManual. The lens will not be moved and will be left where\n>> + it was at that moment.\n>> +\n>> + enum:\n>> + - name: AfStateManual\n>> + value: 0\n>> + description: |\n>> + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n>> + and AfLensPosition controls can be used directly to move the lens.\n>> + - name: AfStateAuto\n>> + value: 1\n>> + description: |\n>> + The AF algorithm is in auto mode (AfModeAuto), and has either just\n>> + been moved into that state, or a scan that was in progress has been\n>> + cancelled.\n>> + - name: AfStateAutoScanning\n>> + value: 2\n>> + description: |\n>> + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n>> + started using the AfTrigger control. The scan can be cancelled by\n>> + sending AfTriggerCancel at which point the algorithm will either\n>> + move back to AfStateAuto or, if the scan actually completes\n>> + before the cancel request is processed, to one of\n>> + AfStateAutoFocused or AfStateAutoFailed.\n>> + - name: AfStateAutoFocused\n>> + value: 3\n>> + description: |\n>> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n>> + completed with the result that the algorithm believes the image is\n>> + now in focus.\n>> + - name: AfStateAutoFailed\n>> + value: 4\n>> + description: |\n>> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n>> + completed with the result that the algorithm did not find a good\n>> + focus position.\n>> + - name: AfStateContinuousScanning\n>> + value: 5\n>> + description: |\n>> + The AF algorithm is in continuous mode (AfModeContinuous) and\n>> + is currently scanning for a good focus position. This occurs when\n>> + the mode is first set to AfModeContinuous, or may happen\n>> + spontaneously when the algorithm believes a re-scan is needed.\n>> + - name: AfStateContinuousScanningPausing\n>> + value: 6\n>> + description: |\n>> + The AF algorithm is in continuous mode (AfModeContinuous) and\n>> + was scanning when AfPauseDeferred was sent. The scan will complete\n>> + at which point the algorithm moves to the\n>> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n>> + state.\n>> + - name: AfStateContinuousScanningPaused\n>> + value: 7\n>> + description: |\n>> + The AF algorithm is in continuous mode (AfModeContinuous) and\n>> + was scanning when AfPauseImmediate was sent. The scan will not\n>> + complete and the algorithm will remain in this state. The scan\n>> + may be resumed by sending AfPauseResume.\n>> + - name: AfStateContinuousFocused\n>> + value: 8\n>> + description: |\n>> + The AF algorithm is in continuous mode (AfModeContinuous) and\n>> + a scan has completed with the algorithm believing it has found a\n>> + good focus position.\n>> + - name: AfStateContinuousFocusedPaused\n>> + value: 9\n>> + description: |\n>> + The AF algorithm was in the AfStateContinuousFocused state and\n>> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n>> + or it was in the AfStateContinuousScanningPausing state and the\n>> + scan has completed successfully. The algorithm will now remain\n>> + in this state, and may be resumed by sending AfPauseResume.\n>> + - name: AfStateContinuousFailed\n>> + value: 10\n>> + description: |\n>> + The AF algorithm is in continuous mode (AfModeContinuous) and\n>> + a scan has completed without finding a good focus position.\n>> + - name: AfStateContinuousFailedPaused\n>> + value: 11\n>> + description: |\n>> + The AF algorithm was in the AfStateContinuousFailed state and\n>> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n>> + or it was in the AfStateContinuousScanningPausing state and the\n>> + scan has completed unsuccessfully. The algorithm will now remain\n>> + in this state, and may be resumed by sending AfPauseResume.\n>> +\n>> ...\n>> --\n>> 2.30.2\n>>\n\nI was also wondering if there were any comments on the state machine\ndiagram or if folks are happy with it. I'm not wild about the\n\"paused/pausing\" thing, but I guess if we want that feature then we\nhave to distinguish these states. I haven't allowed for transitions\nbetween \"AfStateContinuousScanningPausing\" and\n\"AfStateContinuousScanningPaused\" which you could imagine might exist,\nare people OK with that?\n\nThanks again\nDavid","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AF339C3256\n\tfor ;\n\tMon, 4 Apr 2022 12:06:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E847465640;\n\tMon, 4 Apr 2022 14:06:02 +0200 (CEST)","from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com\n\t[IPv6:2a00:1450:4864:20::42e])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C0AD8633A5\n\tfor ;\n\tMon, 4 Apr 2022 14:06:01 +0200 (CEST)","by mail-wr1-x42e.google.com with SMTP id k23so1600716wrd.8\n\tfor ;\n\tMon, 04 Apr 2022 05:06:01 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649073962;\n\tbh=TIATvRrRWorm+6hd6/b++amZSB0iHQF1Fxz0NcWMURo=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=NnUAtHA5jj34m/RsL7SzMgPj6KQkLP7d2IIwqCcmvLVSJgKt0BBspkBlnNswfJBUv\n\t0BGtpUgGQNtXUf7ZMLdsTSTsXinRg2WcNTFQuzKMiuVxl/WOpkqYBr/oGZ4iYNv9KQ\n\t6bwp0ntc9cNpjwGQ/KIns/XHFGEYPiW0jFM1YuItKNMFN8M0ukkLcYKkdpOyIhomD3\n\tppyOZnzR/2X21cGsHiGmnkfV0jcgJ9qlRjc1qI3/b1lG7WpMRqptmypFwe9wVkkgNk\n\tPQd0aeRMF+ubvn39D3fGX6SHGXeRNvnqs75FvCzyGkIwQlhpJ5AhE/0rZ6ju5muHml\n\tGqynrCxZnFKuw==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=EBcWE4lxmkGQLVF+bbfKEjX5g24k+I4v1aMX848fLKM=;\n\tb=bIfNnKsLRQvLnKJTTFIOqHYZj+DkH71KmztBXhWnEX0DUX6uphD3hWsToSFtKcs+J0\n\tj8bBmTy/4mEWbUCRbnO2ODd69L9yOQ51fvwMmpt1wLTiufoE09n01eCvtavVYMyi6Lmz\n\tDWsuJngwDU73PUMHQrvD2p5Z2CruP3FAzjKxjBIli/DCwK4gfQIGBIT1rKOqcxJ+liRz\n\tr0bxh1BiLr3BlWUQr+kdIGvaQDMzyhtDg8x+qi7D39cyAieceYKcufhe/I4nh9r/yslO\n\tyZWUFMH39coGCOFogVCPRuUBNfTKjT16tPybFVgCaOclrh01K9qjv61nfUueAaZNiAAa\n\t6F+w=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"bIfNnKsL\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=EBcWE4lxmkGQLVF+bbfKEjX5g24k+I4v1aMX848fLKM=;\n\tb=IOVFeLbai2loF/o85QZr4kfzf2A73KDE8oPYX9drOiffPbyw0S2VSXNVujxFIefIMi\n\tnaKfpxZCRZCXoDch0h6WmwCaaBcr+Yo8jB9liCxCgd1ZyoUPqoYOGUNpf7H/ez8ShQR0\n\t46ijkFWD2p8feSvYrUAl63xLRi20ZIkgyrVWZ7ntxHwOBaSm368FZdHFbVbQRTaJext/\n\tkn1cqhRwfFaqqIzHZZtvBFg9YVdoZ4lFFZ6keSfLQwX0w87YtctVSewqKJi81i3O6hxD\n\troONa5jEXjQ8n3Yany/jJQWtdWZfTORcrd+hTQFmyfzYlOQ/Ql28O9KwB0y2xAg5WY/y\n\tGw4Q==","X-Gm-Message-State":"AOAM532ehRlSRV5jbvPB6WBCuZ/RZHdmaI+e6Ahc8RKpcwwxWYhxqgfJ\n\tsf8pF3yczgZZeko1XvPWLgQ7hJtbfSFKgV+lgH1WZg==","X-Google-Smtp-Source":"ABdhPJzmFFmbP0d4Dg6sV/KtLXr/Y32HGW8enZz/Uib6ZDR8/de7qDlbOUYbsRNlM6XxIaLOV1Eu/ijKuWKjUgSUFaU=","X-Received":"by 2002:a5d:6505:0:b0:205:9a98:e184 with SMTP id\n\tx5-20020a5d6505000000b002059a98e184mr16056570wru.317.1649073960977;\n\tMon, 04 Apr 2022 05:06:00 -0700 (PDT)","MIME-Version":"1.0","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t","In-Reply-To":"","Date":"Mon, 4 Apr 2022 13:05:50 +0100","Message-ID":"","To":"Hanlin Chen ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"David Plowman via libcamera-devel ","Reply-To":"David Plowman ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22583,"web_url":"https://patchwork.libcamera.org/comment/22583/","msgid":"<20220405203610.peioc4aomqgtxb5w@uno.localdomain>","date":"2022-04-05T20:36:10","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi David,\n sorry for being late :)\n\nOn Thu, Mar 31, 2022 at 04:17:47PM +0100, David Plowman via libcamera-devel wrote:\n> This patch describes a series of controls that allow applications to\n> drive AF algorithms:\n>\n> AfMode - manual, auto or continuous\n> AfLensPosition - set lens position to macro/hyperfocal/infinity\n> AfRange - full, macro or normal\n> AfSpeed - fast or slow\n> AfWindows - AF window locations\n> AfTrigger - start (trigger an AF scan) or cancel\n> AfPause - pause continuous AF\n> LensPosition - position of lens from lens driver\n> AfState - reports the mode, whether scanning/success/failure\n> ---\n> src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> 1 file changed, 313 insertions(+), 60 deletions(-)\n>\n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index 9d4638ae..23607368 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -406,27 +406,6 @@ controls:\n> The camera will cancel any active or completed metering sequence.\n> The AE algorithm is reset to its initial state.\n>\n> - - AfTrigger:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control for AF trigger. Currently identical to\n> - ANDROID_CONTROL_AF_TRIGGER.\n> -\n> - Whether the camera device will trigger autofocus for this request.\n> - enum:\n> - - name: AfTriggerIdle\n> - value: 0\n> - description: The trigger is idle.\n> - - name: AfTriggerStart\n> - value: 1\n> - description: The AF routine is started by the camera.\n> - - name: AfTriggerCancel\n> - value: 2\n> - description: |\n> - The camera will cancel any active trigger and the AF routine is\n> - reset to its initial state.\n> -\n> - NoiseReductionMode:\n> type: int32_t\n> draft: true\n> @@ -507,45 +486,6 @@ controls:\n> The AE algorithm has started a pre-capture metering session.\n> \\sa AePrecaptureTrigger\n>\n> - - AfState:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control to report the current AF algorithm state. Currently identical to\n> - ANDROID_CONTROL_AF_STATE.\n> -\n> - Current state of the AF algorithm.\n> - enum:\n> - - name: AfStateInactive\n> - value: 0\n> - description: The AF algorithm is inactive.\n> - - name: AfStatePassiveScan\n> - value: 1\n> - description: |\n> - AF is performing a passive scan of the scene in continuous\n> - auto-focus mode.\n> - - name: AfStatePassiveFocused\n> - value: 2\n> - description: |\n> - AF believes the scene is in focus, but might restart scanning.\n> - - name: AfStateActiveScan\n> - value: 3\n> - description: |\n> - AF is performing a scan triggered by an AF trigger request.\n> - \\sa AfTrigger\n> - - name: AfStateFocusedLock\n> - value: 4\n> - description: |\n> - AF believes has focused correctly and has locked focus.\n> - - name: AfStateNotFocusedLock\n> - value: 5\n> - description: |\n> - AF has not been able to focus and has locked.\n> - - name: AfStatePassiveUnfocused\n> - value: 6\n> - description: |\n> - AF has completed a passive scan without finding focus.\n> -\n> - AwbState:\n> type: int32_t\n> draft: true\n> @@ -690,4 +630,317 @@ controls:\n> value. All of the custom test patterns will be static (that is the\n> raw image must not vary from frame to frame).\n>\n> + - AfMode:\n> + type: int32_t\n> + draft: true\n\nWas having all these controls as draft intentional ? The previous\ndefinitions where set to drafts as they mapped to the android\ndefinition, but now that we're defining our own ones...\n\n> + description: |\n> + Control to set the mode of the AF (autofocus) algorithm. Applications\n> + are allowed to set a new mode, and to send additional controls for\n> + that new mode, in the same request.\n> +\n> + An implementation may choose not to implement all the modes.\n> +\n> + enum:\n> + - name: AfModeManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode. In this mode it will never\n> + perform any action nor move the lens of its own accord, but an\n> + application can set controls to move the lens.\n> +\n> + In this mode the AfState will always report AfStateManual.\n> + - name: AfModeAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode. This means that the algorithm\n> + will never move the lens or change state unless the AfTrigger\n> + control is used. The AfTrigger control can be used to initiate a\n> + focus scan, the results of which will also be reported by AfState.\n> +\n> + If the autofocus algorithm is moved from AfModeAuto to another\n> + mode while a scan is in progress, the scan is cancelled\n> + immediately, without waiting for the scan to finish.\n> +\n> + When first entering this mode the AfState will report\n> + AfStateAuto. When a trigger control is sent, AfState will\n> + report AfStateAutoScanning for a period before spontaneously\n> + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> + the outcome of the scan. It will remain in this state until\n> + another scan is initiated by the AfTrigger control. If a scan is\n> + cancelled (without changing to another mode), AfState will return\n> + to AfStateAuto.\n> + - name: AfModeContinuous\n> + value: 2\n> + description: |\n> + The AF algorithm is in continuous mode. This means that the lens\n> + can re-start a scan spontaneously at any moment, without any user\n> + intervention. The AfState still reports whether the algorithm is\n> + currently scanning or not, though the application has no ability\n> + to initiate or cancel scans (though it can \"pause\" them), nor to\n> + move the lens for itself.\n> +\n> + When set to AfModeContinuous, the system will immediately initiate\n> + a scan so AfState will report AfStateContinuousScanning, and will\n> + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> + depending on the scan result.\n> +\n> + - AfLensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the position of the lens to one of the following\n> + predefined locations. This control only has any effect when the AfMode\n> + is set to AfModeManual.\n> +\n> + This control is distinct from the LensPosition control, which sets the\n> + lens position using the lens driver's units.\n> + enum:\n> + - name: AfLensPositionMacro\n> + value: 0\n> + description: The closest focal position of the lens.\n> + - name: AfLensPositionHyperfocal\n> + value: 1\n> + description: Hyperfocal position.\n> + - name: AfLensPositionInfinity\n> + value: 2\n> + description: The furthest focal position (usually infinity).\n\nInteresting, bikeshedding on the name apart, is this some 'simplified'\nLensState ? Do you think there is a use case for this ? Why would\nanyone set the AF to auto mode then use some coarse grained control\nlike this one ? Wouldn't it make more sense as an 'hint' to the AF\nalgorithm to restrict the search space ?\n\n(Btw I have a use case for supplying a range of lens positions, which\nhas been previously estimated according to some pre-defined use cases,\nto the AF algorithm in order restrict the AF search range to increase\nthe speed of a scan. Do you think it's a valid use case and it's worth a\ncontrol ?)\n\n> +\n> + - AfRange:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the range of focus distances that is scanned. An\n> + implementation may choose not to implement all the options here.\n\nOnly valid in Auto and Continuous mode, right ?\n\n(This makes me think that we could easily model Continuous by adding a\nTriggerContinuous when in Auto mode. With a TriggerPause we could get rid of\nAfPause... well, it's quite a redesign so I'll stop here unless\nsomeone thinks it's a really good idea :)\n\n> + enum:\n> + - name: AfRangeNormal\n> + value: 0\n> + description: |\n> + A wide range of focus distances is scanned, all the way from\n> + infinity down to close distances, though depending on the\n> + implementation, possibly not including the very closest macro\n> + positions.\n> + - name: AfRangeMacro\n> + value: 1\n> + description: Only close distances are scanned.\n> + - name: AfRangeFull\n> + value: 2\n> + description: |\n> + The full range of focus distances is scanned just as with\n> + AfRangeNormal but this time including the very closest macro\n> + positions.\n> +\n> + - AfSpeed:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control that determines whether the AF algorithm is to move the lens\n> + as quickly as possible or more steadily. For example, during video\n> + recording it may be desirable not to move the lens too abruptly, but\n> + when in a preview mode (waiting for a still capture) it may be\n> + helpful to move the lens as quickly as is reasonably possible.\n> + enum:\n> + - name: AfSpeedNormal\n> + value: 0\n> + description: Move the lens at its usual speed.\n> + - name: AfSpeedFast\n> + value: 1\n> + description: Move the lens more quickly.\n> +\n> + - AfWindows:\n> + type: Rectangle\n> + draft: true\n> + description: |\n> + Sets the focus windows used by the AF algorithm. The units used express\n> + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> + image), as u0.16 format numbers.\n> +\n> + In order to be activated, a rectangle must be programmed with non-zero\n> + width and height. If no rectangles are programmed in this way, then the\n> + system will choose its own single default window in the centre of the\n> + image.\n> +\n> + The details of how the windows are used are platform dependent. We note\n> + that when there is more than one AF window, a typical implementation\n> + might find the optimal focus position for each one and finally select\n> + the window closest to the camera.\n> +\n> + size: [platform dependent]\n\nI think this should just be\n\n size: [n]\n> +\n> + - AfTrigger:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> + and can also be used to terminate a scan early.\n> +\n> + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> +\n> + enum:\n> + - name: AfTriggerStart\n> + value: 0\n> + description: Start an AF scan. Ignored if a scan is in progress.\n> + - name: AfTriggerCancel\n> + value: 1\n> + description: Cancel an AF scan. This does not cause the lens to move\n> + anywhere else. Ignored if no scan is in progress.\n> +\n> + - AfPause:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control has no effect except when in continuous autofocus mode\n> + (AfModeContinuous). It can be used to pause any lens movements while\n> + (for example) images are captured. The algorithm remains inactive\n> + until it is instructed to resume.\n> +\n> + enum:\n> + - name: AfPauseImmediate\n> + value: 0\n> + description: |\n> + Pause the continuous autofocus algorithm immediately, whether or\n> + not any kind of scan is underway. If the AfState was previously\n> + reporting AfStateContinuousScanning it will now change to\n> + AfStateContinuousScanningPaused, and similarly for\n> + AfStateContinuousFocused and AfStateContinuousFailed.\n> + - name AfPauseDeferred\n> + value: 1\n> + description: |\n> + Pause the continuous autofocus algorithm as soon as it is no longer\n> + scanning. If the AfState is currently reporting\n> + AfStateContinuousFocused is will change to\n> + AfStateContinuousFocusedPaused, and similarly in the case of\n> + AfStateContinuousFailed.\n> +\n> + If AfState reports AfStateContinuousScanning it will change to\n> + AfStateContinuousScanningPausing, and then move to one of\n> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> + when the scan completes.\n> + - name: AfPauseResume\n> + value: 2\n> + description: |\n> + Resume continous autofocus operation. The algorithm starts again\n> + from exactly where it left off, and the AfState will drop the\n> + 'Paused' or 'Pausing' part of the state.\n> +\n> + - LensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Acts as a control to instruct the lens to move to a particular position\n> + and also reports back the position of the lens for each frame.\n> +\n> + The units are determined by the lens driver.\n\nUgh, we cannot do this in the control definition. Driver specificities\ncannot surface to the controls definition, which should be platform\nindependent.\n\nI think we'll have to establish a numerical range (go ahead if you\nhave any idea) and translate to the actual driver values in the\nCameraLens class ?\n\n> +\n> + The LensPosition control is ignored unless the AfMode is set to\n> + AfModeManual.\n> +\n> + Note that this control is distinct from AfLensPosition, which allows\n> + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> + than using lens driver units.\n> +\n> + - AfState:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Reports the current state of the AF algorithm. The possible state changes\n> + are described below, although we add that the following general state\n> + changes are also allowed.\n\nwow, there is a lot of states here :)\n\nLet me copy them here\n\n - name: AfStateManual\n - name: AfStateAuto\n - name: AfStateAutoScanning\n - name: AfStateAutoFocused\n - name: AfStateAutoFailed\n - name: AfStateContinuousScanning\n - name: AfStateContinuousScanningPausing\n - name: AfStateContinuousScanningPaused\n - name: AfStateContinuousFocused\n - name: AfStateContinuousFocusedPaused\n - name: AfStateContinuousFailed\n - name: AfStateContinuousFailedPaused\n\n(please note that AfStateAuto seems mis-aligned, unless I messed up\nsomething while cutting&paste. Please check the compiled doc to see\nhow it looks like...)\n\nSo we have three macro groups\n\n - Manual\n\n - Auto\n - Scanning\n - Focus\n - Failed\n\n - Continuous\n\n - Scanning\n - Pausing\n - Paused\n - Focused\n - Paused\n - Failed\n - Paused\n\nThe first group is AfMode (Manual, Auto, Continuous) and can be\ndeduced by inspecting the AfMode metadata.\n\nThe second group is the actual state (Scannng, Focused, Failed)\n\nThe third group is specific to continuous mode as it accepts a Pause\ntrigger.\n\nI understand why you have this, as expressing \"The AfState is set to\n$something when in AfModeManual\" etc etc might get cumbersome, but I\nthink it might be worth a try\n\n1) Manual mode (the bad)\n As we all know there is a delay between when a lens is instructed\n to move and when it reaches its final state.\n\n In Manual mode the lens is moved with LensPosition and\n AfLensPosition.\n\n To know the current state:\n - With LensPosition metadata can be inspected until it doesn't reach the\n desired value.\n - With AfLensPosition it is not clear to me how to know when the lens\n as reched one of the pre-defined positions.\n\n Is it so wrong to report either (Scanning,Focus,Failed) even for\n Manual mode ?\n\n2) AutoMode (the good)\n This seems the easy one, as (Scanning,Focus,Failed) are already\n there\n\n3) Continous (the ugly)\n Continuous mode can be paused. That's a shame :)\n Would that be that bad keeping a Paused or Pausing state which can\n only be returned for Continuous mode.\n\nAll in all, can we reduce the number of states to\n\n - Scanning\n\n The AF algorithm is scanning and has not yet reached a\n stable result. When running in Auto mode an AF scan is\n started by using the AfTrigger control. When in Manual mode\n and a new lens position is requested either by sending a\n LensPosition or AfLensPosition control, the AF state will\n report \"Scanning\" until the lens doesn't reach the desired\n position. Similarly, when running in continuous mode and the\n AF algorithm decides to start a scan spontaneously the\n AfState will report Scanning until a new stable state is\n reached.\n\n - Focus\n\n The AF algorithm has reached a stable state and the image is\n now focused. When running in Manual mode the lens has\n reached the position requested using either LensPosition and\n AfLensPosition.\n\n - Failed\n\n The AF algorithm has completed a scan and without finding a\n good focus position. When running in manual mode, the camera\n has failed moving the lens to the desired position.\n\n - Paused (copying your below text, please see comments below)\n\n The AF algorithm is in continuous mode (AfModeContinuous) and\n was scanning when AfPauseImmediate was sent. The scan will not\n complete and the algorithm will remain in this state.\n\n - Pausing\n\n The AF algorithm is in continuous mode (AfModeContinuous) and\n was scanning when AfPauseDeferred was sent. The scan will complete\n at which point the algorithm moves to the\n AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n state.\n\n\n> +\n> + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> + may be set to AfModeAuto and the algorithm will move to the\n> + AfStateAuto state. If AfTriggerStart is sent at the same time\n> + then the algorithm will move to AfStateAutoScanning state.\n> +\n> + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> + be set to AfModeContinuous and the algorithm will move to\n> + AfStateContinuousScanning.\n> +\n> + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> + be set to AfModeManual and the algorithm will move to\n> + AfStateManual. The lens will not be moved and will be left where\n> + it was at that moment.\n> +\n> + enum:\n> + - name: AfStateManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> + and AfLensPosition controls can be used directly to move the lens.\n> + - name: AfStateAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> + been moved into that state, or a scan that was in progress has been\n> + cancelled.\n> + - name: AfStateAutoScanning\n> + value: 2\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> + started using the AfTrigger control. The scan can be cancelled by\n> + sending AfTriggerCancel at which point the algorithm will either\n> + move back to AfStateAuto or, if the scan actually completes\n> + before the cancel request is processed, to one of\n> + AfStateAutoFocused or AfStateAutoFailed.\n> + - name: AfStateAutoFocused\n> + value: 3\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm believes the image is\n> + now in focus.\n> + - name: AfStateAutoFailed\n> + value: 4\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm did not find a good\n> + focus position.\n> + - name: AfStateContinuousScanning\n> + value: 5\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + is currently scanning for a good focus position. This occurs when\n> + the mode is first set to AfModeContinuous, or may happen\n> + spontaneously when the algorithm believes a re-scan is needed.\n> + - name: AfStateContinuousScanningPausing\n> + value: 6\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseDeferred was sent. The scan will complete\n> + at which point the algorithm moves to the\n> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> + state.\n> + - name: AfStateContinuousScanningPaused\n> + value: 7\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseImmediate was sent. The scan will not\n\nWhy can't the algorithm be paused when it has reached a stable state\nbut only when scanning ? Won't a pause 'freeze' the algorithm in its\ncurrent state no matter what it is ? I understand with\n\"AfPauseDeferred\" that it has to wait for AF to be locked before\nstopping it, but with \"Immediate\" doesn't \"pause\" apply to all states ?\n\nThanks\n j\n\n> + complete and the algorithm will remain in this state. The scan\n> + may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFocused\n> + value: 8\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed with the algorithm believing it has found a\n> + good focus position.\n> + - name: AfStateContinuousFocusedPaused\n> + value: 9\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFocused state and\n> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and the\n> + scan has completed successfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFailed\n> + value: 10\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed without finding a good focus position.\n> + - name: AfStateContinuousFailedPaused\n> + value: 11\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFailed state and\n> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and the\n> + scan has completed unsuccessfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> +\n> ...\n> --\n> 2.30.2\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CED92C3256\n\tfor ;\n\tTue, 5 Apr 2022 20:36:16 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0685165644;\n\tTue, 5 Apr 2022 22:36:16 +0200 (CEST)","from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net\n\t[217.70.183.195])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 11A4F604BB\n\tfor ;\n\tTue, 5 Apr 2022 22:36:15 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id A2C3E60005;\n\tTue, 5 Apr 2022 20:36:12 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649190976;\n\tbh=0qvyq0qTz9mYId19u5HUP2Ym+yrP2mT4PSjBxb24RZc=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=LjyOg4kbA5X3v3WJ/GXs3iBpVOEcHBBJX1wQhzTioFSFqfvz001ZnhJEOQGxD0t0s\n\tP/HpwCQB2QPz6o8wqZMMaYuMiv/KfyBY+SgJnf2J30bWFbWtXNuqcXkc3UApFJg10S\n\tSRx2lmyZgzfKBPWqBSMel9kaOwCoPuFWytzko889XRWeJgQKd3NrWEzR5Pwpg/oYo+\n\tG5piLCZjbRYGjFJZJxw2dlZfUBY2QBiw/KhBpOydOw59BxElMfy5AhUhnb+via83vk\n\tAByYP3aM5nz75Ktiy+QweSNruM8CyqkNgvyXvc9xx7S+Gwg+ncdtX2xUlc6V7sE17F\n\tKVOPFaRy7Mrag==","Date":"Tue, 5 Apr 2022 22:36:10 +0200","To":"David Plowman ","Message-ID":"<20220405203610.peioc4aomqgtxb5w@uno.localdomain>","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220331151747.19458-2-david.plowman@raspberrypi.com>","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22626,"web_url":"https://patchwork.libcamera.org/comment/22626/","msgid":"","date":"2022-04-06T13:05:20","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi Jacopo\n\nThanks for taking the time to read all this!\n\nOn Tue, 5 Apr 2022 at 21:36, Jacopo Mondi wrote:\n>\n> Hi David,\n> sorry for being late :)\n>\n> On Thu, Mar 31, 2022 at 04:17:47PM +0100, David Plowman via libcamera-devel wrote:\n> > This patch describes a series of controls that allow applications to\n> > drive AF algorithms:\n> >\n> > AfMode - manual, auto or continuous\n> > AfLensPosition - set lens position to macro/hyperfocal/infinity\n> > AfRange - full, macro or normal\n> > AfSpeed - fast or slow\n> > AfWindows - AF window locations\n> > AfTrigger - start (trigger an AF scan) or cancel\n> > AfPause - pause continuous AF\n> > LensPosition - position of lens from lens driver\n> > AfState - reports the mode, whether scanning/success/failure\n> > ---\n> > src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> > 1 file changed, 313 insertions(+), 60 deletions(-)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > index 9d4638ae..23607368 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -406,27 +406,6 @@ controls:\n> > The camera will cancel any active or completed metering sequence.\n> > The AE algorithm is reset to its initial state.\n> >\n> > - - AfTrigger:\n> > - type: int32_t\n> > - draft: true\n> > - description: |\n> > - Control for AF trigger. Currently identical to\n> > - ANDROID_CONTROL_AF_TRIGGER.\n> > -\n> > - Whether the camera device will trigger autofocus for this request.\n> > - enum:\n> > - - name: AfTriggerIdle\n> > - value: 0\n> > - description: The trigger is idle.\n> > - - name: AfTriggerStart\n> > - value: 1\n> > - description: The AF routine is started by the camera.\n> > - - name: AfTriggerCancel\n> > - value: 2\n> > - description: |\n> > - The camera will cancel any active trigger and the AF routine is\n> > - reset to its initial state.\n> > -\n> > - NoiseReductionMode:\n> > type: int32_t\n> > draft: true\n> > @@ -507,45 +486,6 @@ controls:\n> > The AE algorithm has started a pre-capture metering session.\n> > \\sa AePrecaptureTrigger\n> >\n> > - - AfState:\n> > - type: int32_t\n> > - draft: true\n> > - description: |\n> > - Control to report the current AF algorithm state. Currently identical to\n> > - ANDROID_CONTROL_AF_STATE.\n> > -\n> > - Current state of the AF algorithm.\n> > - enum:\n> > - - name: AfStateInactive\n> > - value: 0\n> > - description: The AF algorithm is inactive.\n> > - - name: AfStatePassiveScan\n> > - value: 1\n> > - description: |\n> > - AF is performing a passive scan of the scene in continuous\n> > - auto-focus mode.\n> > - - name: AfStatePassiveFocused\n> > - value: 2\n> > - description: |\n> > - AF believes the scene is in focus, but might restart scanning.\n> > - - name: AfStateActiveScan\n> > - value: 3\n> > - description: |\n> > - AF is performing a scan triggered by an AF trigger request.\n> > - \\sa AfTrigger\n> > - - name: AfStateFocusedLock\n> > - value: 4\n> > - description: |\n> > - AF believes has focused correctly and has locked focus.\n> > - - name: AfStateNotFocusedLock\n> > - value: 5\n> > - description: |\n> > - AF has not been able to focus and has locked.\n> > - - name: AfStatePassiveUnfocused\n> > - value: 6\n> > - description: |\n> > - AF has completed a passive scan without finding focus.\n> > -\n> > - AwbState:\n> > type: int32_t\n> > draft: true\n> > @@ -690,4 +630,317 @@ controls:\n> > value. All of the custom test patterns will be static (that is the\n> > raw image must not vary from frame to frame).\n> >\n> > + - AfMode:\n> > + type: int32_t\n> > + draft: true\n>\n> Was having all these controls as draft intentional ? The previous\n> definitions where set to drafts as they mapped to the android\n> definition, but now that we're defining our own ones...\n\nNot really... I only made them \"draft\" because it seemed like\neverything was \"draft\" at first! But happy to change that. Are you\nsaying that these definitely should *not* be draft, then?\n\n>\n> > + description: |\n> > + Control to set the mode of the AF (autofocus) algorithm. Applications\n> > + are allowed to set a new mode, and to send additional controls for\n> > + that new mode, in the same request.\n> > +\n> > + An implementation may choose not to implement all the modes.\n> > +\n> > + enum:\n> > + - name: AfModeManual\n> > + value: 0\n> > + description: |\n> > + The AF algorithm is in manual mode. In this mode it will never\n> > + perform any action nor move the lens of its own accord, but an\n> > + application can set controls to move the lens.\n> > +\n> > + In this mode the AfState will always report AfStateManual.\n> > + - name: AfModeAuto\n> > + value: 1\n> > + description: |\n> > + The AF algorithm is in auto mode. This means that the algorithm\n> > + will never move the lens or change state unless the AfTrigger\n> > + control is used. The AfTrigger control can be used to initiate a\n> > + focus scan, the results of which will also be reported by AfState.\n> > +\n> > + If the autofocus algorithm is moved from AfModeAuto to another\n> > + mode while a scan is in progress, the scan is cancelled\n> > + immediately, without waiting for the scan to finish.\n> > +\n> > + When first entering this mode the AfState will report\n> > + AfStateAuto. When a trigger control is sent, AfState will\n> > + report AfStateAutoScanning for a period before spontaneously\n> > + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> > + the outcome of the scan. It will remain in this state until\n> > + another scan is initiated by the AfTrigger control. If a scan is\n> > + cancelled (without changing to another mode), AfState will return\n> > + to AfStateAuto.\n> > + - name: AfModeContinuous\n> > + value: 2\n> > + description: |\n> > + The AF algorithm is in continuous mode. This means that the lens\n> > + can re-start a scan spontaneously at any moment, without any user\n> > + intervention. The AfState still reports whether the algorithm is\n> > + currently scanning or not, though the application has no ability\n> > + to initiate or cancel scans (though it can \"pause\" them), nor to\n> > + move the lens for itself.\n> > +\n> > + When set to AfModeContinuous, the system will immediately initiate\n> > + a scan so AfState will report AfStateContinuousScanning, and will\n> > + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> > + depending on the scan result.\n> > +\n> > + - AfLensPosition:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control to set the position of the lens to one of the following\n> > + predefined locations. This control only has any effect when the AfMode\n> > + is set to AfModeManual.\n> > +\n> > + This control is distinct from the LensPosition control, which sets the\n> > + lens position using the lens driver's units.\n> > + enum:\n> > + - name: AfLensPositionMacro\n> > + value: 0\n> > + description: The closest focal position of the lens.\n> > + - name: AfLensPositionHyperfocal\n> > + value: 1\n> > + description: Hyperfocal position.\n> > + - name: AfLensPositionInfinity\n> > + value: 2\n> > + description: The furthest focal position (usually infinity).\n>\n> Interesting, bikeshedding on the name apart, is this some 'simplified'\n> LensState ? Do you think there is a use case for this ? Why would\n> anyone set the AF to auto mode then use some coarse grained control\n> like this one ? Wouldn't it make more sense as an 'hint' to the AF\n> algorithm to restrict the search space ?\n\nIn the light of your comments below about the LensPosition not using\ndriver units, then maybe this could be deleted, and we could select\nsome kind of canonical range for the lens?\n\nThe most basic use case would be following on from the capture of an\nimage after an AF scan:\n\n- capture image\n- AfMode to manual\n- Lens to hyperfocal\n- AfMode to auto\n- now wait for the shutter button to be pressed before you start\nanother AF scan.\n\nIf your UI has a \"macro\" capture mode of some kind, maybe it's \"Lens\nto macro\" instead of \"Lens to hyperfocal\".\n\nBut as noted, if LensPosition allows us to select hyperfocal/macro\ndirectly, then we can do without this one.\n\n>\n> (Btw I have a use case for supplying a range of lens positions, which\n> has been previously estimated according to some pre-defined use cases,\n> to the AF algorithm in order restrict the AF search range to increase\n> the speed of a scan. Do you think it's a valid use case and it's worth a\n> control ?)\n\nI'd imagined doing this kind of thing with the AfSpeed and AfRange\ncontrols and by having the AF algorithm tuned for the module. I'm not\nconvinced about passing in lens positions from outside the camera\nsystem. Where would they come from? Shouldn't the camera system know\nwhat positions to use?\n\nIn the Raspberry Pi world we would of course store this kind of\ninformation in the tuning file. I've noticed some discussion elsewhere\nabout supplying configuration to the IPAs - it seems like this might\nbe a case in point? I think that for real camera systems, in the end,\nthere is no escape from \"configuration\" (aka. \"tuning\").\n\n>\n> > +\n> > + - AfRange:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control to set the range of focus distances that is scanned. An\n> > + implementation may choose not to implement all the options here.\n>\n> Only valid in Auto and Continuous mode, right ?\n\nGenerally I've been expecting that you could change settings like this\nin Manual mode too, though it obviously can't have any effect until\nyou go back to Auto/Continuous. Does that sound reasonable?\n\n>\n> (This makes me think that we could easily model Continuous by adding a\n> TriggerContinuous when in Auto mode. With a TriggerPause we could get rid of\n> AfPause... well, it's quite a redesign so I'll stop here unless\n> someone thinks it's a really good idea :)\n\nActually I'm quite strongly against anything like this. I think Auto\nand Continuous should be quite separate. The reason is that Auto is\n(relatively) easy to implement, you do a scan over some range and\nchoose the best spot. But Continuous AF is an absolute nightmare to do\nwell. You have all these problems:\n\n- The algo doesn't realise it needs to rescan and so seems to be\n\"stuck\". (Watch people waving their hands in front of the lens to try\nand unstick it.) So you add ever more tests and arbitrary thresholds\non different statistics to tell if a rescan is needed, and then the\nwhole thing becomes untunable.\n- The algo rescans too often. A car in the distance goes by and the\nwhole image see-saws in and out while a rescan happens, ending back at\nthe same place. Yet more tests, thresholds...\n- You can never make all your tests and thresholds work at the same\ntime but in different conditions. So you make them variable depending\non things like exposure and gain. Now you are truly doomed.\n- The algo is generally not robust to camera shake, which frequently\ncauses it to end up \"failing\" with a completely out-of-focus image.\nAnd then you can't get it to rescan... cue more hand-waving in front\nof the lens. You implement more special tests and timeouts... At this\npoint you will wish for the earth to swallow you up.\n- When the algo rescans it starts off going the wrong way causing\nextra see-sawing.\n- You try to solve the last problem by doing really small initial\nsteps. Only then it ends up statistically unstable.\n\nFrom the above you may detect that we have been there and have several\nT-shirts. Speaking for myself, I have no plans to implement Continuous\nAF without PDAF sensors, as PDAF is quite a big help.\n\n>\n> > + enum:\n> > + - name: AfRangeNormal\n> > + value: 0\n> > + description: |\n> > + A wide range of focus distances is scanned, all the way from\n> > + infinity down to close distances, though depending on the\n> > + implementation, possibly not including the very closest macro\n> > + positions.\n> > + - name: AfRangeMacro\n> > + value: 1\n> > + description: Only close distances are scanned.\n> > + - name: AfRangeFull\n> > + value: 2\n> > + description: |\n> > + The full range of focus distances is scanned just as with\n> > + AfRangeNormal but this time including the very closest macro\n> > + positions.\n> > +\n> > + - AfSpeed:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control that determines whether the AF algorithm is to move the lens\n> > + as quickly as possible or more steadily. For example, during video\n> > + recording it may be desirable not to move the lens too abruptly, but\n> > + when in a preview mode (waiting for a still capture) it may be\n> > + helpful to move the lens as quickly as is reasonably possible.\n> > + enum:\n> > + - name: AfSpeedNormal\n> > + value: 0\n> > + description: Move the lens at its usual speed.\n> > + - name: AfSpeedFast\n> > + value: 1\n> > + description: Move the lens more quickly.\n> > +\n> > + - AfWindows:\n> > + type: Rectangle\n> > + draft: true\n> > + description: |\n> > + Sets the focus windows used by the AF algorithm. The units used express\n> > + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> > + image), as u0.16 format numbers.\n> > +\n> > + In order to be activated, a rectangle must be programmed with non-zero\n> > + width and height. If no rectangles are programmed in this way, then the\n> > + system will choose its own single default window in the centre of the\n> > + image.\n> > +\n> > + The details of how the windows are used are platform dependent. We note\n> > + that when there is more than one AF window, a typical implementation\n> > + might find the optimal focus position for each one and finally select\n> > + the window closest to the camera.\n> > +\n> > + size: [platform dependent]\n>\n> I think this should just be\n>\n> size: [n]\n\nOK!\n\n> > +\n> > + - AfTrigger:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> > + and can also be used to terminate a scan early.\n> > +\n> > + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> > +\n> > + enum:\n> > + - name: AfTriggerStart\n> > + value: 0\n> > + description: Start an AF scan. Ignored if a scan is in progress.\n> > + - name: AfTriggerCancel\n> > + value: 1\n> > + description: Cancel an AF scan. This does not cause the lens to move\n> > + anywhere else. Ignored if no scan is in progress.\n> > +\n> > + - AfPause:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + This control has no effect except when in continuous autofocus mode\n> > + (AfModeContinuous). It can be used to pause any lens movements while\n> > + (for example) images are captured. The algorithm remains inactive\n> > + until it is instructed to resume.\n> > +\n> > + enum:\n> > + - name: AfPauseImmediate\n> > + value: 0\n> > + description: |\n> > + Pause the continuous autofocus algorithm immediately, whether or\n> > + not any kind of scan is underway. If the AfState was previously\n> > + reporting AfStateContinuousScanning it will now change to\n> > + AfStateContinuousScanningPaused, and similarly for\n> > + AfStateContinuousFocused and AfStateContinuousFailed.\n> > + - name AfPauseDeferred\n> > + value: 1\n> > + description: |\n> > + Pause the continuous autofocus algorithm as soon as it is no longer\n> > + scanning. If the AfState is currently reporting\n> > + AfStateContinuousFocused is will change to\n> > + AfStateContinuousFocusedPaused, and similarly in the case of\n> > + AfStateContinuousFailed.\n> > +\n> > + If AfState reports AfStateContinuousScanning it will change to\n> > + AfStateContinuousScanningPausing, and then move to one of\n> > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > + when the scan completes.\n> > + - name: AfPauseResume\n> > + value: 2\n> > + description: |\n> > + Resume continous autofocus operation. The algorithm starts again\n> > + from exactly where it left off, and the AfState will drop the\n> > + 'Paused' or 'Pausing' part of the state.\n> > +\n> > + - LensPosition:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Acts as a control to instruct the lens to move to a particular position\n> > + and also reports back the position of the lens for each frame.\n> > +\n> > + The units are determined by the lens driver.\n>\n> Ugh, we cannot do this in the control definition. Driver specificities\n> cannot surface to the controls definition, which should be platform\n> independent.\n>\n> I think we'll have to establish a numerical range (go ahead if you\n> have any idea) and translate to the actual driver values in the\n> CameraLens class ?\n\nI wasn't aware of this, but maybe it's a good thing. The question is\nwhat kind of range to use, and how do you know what limits apply for\nany given camera module? How about a dioptre-like scale, normalised\nfor the hyperfocal distance H. It would go like this:\n\nThe value n would mean being focused at distance H/n (n does not have\nto be an integer). So:\n\nThe value 1 would mean hyperfocal position (focal distance H/1).\nThe value 0 would mean infinity (focal distance H/0).\nLarger values would get progressively closer.\n\nSetting a control value outside the allowed range will clamp it to\nthat end of the range.\n\nIn the Raspberry Pi world it would be sufficient for the tuning file\nto know the allowable range of values. More generally, I could imagine\nan application might want to say \"focus at precisely 1m\" in which case\nit would have to know the value of H from a property or something,\nthough providing such information would be optional (some lenses\nsimply aren't precisely calibrated). Perhaps an application might want\nto get the allowable range of (maybe approximate) focal distances too?\n\n>\n> > +\n> > + The LensPosition control is ignored unless the AfMode is set to\n> > + AfModeManual.\n> > +\n> > + Note that this control is distinct from AfLensPosition, which allows\n> > + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> > + than using lens driver units.\n> > +\n> > + - AfState:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Reports the current state of the AF algorithm. The possible state changes\n> > + are described below, although we add that the following general state\n> > + changes are also allowed.\n>\n> wow, there is a lot of states here :)\n\nDid you follow the link to the state transition diagram? I would hope\nthat should go some way to explaining it! Here's the link again for\nconvenience:\n\nhttps://docs.google.com/drawings/d/1rU2nS82dEC8Y2eY0lKPkQQhd67gKLOl7PvSeUcL1j_E/edit?usp=sharing\n\n>\n> Let me copy them here\n>\n> - name: AfStateManual\n> - name: AfStateAuto\n> - name: AfStateAutoScanning\n> - name: AfStateAutoFocused\n> - name: AfStateAutoFailed\n> - name: AfStateContinuousScanning\n> - name: AfStateContinuousScanningPausing\n> - name: AfStateContinuousScanningPaused\n> - name: AfStateContinuousFocused\n> - name: AfStateContinuousFocusedPaused\n> - name: AfStateContinuousFailed\n> - name: AfStateContinuousFailedPaused\n\nI \"multiplied out\" all the different parts of the state to make a\nsingle long list. Actually I think that makes it more Android-like.\nBut we could reduce this number if, for example, we're happy to report\nthe state of AfMode separately on every frame. I don't mind - do we\nhave a general preference for this kind of thing?\n\n>\n> (please note that AfStateAuto seems mis-aligned, unless I messed up\n> something while cutting&paste. Please check the compiled doc to see\n> how it looks like...)\n\nI will check.\n\n>\n> So we have three macro groups\n>\n> - Manual\n>\n> - Auto\n> - Scanning\n> - Focus\n> - Failed\n>\n> - Continuous\n>\n> - Scanning\n> - Pausing\n> - Paused\n> - Focused\n> - Paused\n> - Failed\n> - Paused\n>\n> The first group is AfMode (Manual, Auto, Continuous) and can be\n> deduced by inspecting the AfMode metadata.\n>\n> The second group is the actual state (Scannng, Focused, Failed)\n>\n> The third group is specific to continuous mode as it accepts a Pause\n> trigger.\n>\n> I understand why you have this, as expressing \"The AfState is set to\n> $something when in AfModeManual\" etc etc might get cumbersome, but I\n> think it might be worth a try\n>\n> 1) Manual mode (the bad)\n> As we all know there is a delay between when a lens is instructed\n> to move and when it reaches its final state.\n>\n> In Manual mode the lens is moved with LensPosition and\n> AfLensPosition.\n>\n> To know the current state:\n> - With LensPosition metadata can be inspected until it doesn't reach the\n> desired value.\n> - With AfLensPosition it is not clear to me how to know when the lens\n> as reched one of the pre-defined positions.\n\nIt sounds like we're deleting AfLensPosition.\n\n>\n> Is it so wrong to report either (Scanning,Focus,Failed) even for\n> Manual mode ?\n\nI don't think it's wrong. If we have a single long list of states then\nit just makes for more states. If we don't then it's harmless, though\nI'm not aware of a use for it.\n\n>\n> 2) AutoMode (the good)\n> This seems the easy one, as (Scanning,Focus,Failed) are already\n> there\n>\n> 3) Continous (the ugly)\n> Continuous mode can be paused. That's a shame :)\n> Would that be that bad keeping a Paused or Pausing state which can\n> only be returned for Continuous mode.\n\nThe whole paused/pausing thing rather complicates things, but it does\nseem to me that it's necessary. If you were in Continuous mode, you\nwould rather the lens didn't set off again just as you want to capture\na sequence of pictures.\n\n>\n> All in all, can we reduce the number of states to\n>\n> - Scanning\n>\n> The AF algorithm is scanning and has not yet reached a\n> stable result. When running in Auto mode an AF scan is\n> started by using the AfTrigger control. When in Manual mode\n> and a new lens position is requested either by sending a\n> LensPosition or AfLensPosition control, the AF state will\n> report \"Scanning\" until the lens doesn't reach the desired\n> position. Similarly, when running in continuous mode and the\n> AF algorithm decides to start a scan spontaneously the\n> AfState will report Scanning until a new stable state is\n> reached.\n>\n> - Focus\n>\n> The AF algorithm has reached a stable state and the image is\n> now focused. When running in Manual mode the lens has\n> reached the position requested using either LensPosition and\n> AfLensPosition.\n>\n> - Failed\n>\n> The AF algorithm has completed a scan and without finding a\n> good focus position. When running in manual mode, the camera\n> has failed moving the lens to the desired position.\n>\n> - Paused (copying your below text, please see comments below)\n>\n> The AF algorithm is in continuous mode (AfModeContinuous) and\n> was scanning when AfPauseImmediate was sent. The scan will not\n> complete and the algorithm will remain in this state.\n>\n> - Pausing\n>\n> The AF algorithm is in continuous mode (AfModeContinuous) and\n> was scanning when AfPauseDeferred was sent. The scan will complete\n> at which point the algorithm moves to the\n> AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> state.\n\nI think we could get by if we report the following for every frame:\n\nAfMode - Manual, Auto or Continuous\nAfState - Scanning, Focused or Failed\nAfPause - Resume (which means it's running), Immediate or Deferred\n(these states only apply during Continuous; the Deferred value can\nonly occur when also Scanning).\n\nOr we could wrap AfState and AfPause together:\n\nAfState - Scanning, Focused. Failed, ScanningPausing, ScanningPaused,\nFocusedPaused, FailedPaused.\n\nThat makes some of the interaction between pausing and scanning\nclearer, but then it's more states again. I don't know...\n\n>\n>\n> > +\n> > + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> > + may be set to AfModeAuto and the algorithm will move to the\n> > + AfStateAuto state. If AfTriggerStart is sent at the same time\n> > + then the algorithm will move to AfStateAutoScanning state.\n> > +\n> > + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> > + be set to AfModeContinuous and the algorithm will move to\n> > + AfStateContinuousScanning.\n> > +\n> > + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> > + be set to AfModeManual and the algorithm will move to\n> > + AfStateManual. The lens will not be moved and will be left where\n> > + it was at that moment.\n> > +\n> > + enum:\n> > + - name: AfStateManual\n> > + value: 0\n> > + description: |\n> > + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> > + and AfLensPosition controls can be used directly to move the lens.\n> > + - name: AfStateAuto\n> > + value: 1\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> > + been moved into that state, or a scan that was in progress has been\n> > + cancelled.\n> > + - name: AfStateAutoScanning\n> > + value: 2\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> > + started using the AfTrigger control. The scan can be cancelled by\n> > + sending AfTriggerCancel at which point the algorithm will either\n> > + move back to AfStateAuto or, if the scan actually completes\n> > + before the cancel request is processed, to one of\n> > + AfStateAutoFocused or AfStateAutoFailed.\n> > + - name: AfStateAutoFocused\n> > + value: 3\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > + completed with the result that the algorithm believes the image is\n> > + now in focus.\n> > + - name: AfStateAutoFailed\n> > + value: 4\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > + completed with the result that the algorithm did not find a good\n> > + focus position.\n> > + - name: AfStateContinuousScanning\n> > + value: 5\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + is currently scanning for a good focus position. This occurs when\n> > + the mode is first set to AfModeContinuous, or may happen\n> > + spontaneously when the algorithm believes a re-scan is needed.\n> > + - name: AfStateContinuousScanningPausing\n> > + value: 6\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + was scanning when AfPauseDeferred was sent. The scan will complete\n> > + at which point the algorithm moves to the\n> > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > + state.\n> > + - name: AfStateContinuousScanningPaused\n> > + value: 7\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + was scanning when AfPauseImmediate was sent. The scan will not\n>\n> Why can't the algorithm be paused when it has reached a stable state\n> but only when scanning ? Won't a pause 'freeze' the algorithm in its\n> current state no matter what it is ? I understand with\n> \"AfPauseDeferred\" that it has to wait for AF to be locked before\n> stopping it, but with \"Immediate\" doesn't \"pause\" apply to all states ?\n\nWasn't sure I understood this. It can be paused once it's reached\nFocused or Failed, and then both PauseImmediate and PauseDeferred will\ncause it to report FocusedPaused or FailedPaused. Was that the\nquestion?\n\nAnyway, thanks very much!\n\nDavid\n\n>\n> Thanks\n> j\n>\n> > + complete and the algorithm will remain in this state. The scan\n> > + may be resumed by sending AfPauseResume.\n> > + - name: AfStateContinuousFocused\n> > + value: 8\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + a scan has completed with the algorithm believing it has found a\n> > + good focus position.\n> > + - name: AfStateContinuousFocusedPaused\n> > + value: 9\n> > + description: |\n> > + The AF algorithm was in the AfStateContinuousFocused state and\n> > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > + or it was in the AfStateContinuousScanningPausing state and the\n> > + scan has completed successfully. The algorithm will now remain\n> > + in this state, and may be resumed by sending AfPauseResume.\n> > + - name: AfStateContinuousFailed\n> > + value: 10\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + a scan has completed without finding a good focus position.\n> > + - name: AfStateContinuousFailedPaused\n> > + value: 11\n> > + description: |\n> > + The AF algorithm was in the AfStateContinuousFailed state and\n> > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > + or it was in the AfStateContinuousScanningPausing state and the\n> > + scan has completed unsuccessfully. The algorithm will now remain\n> > + in this state, and may be resumed by sending AfPauseResume.\n> > +\n> > ...\n> > --\n> > 2.30.2\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5B77AC0F1B\n\tfor ;\n\tWed, 6 Apr 2022 13:05:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9C1D865643;\n\tWed, 6 Apr 2022 15:05:34 +0200 (CEST)","from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com\n\t[IPv6:2a00:1450:4864:20::32d])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 86886604B8\n\tfor ;\n\tWed, 6 Apr 2022 15:05:32 +0200 (CEST)","by mail-wm1-x32d.google.com with SMTP id\n\tl9-20020a05600c4f0900b0038ccd1b8642so2985088wmq.0\n\tfor ;\n\tWed, 06 Apr 2022 06:05:32 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649250334;\n\tbh=d8T5+Y+DgvqAosHTL+IuZuMNCFC7ISwyUzv4SIwIWxE=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=1sUnjTQMYCaI0Px0mbuCh55+Eh186oXy/NfEz0YccQwlfg+Ccsy0O5E8CPr9xAlgh\n\t99iQd1ELp/Sstl1F+ph1VUByUsL5swZiaUHUrtyX9iZjtFroogZ64cTxMzWvxGTb7/\n\t5+BNe/bc91oXMA76zPY7PBuDHmfC95hgc/+GPl8oBjP5qzlX+5eR5AoE03IiZwlu07\n\tyulQPFNe7DWtKx1uF4nM7YbzuyfT6zgHLeUgxiGTO3uR2XsWjx/BkdkeNjnGaWsxhV\n\tm75Y5Qkf0kXaLbFnrbmoZqda56pJD8C7scPGB+MFGg9Ibk0D4pCcXTYgWDR/9mTTdl\n\ttNKaMnUeeOAlg==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=NXa5PcIq5rY9WV8EGQ5Yg69lwwCsAVWBxwR8IK5SyY4=;\n\tb=fggRM/RZVP2ZUUpBrJPzqP3IC4OMREs+mnVDWuypuDucTXHPKU3/64c+qu/eyfGmG7\n\t1+8D1uqmCSnz3aQiprCHmKe2QjNS31kNfXxw4ONe8Csr+A65/0/3bP8A7gvNuzWf6/Cr\n\tD83U9ueAA9tlGdkOO7Le4T6BD1sl62ooJspsLQSFpOeEKDAf46yW2JlkAsD0c17wQgb9\n\tZJYGH6qHi3xtgXlRuRyW9DAyXuUU0ZjwQseQNvEqDmemYrIKcph4v80LnEQJu+6lTo/K\n\tnze5EaPgwpGGwjSNM9O5UbnnJl2Tabb/LC52/obyGNg1+P2+FcLNcNCnYT/pTXBXdPxv\n\teGJQ=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"fggRM/RZ\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=NXa5PcIq5rY9WV8EGQ5Yg69lwwCsAVWBxwR8IK5SyY4=;\n\tb=SQMKYYI9dMrXSwvkA/peH0k+tt/F+T20Dx/Fr1SwlycEmvrm2GJ5qIoBrbRWsVuXK7\n\tIh2JIbNTLUp4Uml7Iirz5UZMK4cQ24n81pVlBbHaLlYxdzBNQ7d58ReT8bym851lccdz\n\tMEW46lIjjdkuJUU3xbZGGyGRLvlZ6XFoI5nMQYX44P8+6uoYHYpFKtG2/a9fLGQCEGJb\n\t/pKzg/Kpz9bgltHYk+o3bCX5Vwhd8EWPDRudXGHqvkyFpxxdJKeOTOJmHrqZR1Uns8dE\n\t1ibaQO4dO+z5/U71DtGUrXUiN3wZjcVdO1HC8YqIbRVAcKwrtAu8Uq+iFW2CPR6t93Qq\n\tBADw==","X-Gm-Message-State":"AOAM531Zlbh+8isk19xJ43CC1/qC8WDtCTi7AuOebMb10Jjv4IoAODZz\n\tSb5RKrOOs4N+ZA4Gx/xYgZjYkvkqdloxxPWWtYhz9Q==","X-Google-Smtp-Source":"ABdhPJx0Zf4n2oa/FTrvAsyeSenOSQW8/guI2oJLc5HF/SLuOsIXXKE/7Fs3mqfDE9aWtZNhRyqU58x92qQsc8nmno4=","X-Received":"by 2002:a05:600c:3785:b0:38c:9b55:a477 with SMTP id\n\to5-20020a05600c378500b0038c9b55a477mr7252671wmr.164.1649250331630;\n\tWed, 06 Apr 2022 06:05:31 -0700 (PDT)","MIME-Version":"1.0","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t<20220405203610.peioc4aomqgtxb5w@uno.localdomain>","In-Reply-To":"<20220405203610.peioc4aomqgtxb5w@uno.localdomain>","Date":"Wed, 6 Apr 2022 14:05:20 +0100","Message-ID":"","To":"Jacopo Mondi ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"David Plowman via libcamera-devel ","Reply-To":"David Plowman ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22630,"web_url":"https://patchwork.libcamera.org/comment/22630/","msgid":"<20220406163358.hhvc7vr5jzx6ml56@uno.localdomain>","date":"2022-04-06T16:33:58","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi David,\n + Han-lin for an Android question on metadata\n\nOn Wed, Apr 06, 2022 at 02:05:20PM +0100, David Plowman wrote:\n> Hi Jacopo\n>\n> Thanks for taking the time to read all this!\n>\n> On Tue, 5 Apr 2022 at 21:36, Jacopo Mondi wrote:\n> >\n> > Hi David,\n> > sorry for being late :)\n> >\n> > On Thu, Mar 31, 2022 at 04:17:47PM +0100, David Plowman via libcamera-devel wrote:\n> > > This patch describes a series of controls that allow applications to\n> > > drive AF algorithms:\n> > >\n> > > AfMode - manual, auto or continuous\n> > > AfLensPosition - set lens position to macro/hyperfocal/infinity\n> > > AfRange - full, macro or normal\n> > > AfSpeed - fast or slow\n> > > AfWindows - AF window locations\n> > > AfTrigger - start (trigger an AF scan) or cancel\n> > > AfPause - pause continuous AF\n> > > LensPosition - position of lens from lens driver\n> > > AfState - reports the mode, whether scanning/success/failure\n> > > ---\n> > > src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> > > 1 file changed, 313 insertions(+), 60 deletions(-)\n> > >\n> > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > index 9d4638ae..23607368 100644\n> > > --- a/src/libcamera/control_ids.yaml\n> > > +++ b/src/libcamera/control_ids.yaml\n> > > @@ -406,27 +406,6 @@ controls:\n> > > The camera will cancel any active or completed metering sequence.\n> > > The AE algorithm is reset to its initial state.\n> > >\n> > > - - AfTrigger:\n> > > - type: int32_t\n> > > - draft: true\n> > > - description: |\n> > > - Control for AF trigger. Currently identical to\n> > > - ANDROID_CONTROL_AF_TRIGGER.\n> > > -\n> > > - Whether the camera device will trigger autofocus for this request.\n> > > - enum:\n> > > - - name: AfTriggerIdle\n> > > - value: 0\n> > > - description: The trigger is idle.\n> > > - - name: AfTriggerStart\n> > > - value: 1\n> > > - description: The AF routine is started by the camera.\n> > > - - name: AfTriggerCancel\n> > > - value: 2\n> > > - description: |\n> > > - The camera will cancel any active trigger and the AF routine is\n> > > - reset to its initial state.\n> > > -\n> > > - NoiseReductionMode:\n> > > type: int32_t\n> > > draft: true\n> > > @@ -507,45 +486,6 @@ controls:\n> > > The AE algorithm has started a pre-capture metering session.\n> > > \\sa AePrecaptureTrigger\n> > >\n> > > - - AfState:\n> > > - type: int32_t\n> > > - draft: true\n> > > - description: |\n> > > - Control to report the current AF algorithm state. Currently identical to\n> > > - ANDROID_CONTROL_AF_STATE.\n> > > -\n> > > - Current state of the AF algorithm.\n> > > - enum:\n> > > - - name: AfStateInactive\n> > > - value: 0\n> > > - description: The AF algorithm is inactive.\n> > > - - name: AfStatePassiveScan\n> > > - value: 1\n> > > - description: |\n> > > - AF is performing a passive scan of the scene in continuous\n> > > - auto-focus mode.\n> > > - - name: AfStatePassiveFocused\n> > > - value: 2\n> > > - description: |\n> > > - AF believes the scene is in focus, but might restart scanning.\n> > > - - name: AfStateActiveScan\n> > > - value: 3\n> > > - description: |\n> > > - AF is performing a scan triggered by an AF trigger request.\n> > > - \\sa AfTrigger\n> > > - - name: AfStateFocusedLock\n> > > - value: 4\n> > > - description: |\n> > > - AF believes has focused correctly and has locked focus.\n> > > - - name: AfStateNotFocusedLock\n> > > - value: 5\n> > > - description: |\n> > > - AF has not been able to focus and has locked.\n> > > - - name: AfStatePassiveUnfocused\n> > > - value: 6\n> > > - description: |\n> > > - AF has completed a passive scan without finding focus.\n> > > -\n> > > - AwbState:\n> > > type: int32_t\n> > > draft: true\n> > > @@ -690,4 +630,317 @@ controls:\n> > > value. All of the custom test patterns will be static (that is the\n> > > raw image must not vary from frame to frame).\n> > >\n> > > + - AfMode:\n> > > + type: int32_t\n> > > + draft: true\n> >\n> > Was having all these controls as draft intentional ? The previous\n> > definitions where set to drafts as they mapped to the android\n> > definition, but now that we're defining our own ones...\n>\n> Not really... I only made them \"draft\" because it seemed like\n> everything was \"draft\" at first! But happy to change that. Are you\n> saying that these definitely should *not* be draft, then?\n>\n\nYeah, \"draft\" was used to import controls from Android as they are for\nsake of CTS compliance. Let's drop draft from the ones we're defining\n\n> >\n> > > + description: |\n> > > + Control to set the mode of the AF (autofocus) algorithm. Applications\n> > > + are allowed to set a new mode, and to send additional controls for\n> > > + that new mode, in the same request.\n> > > +\n> > > + An implementation may choose not to implement all the modes.\n> > > +\n> > > + enum:\n> > > + - name: AfModeManual\n> > > + value: 0\n> > > + description: |\n> > > + The AF algorithm is in manual mode. In this mode it will never\n> > > + perform any action nor move the lens of its own accord, but an\n> > > + application can set controls to move the lens.\n> > > +\n> > > + In this mode the AfState will always report AfStateManual.\n> > > + - name: AfModeAuto\n> > > + value: 1\n> > > + description: |\n> > > + The AF algorithm is in auto mode. This means that the algorithm\n> > > + will never move the lens or change state unless the AfTrigger\n> > > + control is used. The AfTrigger control can be used to initiate a\n> > > + focus scan, the results of which will also be reported by AfState.\n> > > +\n> > > + If the autofocus algorithm is moved from AfModeAuto to another\n> > > + mode while a scan is in progress, the scan is cancelled\n> > > + immediately, without waiting for the scan to finish.\n> > > +\n> > > + When first entering this mode the AfState will report\n> > > + AfStateAuto. When a trigger control is sent, AfState will\n> > > + report AfStateAutoScanning for a period before spontaneously\n> > > + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> > > + the outcome of the scan. It will remain in this state until\n> > > + another scan is initiated by the AfTrigger control. If a scan is\n> > > + cancelled (without changing to another mode), AfState will return\n> > > + to AfStateAuto.\n> > > + - name: AfModeContinuous\n> > > + value: 2\n> > > + description: |\n> > > + The AF algorithm is in continuous mode. This means that the lens\n> > > + can re-start a scan spontaneously at any moment, without any user\n> > > + intervention. The AfState still reports whether the algorithm is\n> > > + currently scanning or not, though the application has no ability\n> > > + to initiate or cancel scans (though it can \"pause\" them), nor to\n> > > + move the lens for itself.\n> > > +\n> > > + When set to AfModeContinuous, the system will immediately initiate\n> > > + a scan so AfState will report AfStateContinuousScanning, and will\n> > > + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> > > + depending on the scan result.\n> > > +\n> > > + - AfLensPosition:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + Control to set the position of the lens to one of the following\n> > > + predefined locations. This control only has any effect when the AfMode\n> > > + is set to AfModeManual.\n> > > +\n> > > + This control is distinct from the LensPosition control, which sets the\n> > > + lens position using the lens driver's units.\n> > > + enum:\n> > > + - name: AfLensPositionMacro\n> > > + value: 0\n> > > + description: The closest focal position of the lens.\n> > > + - name: AfLensPositionHyperfocal\n> > > + value: 1\n> > > + description: Hyperfocal position.\n> > > + - name: AfLensPositionInfinity\n> > > + value: 2\n> > > + description: The furthest focal position (usually infinity).\n> >\n> > Interesting, bikeshedding on the name apart, is this some 'simplified'\n> > LensState ? Do you think there is a use case for this ? Why would\n> > anyone set the AF to auto mode then use some coarse grained control\n> > like this one ? Wouldn't it make more sense as an 'hint' to the AF\n> > algorithm to restrict the search space ?\n>\n> In the light of your comments below about the LensPosition not using\n> driver units, then maybe this could be deleted, and we could select\n> some kind of canonical range for the lens?\n>\n\nAh wait. What I was suggesting (\"An \"hint\" to the AF algorithm to\nrestrict the search space\") is basically AfRange.\n\n> The most basic use case would be following on from the capture of an\n> image after an AF scan:\n>\n> - capture image\n> - AfMode to manual\n> - Lens to hyperfocal\n> - AfMode to auto\n> - now wait for the shutter button to be pressed before you start\n> another AF scan.\n\nNaive question:\nWouldn't setting the lens to hyperfocal render the preview unusable ?\n\n>\n> If your UI has a \"macro\" capture mode of some kind, maybe it's \"Lens\n> to macro\" instead of \"Lens to hyperfocal\".\n\nWhy is this different than starting an AF scan (without going through\nManual) with an AfRange that suggests in which range to scan ? Is this\nan optimization to forcefully move the lens closer to where it will\nlikely end being moved to ?\n\n>\n> But as noted, if LensPosition allows us to select hyperfocal/macro\n> directly, then we can do without this one.\n>\n\nProviding my above understanding is correct, it would be a bit less intuitive\nfor applications to use a numerical value instead of some easier\n\"Macro\"/\"Hyperfocal\". Of course, wrappers like your one can simplify this\nfor their consumers.\n\n> >\n> > (Btw I have a use case for supplying a range of lens positions, which\n> > has been previously estimated according to some pre-defined use cases,\n> > to the AF algorithm in order restrict the AF search range to increase\n> > the speed of a scan. Do you think it's a valid use case and it's worth a\n> > control ?)\n>\n> I'd imagined doing this kind of thing with the AfSpeed and AfRange\n> controls and by having the AF algorithm tuned for the module. I'm not\n> convinced about passing in lens positions from outside the camera\n> system. Where would they come from? Shouldn't the camera system know\n> what positions to use?\n>\n\nA possible use case, for controlled environments with low latency\nrequirements, is to run tests beforehand and for each 'scene' define\nwhat is the ideal lens movement range. When the application starts an\nAF scan it would tell the AF algorithm about the 'preferred' range.\n\nI understand this works backwards, as the lens position should be the\nresult of the AF algorithm run not one of its inputs :)\n\n> In the Raspberry Pi world we would of course store this kind of\n> information in the tuning file. I've noticed some discussion elsewhere\n> about supplying configuration to the IPAs - it seems like this might\n> be a case in point? I think that for real camera systems, in the end,\n> there is no escape from \"configuration\" (aka. \"tuning\").\n>\n> >\n> > > +\n> > > + - AfRange:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + Control to set the range of focus distances that is scanned. An\n> > > + implementation may choose not to implement all the options here.\n> >\n> > Only valid in Auto and Continuous mode, right ?\n>\n> Generally I've been expecting that you could change settings like this\n> in Manual mode too, though it obviously can't have any effect until\n> you go back to Auto/Continuous. Does that sound reasonable?\n\nI would rather assume thaat if a control is not valid in the current\ncontext is simply ignored and not cached to be re-applied when the\ncontext change ?\n\n>\n> >\n> > (This makes me think that we could easily model Continuous by adding a\n> > TriggerContinuous when in Auto mode. With a TriggerPause we could get rid of\n> > AfPause... well, it's quite a redesign so I'll stop here unless\n> > someone thinks it's a really good idea :)\n>\n> Actually I'm quite strongly against anything like this. I think Auto\n> and Continuous should be quite separate. The reason is that Auto is\n> (relatively) easy to implement, you do a scan over some range and\n> choose the best spot. But Continuous AF is an absolute nightmare to do\n> well. You have all these problems:\n>\n> - The algo doesn't realise it needs to rescan and so seems to be\n> \"stuck\". (Watch people waving their hands in front of the lens to try\n> and unstick it.) So you add ever more tests and arbitrary thresholds\n> on different statistics to tell if a rescan is needed, and then the\n> whole thing becomes untunable.\n> - The algo rescans too often. A car in the distance goes by and the\n> whole image see-saws in and out while a rescan happens, ending back at\n> the same place. Yet more tests, thresholds...\n> - You can never make all your tests and thresholds work at the same\n> time but in different conditions. So you make them variable depending\n> on things like exposure and gain. Now you are truly doomed.\n> - The algo is generally not robust to camera shake, which frequently\n> causes it to end up \"failing\" with a completely out-of-focus image.\n> And then you can't get it to rescan... cue more hand-waving in front\n> of the lens. You implement more special tests and timeouts... At this\n> point you will wish for the earth to swallow you up.\n> - When the algo rescans it starts off going the wrong way causing\n> extra see-sawing.\n> - You try to solve the last problem by doing really small initial\n> steps. Only then it ends up statistically unstable.\n>\n> From the above you may detect that we have been there and have several\n> T-shirts. Speaking for myself, I have no plans to implement Continuous\n> AF without PDAF sensors, as PDAF is quite a big help.\n>\n\nYes, it feels like you know what you're talking about :)\n\nI understand you would like be cautious with CAF, but if one doesn't\nwant to implement CAF can't she simply not report TriggerContinuous ?\n\nAnyway, that was just an idea thrown in the mixer, don't worry about\nit.\n\n> >\n> > > + enum:\n> > > + - name: AfRangeNormal\n> > > + value: 0\n> > > + description: |\n> > > + A wide range of focus distances is scanned, all the way from\n> > > + infinity down to close distances, though depending on the\n> > > + implementation, possibly not including the very closest macro\n> > > + positions.\n> > > + - name: AfRangeMacro\n> > > + value: 1\n> > > + description: Only close distances are scanned.\n> > > + - name: AfRangeFull\n> > > + value: 2\n> > > + description: |\n> > > + The full range of focus distances is scanned just as with\n> > > + AfRangeNormal but this time including the very closest macro\n> > > + positions.\n> > > +\n> > > + - AfSpeed:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + Control that determines whether the AF algorithm is to move the lens\n> > > + as quickly as possible or more steadily. For example, during video\n> > > + recording it may be desirable not to move the lens too abruptly, but\n> > > + when in a preview mode (waiting for a still capture) it may be\n> > > + helpful to move the lens as quickly as is reasonably possible.\n> > > + enum:\n> > > + - name: AfSpeedNormal\n> > > + value: 0\n> > > + description: Move the lens at its usual speed.\n> > > + - name: AfSpeedFast\n> > > + value: 1\n> > > + description: Move the lens more quickly.\n> > > +\n> > > + - AfWindows:\n> > > + type: Rectangle\n> > > + draft: true\n> > > + description: |\n> > > + Sets the focus windows used by the AF algorithm. The units used express\n> > > + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> > > + image), as u0.16 format numbers.\n> > > +\n> > > + In order to be activated, a rectangle must be programmed with non-zero\n> > > + width and height. If no rectangles are programmed in this way, then the\n> > > + system will choose its own single default window in the centre of the\n> > > + image.\n> > > +\n> > > + The details of how the windows are used are platform dependent. We note\n> > > + that when there is more than one AF window, a typical implementation\n> > > + might find the optimal focus position for each one and finally select\n> > > + the window closest to the camera.\n> > > +\n> > > + size: [platform dependent]\n> >\n> > I think this should just be\n> >\n> > size: [n]\n>\n> OK!\n>\n> > > +\n> > > + - AfTrigger:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> > > + and can also be used to terminate a scan early.\n> > > +\n> > > + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> > > +\n> > > + enum:\n> > > + - name: AfTriggerStart\n> > > + value: 0\n> > > + description: Start an AF scan. Ignored if a scan is in progress.\n> > > + - name: AfTriggerCancel\n> > > + value: 1\n> > > + description: Cancel an AF scan. This does not cause the lens to move\n> > > + anywhere else. Ignored if no scan is in progress.\n> > > +\n> > > + - AfPause:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + This control has no effect except when in continuous autofocus mode\n> > > + (AfModeContinuous). It can be used to pause any lens movements while\n> > > + (for example) images are captured. The algorithm remains inactive\n> > > + until it is instructed to resume.\n> > > +\n> > > + enum:\n> > > + - name: AfPauseImmediate\n> > > + value: 0\n> > > + description: |\n> > > + Pause the continuous autofocus algorithm immediately, whether or\n> > > + not any kind of scan is underway. If the AfState was previously\n> > > + reporting AfStateContinuousScanning it will now change to\n> > > + AfStateContinuousScanningPaused, and similarly for\n> > > + AfStateContinuousFocused and AfStateContinuousFailed.\n> > > + - name AfPauseDeferred\n> > > + value: 1\n> > > + description: |\n> > > + Pause the continuous autofocus algorithm as soon as it is no longer\n> > > + scanning. If the AfState is currently reporting\n> > > + AfStateContinuousFocused is will change to\n> > > + AfStateContinuousFocusedPaused, and similarly in the case of\n> > > + AfStateContinuousFailed.\n> > > +\n> > > + If AfState reports AfStateContinuousScanning it will change to\n> > > + AfStateContinuousScanningPausing, and then move to one of\n> > > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > + when the scan completes.\n> > > + - name: AfPauseResume\n> > > + value: 2\n> > > + description: |\n> > > + Resume continous autofocus operation. The algorithm starts again\n> > > + from exactly where it left off, and the AfState will drop the\n> > > + 'Paused' or 'Pausing' part of the state.\n> > > +\n> > > + - LensPosition:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + Acts as a control to instruct the lens to move to a particular position\n> > > + and also reports back the position of the lens for each frame.\n> > > +\n> > > + The units are determined by the lens driver.\n> >\n> > Ugh, we cannot do this in the control definition. Driver specificities\n> > cannot surface to the controls definition, which should be platform\n> > independent.\n> >\n> > I think we'll have to establish a numerical range (go ahead if you\n> > have any idea) and translate to the actual driver values in the\n> > CameraLens class ?\n>\n> I wasn't aware of this, but maybe it's a good thing. The question is\n\nWell, isn't it the same with \"Brightness\", for example ?\n\n> what kind of range to use, and how do you know what limits apply for\n> any given camera module? How about a dioptre-like scale, normalised\n> for the hyperfocal distance H. It would go like this:\n>\n> The value n would mean being focused at distance H/n (n does not have\n> to be an integer). So:\n>\n> The value 1 would mean hyperfocal position (focal distance H/1).\n> The value 0 would mean infinity (focal distance H/0).\n> Larger values would get progressively closer.\n>\n> Setting a control value outside the allowed range will clamp it to\n> that end of the range.\n\nI sincerely don't know enough about this to tell if it's a good idea\nor not. Hence, it seems like a good idea to me :)\n\n>\n> In the Raspberry Pi world it would be sufficient for the tuning file\n> to know the allowable range of values. More generally, I could imagine\n> an application might want to say \"focus at precisely 1m\" in which case\n> it would have to know the value of H from a property or something,\n> though providing such information would be optional (some lenses\n> simply aren't precisely calibrated). Perhaps an application might want\n> to get the allowable range of (maybe approximate) focal distances too?\n>\n> >\n> > > +\n> > > + The LensPosition control is ignored unless the AfMode is set to\n> > > + AfModeManual.\n> > > +\n> > > + Note that this control is distinct from AfLensPosition, which allows\n> > > + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> > > + than using lens driver units.\n> > > +\n> > > + - AfState:\n> > > + type: int32_t\n> > > + draft: true\n> > > + description: |\n> > > + Reports the current state of the AF algorithm. The possible state changes\n> > > + are described below, although we add that the following general state\n> > > + changes are also allowed.\n> >\n> > wow, there is a lot of states here :)\n>\n> Did you follow the link to the state transition diagram? I would hope\n> that should go some way to explaining it! Here's the link again for\n> convenience:\n>\n> https://docs.google.com/drawings/d/1rU2nS82dEC8Y2eY0lKPkQQhd67gKLOl7PvSeUcL1j_E/edit?usp=sharing\n>\n\nMissed it. Thanks for providing it.\n\n> >\n> > Let me copy them here\n> >\n> > - name: AfStateManual\n> > - name: AfStateAuto\n> > - name: AfStateAutoScanning\n> > - name: AfStateAutoFocused\n> > - name: AfStateAutoFailed\n> > - name: AfStateContinuousScanning\n> > - name: AfStateContinuousScanningPausing\n> > - name: AfStateContinuousScanningPaused\n> > - name: AfStateContinuousFocused\n> > - name: AfStateContinuousFocusedPaused\n> > - name: AfStateContinuousFailed\n> > - name: AfStateContinuousFailedPaused\n>\n> I \"multiplied out\" all the different parts of the state to make a\n> single long list. Actually I think that makes it more Android-like.\n> But we could reduce this number if, for example, we're happy to report\n> the state of AfMode separately on every frame. I don't mind - do we\n> have a general preference for this kind of thing?\n>\n\nGood question.\n\nMore generally we need to establish a policy for metadata. For\ncontrols we know that controls should be attached to a request only\nwhen their value changes. If a control is not supplied with new\nvalues, then the old (or default) one will stay valid.\n\nThis implies the PH/IPA state machine is then stateful and 'remembers'\nwhat was the last value a control was set to.\n\nWhat about metadata instead ? Can we assume the same stateful-ness in\napplications too, or is it a burden we cannot impose on them ?\n\nHan-lin, do you know how does that work with Android ? Does it report\nmetadata even when their values do not change or do they assume that,\nlike controls, a metadata is reported only when its value has changed.\n\nWhat would the preference be for the RPi applications ?\n\n> >\n> > (please note that AfStateAuto seems mis-aligned, unless I messed up\n> > something while cutting&paste. Please check the compiled doc to see\n> > how it looks like...)\n>\n> I will check.\n>\n> >\n> > So we have three macro groups\n> >\n> > - Manual\n> >\n> > - Auto\n> > - Scanning\n> > - Focus\n> > - Failed\n> >\n> > - Continuous\n> >\n> > - Scanning\n> > - Pausing\n> > - Paused\n> > - Focused\n> > - Paused\n> > - Failed\n> > - Paused\n> >\n> > The first group is AfMode (Manual, Auto, Continuous) and can be\n> > deduced by inspecting the AfMode metadata.\n> >\n> > The second group is the actual state (Scannng, Focused, Failed)\n> >\n> > The third group is specific to continuous mode as it accepts a Pause\n> > trigger.\n> >\n> > I understand why you have this, as expressing \"The AfState is set to\n> > $something when in AfModeManual\" etc etc might get cumbersome, but I\n> > think it might be worth a try\n> >\n> > 1) Manual mode (the bad)\n> > As we all know there is a delay between when a lens is instructed\n> > to move and when it reaches its final state.\n> >\n> > In Manual mode the lens is moved with LensPosition and\n> > AfLensPosition.\n> >\n> > To know the current state:\n> > - With LensPosition metadata can be inspected until it doesn't reach the\n> > desired value.\n> > - With AfLensPosition it is not clear to me how to know when the lens\n> > as reched one of the pre-defined positions.\n>\n> It sounds like we're deleting AfLensPosition.\n>\n> >\n> > Is it so wrong to report either (Scanning,Focus,Failed) even for\n> > Manual mode ?\n>\n> I don't think it's wrong. If we have a single long list of states then\n> it just makes for more states. If we don't then it's harmless, though\n> I'm not aware of a use for it.\n>\n> >\n> > 2) AutoMode (the good)\n> > This seems the easy one, as (Scanning,Focus,Failed) are already\n> > there\n> >\n> > 3) Continous (the ugly)\n> > Continuous mode can be paused. That's a shame :)\n> > Would that be that bad keeping a Paused or Pausing state which can\n> > only be returned for Continuous mode.\n>\n> The whole paused/pausing thing rather complicates things, but it does\n> seem to me that it's necessary. If you were in Continuous mode, you\n> would rather the lens didn't set off again just as you want to capture\n> a sequence of pictures.\n>\n> >\n> > All in all, can we reduce the number of states to\n> >\n> > - Scanning\n> >\n> > The AF algorithm is scanning and has not yet reached a\n> > stable result. When running in Auto mode an AF scan is\n> > started by using the AfTrigger control. When in Manual mode\n> > and a new lens position is requested either by sending a\n> > LensPosition or AfLensPosition control, the AF state will\n> > report \"Scanning\" until the lens doesn't reach the desired\n> > position. Similarly, when running in continuous mode and the\n> > AF algorithm decides to start a scan spontaneously the\n> > AfState will report Scanning until a new stable state is\n> > reached.\n> >\n> > - Focus\n> >\n> > The AF algorithm has reached a stable state and the image is\n> > now focused. When running in Manual mode the lens has\n> > reached the position requested using either LensPosition and\n> > AfLensPosition.\n> >\n> > - Failed\n> >\n> > The AF algorithm has completed a scan and without finding a\n> > good focus position. When running in manual mode, the camera\n> > has failed moving the lens to the desired position.\n> >\n> > - Paused (copying your below text, please see comments below)\n> >\n> > The AF algorithm is in continuous mode (AfModeContinuous) and\n> > was scanning when AfPauseImmediate was sent. The scan will not\n> > complete and the algorithm will remain in this state.\n> >\n> > - Pausing\n> >\n> > The AF algorithm is in continuous mode (AfModeContinuous) and\n> > was scanning when AfPauseDeferred was sent. The scan will complete\n> > at which point the algorithm moves to the\n> > AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > state.\n>\n> I think we could get by if we report the following for every frame:\n>\n> AfMode - Manual, Auto or Continuous\n> AfState - Scanning, Focused or Failed\n> AfPause - Resume (which means it's running), Immediate or Deferred\n> (these states only apply during Continuous; the Deferred value can\n> only occur when also Scanning).\n>\n> Or we could wrap AfState and AfPause together:\n>\n> AfState - Scanning, Focused. Failed, ScanningPausing, ScanningPaused,\n> FocusedPaused, FailedPaused.\n>\n> That makes some of the interaction between pausing and scanning\n> clearer, but then it's more states again. I don't know...\n>\n\nLet's clarify what the policy on metadata is first.\n\n> >\n> >\n> > > +\n> > > + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> > > + may be set to AfModeAuto and the algorithm will move to the\n> > > + AfStateAuto state. If AfTriggerStart is sent at the same time\n> > > + then the algorithm will move to AfStateAutoScanning state.\n> > > +\n> > > + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> > > + be set to AfModeContinuous and the algorithm will move to\n> > > + AfStateContinuousScanning.\n> > > +\n> > > + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> > > + be set to AfModeManual and the algorithm will move to\n> > > + AfStateManual. The lens will not be moved and will be left where\n> > > + it was at that moment.\n> > > +\n> > > + enum:\n> > > + - name: AfStateManual\n> > > + value: 0\n> > > + description: |\n> > > + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> > > + and AfLensPosition controls can be used directly to move the lens.\n> > > + - name: AfStateAuto\n> > > + value: 1\n> > > + description: |\n> > > + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> > > + been moved into that state, or a scan that was in progress has been\n> > > + cancelled.\n> > > + - name: AfStateAutoScanning\n> > > + value: 2\n> > > + description: |\n> > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> > > + started using the AfTrigger control. The scan can be cancelled by\n> > > + sending AfTriggerCancel at which point the algorithm will either\n> > > + move back to AfStateAuto or, if the scan actually completes\n> > > + before the cancel request is processed, to one of\n> > > + AfStateAutoFocused or AfStateAutoFailed.\n> > > + - name: AfStateAutoFocused\n> > > + value: 3\n> > > + description: |\n> > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > > + completed with the result that the algorithm believes the image is\n> > > + now in focus.\n> > > + - name: AfStateAutoFailed\n> > > + value: 4\n> > > + description: |\n> > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > > + completed with the result that the algorithm did not find a good\n> > > + focus position.\n> > > + - name: AfStateContinuousScanning\n> > > + value: 5\n> > > + description: |\n> > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > + is currently scanning for a good focus position. This occurs when\n> > > + the mode is first set to AfModeContinuous, or may happen\n> > > + spontaneously when the algorithm believes a re-scan is needed.\n> > > + - name: AfStateContinuousScanningPausing\n> > > + value: 6\n> > > + description: |\n> > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > + was scanning when AfPauseDeferred was sent. The scan will complete\n> > > + at which point the algorithm moves to the\n> > > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > + state.\n> > > + - name: AfStateContinuousScanningPaused\n> > > + value: 7\n> > > + description: |\n> > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > + was scanning when AfPauseImmediate was sent. The scan will not\n> >\n> > Why can't the algorithm be paused when it has reached a stable state\n> > but only when scanning ? Won't a pause 'freeze' the algorithm in its\n> > current state no matter what it is ? I understand with\n> > \"AfPauseDeferred\" that it has to wait for AF to be locked before\n> > stopping it, but with \"Immediate\" doesn't \"pause\" apply to all states ?\n>\n> Wasn't sure I understood this. It can be paused once it's reached\n> Focused or Failed, and then both PauseImmediate and PauseDeferred will\n> cause it to report FocusedPaused or FailedPaused. Was that the\n> question?\n\nYou're right, I have probably missed FocusedPaused and FailedPaused.\n:)\n\nThanks\n j\n\n>\n> Anyway, thanks very much!\n>\n> David\n>\n> >\n> > Thanks\n> > j\n> >\n> > > + complete and the algorithm will remain in this state. The scan\n> > > + may be resumed by sending AfPauseResume.\n> > > + - name: AfStateContinuousFocused\n> > > + value: 8\n> > > + description: |\n> > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > + a scan has completed with the algorithm believing it has found a\n> > > + good focus position.\n> > > + - name: AfStateContinuousFocusedPaused\n> > > + value: 9\n> > > + description: |\n> > > + The AF algorithm was in the AfStateContinuousFocused state and\n> > > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > > + or it was in the AfStateContinuousScanningPausing state and the\n> > > + scan has completed successfully. The algorithm will now remain\n> > > + in this state, and may be resumed by sending AfPauseResume.\n> > > + - name: AfStateContinuousFailed\n> > > + value: 10\n> > > + description: |\n> > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > + a scan has completed without finding a good focus position.\n> > > + - name: AfStateContinuousFailedPaused\n> > > + value: 11\n> > > + description: |\n> > > + The AF algorithm was in the AfStateContinuousFailed state and\n> > > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > > + or it was in the AfStateContinuousScanningPausing state and the\n> > > + scan has completed unsuccessfully. The algorithm will now remain\n> > > + in this state, and may be resumed by sending AfPauseResume.\n> > > +\n> > > ...\n> > > --\n> > > 2.30.2\n> > >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2AE03C3256\n\tfor ;\n\tWed, 6 Apr 2022 16:34:05 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8219765642;\n\tWed, 6 Apr 2022 18:34:04 +0200 (CEST)","from relay10.mail.gandi.net (relay10.mail.gandi.net\n\t[217.70.178.230])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BC210604B8\n\tfor ;\n\tWed, 6 Apr 2022 18:34:02 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id D4488240007;\n\tWed, 6 Apr 2022 16:34:00 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649262844;\n\tbh=PLt1nEE3mDdROeNGOoUfqPcGkpHWcrMmNlDzhO0hQtY=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=LLw4QyxTcM5rA2vCqqNdTlJcUhANPbrtgJB8+jK9zNdGdvti4Qv27+aMW1wJKOsgu\n\teCnQ2hraqxWNZu7LDGEnyEvll9o72Le4I1Zd5zh0oHiyd6u/qXKt2xKHDHyJsa5v+q\n\tToh3H6f41xjjwDaN4wa/pOsIYzdPKYYty2VNlG07sMr7xfN7cgUV+Vny+oNSg15XT1\n\tURMfJMQkxngET8qiFdnSioZyVRzi4zRROiK1m4KQp6YhKnmxHd8DnOPHKPyldsqHXH\n\tgmEk05Sx79dBMrh83TQCMrfk8pHlQZmNSqiANrOHSPuyJpgZoycVAZzGBWMYY1b8ko\n\tMsbj4ARgX26lg==","Date":"Wed, 6 Apr 2022 18:33:58 +0200","To":"David Plowman ,\n\tHanlin Chen ","Message-ID":"<20220406163358.hhvc7vr5jzx6ml56@uno.localdomain>","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t<20220405203610.peioc4aomqgtxb5w@uno.localdomain>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22641,"web_url":"https://patchwork.libcamera.org/comment/22641/","msgid":"","date":"2022-04-07T09:05:32","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi Jacopo\n\nThanks for the reply!\n\nOn Wed, 6 Apr 2022 at 17:34, Jacopo Mondi wrote:\n>\n> Hi David,\n> + Han-lin for an Android question on metadata\n>\n> On Wed, Apr 06, 2022 at 02:05:20PM +0100, David Plowman wrote:\n> > Hi Jacopo\n> >\n> > Thanks for taking the time to read all this!\n> >\n> > On Tue, 5 Apr 2022 at 21:36, Jacopo Mondi wrote:\n> > >\n> > > Hi David,\n> > > sorry for being late :)\n> > >\n> > > On Thu, Mar 31, 2022 at 04:17:47PM +0100, David Plowman via libcamera-devel wrote:\n> > > > This patch describes a series of controls that allow applications to\n> > > > drive AF algorithms:\n> > > >\n> > > > AfMode - manual, auto or continuous\n> > > > AfLensPosition - set lens position to macro/hyperfocal/infinity\n> > > > AfRange - full, macro or normal\n> > > > AfSpeed - fast or slow\n> > > > AfWindows - AF window locations\n> > > > AfTrigger - start (trigger an AF scan) or cancel\n> > > > AfPause - pause continuous AF\n> > > > LensPosition - position of lens from lens driver\n> > > > AfState - reports the mode, whether scanning/success/failure\n> > > > ---\n> > > > src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> > > > 1 file changed, 313 insertions(+), 60 deletions(-)\n> > > >\n> > > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > > index 9d4638ae..23607368 100644\n> > > > --- a/src/libcamera/control_ids.yaml\n> > > > +++ b/src/libcamera/control_ids.yaml\n> > > > @@ -406,27 +406,6 @@ controls:\n> > > > The camera will cancel any active or completed metering sequence.\n> > > > The AE algorithm is reset to its initial state.\n> > > >\n> > > > - - AfTrigger:\n> > > > - type: int32_t\n> > > > - draft: true\n> > > > - description: |\n> > > > - Control for AF trigger. Currently identical to\n> > > > - ANDROID_CONTROL_AF_TRIGGER.\n> > > > -\n> > > > - Whether the camera device will trigger autofocus for this request.\n> > > > - enum:\n> > > > - - name: AfTriggerIdle\n> > > > - value: 0\n> > > > - description: The trigger is idle.\n> > > > - - name: AfTriggerStart\n> > > > - value: 1\n> > > > - description: The AF routine is started by the camera.\n> > > > - - name: AfTriggerCancel\n> > > > - value: 2\n> > > > - description: |\n> > > > - The camera will cancel any active trigger and the AF routine is\n> > > > - reset to its initial state.\n> > > > -\n> > > > - NoiseReductionMode:\n> > > > type: int32_t\n> > > > draft: true\n> > > > @@ -507,45 +486,6 @@ controls:\n> > > > The AE algorithm has started a pre-capture metering session.\n> > > > \\sa AePrecaptureTrigger\n> > > >\n> > > > - - AfState:\n> > > > - type: int32_t\n> > > > - draft: true\n> > > > - description: |\n> > > > - Control to report the current AF algorithm state. Currently identical to\n> > > > - ANDROID_CONTROL_AF_STATE.\n> > > > -\n> > > > - Current state of the AF algorithm.\n> > > > - enum:\n> > > > - - name: AfStateInactive\n> > > > - value: 0\n> > > > - description: The AF algorithm is inactive.\n> > > > - - name: AfStatePassiveScan\n> > > > - value: 1\n> > > > - description: |\n> > > > - AF is performing a passive scan of the scene in continuous\n> > > > - auto-focus mode.\n> > > > - - name: AfStatePassiveFocused\n> > > > - value: 2\n> > > > - description: |\n> > > > - AF believes the scene is in focus, but might restart scanning.\n> > > > - - name: AfStateActiveScan\n> > > > - value: 3\n> > > > - description: |\n> > > > - AF is performing a scan triggered by an AF trigger request.\n> > > > - \\sa AfTrigger\n> > > > - - name: AfStateFocusedLock\n> > > > - value: 4\n> > > > - description: |\n> > > > - AF believes has focused correctly and has locked focus.\n> > > > - - name: AfStateNotFocusedLock\n> > > > - value: 5\n> > > > - description: |\n> > > > - AF has not been able to focus and has locked.\n> > > > - - name: AfStatePassiveUnfocused\n> > > > - value: 6\n> > > > - description: |\n> > > > - AF has completed a passive scan without finding focus.\n> > > > -\n> > > > - AwbState:\n> > > > type: int32_t\n> > > > draft: true\n> > > > @@ -690,4 +630,317 @@ controls:\n> > > > value. All of the custom test patterns will be static (that is the\n> > > > raw image must not vary from frame to frame).\n> > > >\n> > > > + - AfMode:\n> > > > + type: int32_t\n> > > > + draft: true\n> > >\n> > > Was having all these controls as draft intentional ? The previous\n> > > definitions where set to drafts as they mapped to the android\n> > > definition, but now that we're defining our own ones...\n> >\n> > Not really... I only made them \"draft\" because it seemed like\n> > everything was \"draft\" at first! But happy to change that. Are you\n> > saying that these definitely should *not* be draft, then?\n> >\n>\n> Yeah, \"draft\" was used to import controls from Android as they are for\n> sake of CTS compliance. Let's drop draft from the ones we're defining\n\nWill do!\n\n>\n> > >\n> > > > + description: |\n> > > > + Control to set the mode of the AF (autofocus) algorithm. Applications\n> > > > + are allowed to set a new mode, and to send additional controls for\n> > > > + that new mode, in the same request.\n> > > > +\n> > > > + An implementation may choose not to implement all the modes.\n> > > > +\n> > > > + enum:\n> > > > + - name: AfModeManual\n> > > > + value: 0\n> > > > + description: |\n> > > > + The AF algorithm is in manual mode. In this mode it will never\n> > > > + perform any action nor move the lens of its own accord, but an\n> > > > + application can set controls to move the lens.\n> > > > +\n> > > > + In this mode the AfState will always report AfStateManual.\n> > > > + - name: AfModeAuto\n> > > > + value: 1\n> > > > + description: |\n> > > > + The AF algorithm is in auto mode. This means that the algorithm\n> > > > + will never move the lens or change state unless the AfTrigger\n> > > > + control is used. The AfTrigger control can be used to initiate a\n> > > > + focus scan, the results of which will also be reported by AfState.\n> > > > +\n> > > > + If the autofocus algorithm is moved from AfModeAuto to another\n> > > > + mode while a scan is in progress, the scan is cancelled\n> > > > + immediately, without waiting for the scan to finish.\n> > > > +\n> > > > + When first entering this mode the AfState will report\n> > > > + AfStateAuto. When a trigger control is sent, AfState will\n> > > > + report AfStateAutoScanning for a period before spontaneously\n> > > > + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> > > > + the outcome of the scan. It will remain in this state until\n> > > > + another scan is initiated by the AfTrigger control. If a scan is\n> > > > + cancelled (without changing to another mode), AfState will return\n> > > > + to AfStateAuto.\n> > > > + - name: AfModeContinuous\n> > > > + value: 2\n> > > > + description: |\n> > > > + The AF algorithm is in continuous mode. This means that the lens\n> > > > + can re-start a scan spontaneously at any moment, without any user\n> > > > + intervention. The AfState still reports whether the algorithm is\n> > > > + currently scanning or not, though the application has no ability\n> > > > + to initiate or cancel scans (though it can \"pause\" them), nor to\n> > > > + move the lens for itself.\n> > > > +\n> > > > + When set to AfModeContinuous, the system will immediately initiate\n> > > > + a scan so AfState will report AfStateContinuousScanning, and will\n> > > > + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> > > > + depending on the scan result.\n> > > > +\n> > > > + - AfLensPosition:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + Control to set the position of the lens to one of the following\n> > > > + predefined locations. This control only has any effect when the AfMode\n> > > > + is set to AfModeManual.\n> > > > +\n> > > > + This control is distinct from the LensPosition control, which sets the\n> > > > + lens position using the lens driver's units.\n> > > > + enum:\n> > > > + - name: AfLensPositionMacro\n> > > > + value: 0\n> > > > + description: The closest focal position of the lens.\n> > > > + - name: AfLensPositionHyperfocal\n> > > > + value: 1\n> > > > + description: Hyperfocal position.\n> > > > + - name: AfLensPositionInfinity\n> > > > + value: 2\n> > > > + description: The furthest focal position (usually infinity).\n> > >\n> > > Interesting, bikeshedding on the name apart, is this some 'simplified'\n> > > LensState ? Do you think there is a use case for this ? Why would\n> > > anyone set the AF to auto mode then use some coarse grained control\n> > > like this one ? Wouldn't it make more sense as an 'hint' to the AF\n> > > algorithm to restrict the search space ?\n> >\n> > In the light of your comments below about the LensPosition not using\n> > driver units, then maybe this could be deleted, and we could select\n> > some kind of canonical range for the lens?\n> >\n>\n> Ah wait. What I was suggesting (\"An \"hint\" to the AF algorithm to\n> restrict the search space\") is basically AfRange.\n\nI wonder if there's a case for a \"custom\" value for AfRange, and we\ncould then even have a control that sets the custom range?\n\n>\n> > The most basic use case would be following on from the capture of an\n> > image after an AF scan:\n> >\n> > - capture image\n> > - AfMode to manual\n> > - Lens to hyperfocal\n> > - AfMode to auto\n> > - now wait for the shutter button to be pressed before you start\n> > another AF scan.\n>\n> Naive question:\n> Wouldn't setting the lens to hyperfocal render the preview unusable ?\n\nHyperfocal is the place where \"most stuff\" is normally in focus\n(strictly speaking, has the greatest depth of field), so in the\nabsence of other information it's normally the best place to put the\nlens. It's just for the preview, you'd typically run another scan for\na capture.\n\n(Some systems run Continuous AF during preview, of course. But you\nknow my anxiety about CAF algorithms... in my opinion just putting the\nlens to hyperfocal is way less annoying than a not-very-good CAF\nimplementation!)\n\n>\n> >\n> > If your UI has a \"macro\" capture mode of some kind, maybe it's \"Lens\n> > to macro\" instead of \"Lens to hyperfocal\".\n>\n> Why is this different than starting an AF scan (without going through\n> Manual) with an AfRange that suggests in which range to scan ? Is this\n> an optimization to forcefully move the lens closer to where it will\n> likely end being moved to ?\n\nIf you know you're doing a macro capture it would be helpful for the\npreview to put the lens somewhere where macro things are likely to be\nmore in focus, instead of using hyperfocal which will likely make\nclose objects very blurry.\n\n>\n> >\n> > But as noted, if LensPosition allows us to select hyperfocal/macro\n> > directly, then we can do without this one.\n> >\n>\n> Providing my above understanding is correct, it would be a bit less intuitive\n> for applications to use a numerical value instead of some easier\n> \"Macro\"/\"Hyperfocal\". Of course, wrappers like your one can simplify this\n> for their consumers.\n\nTrue, though I'm now quite liking the \"dioptre-like\" canonical range\ndescribed below, where 1 = hyperfocal, 0 = infinity, large numbers =\nclose\n\n>\n> > >\n> > > (Btw I have a use case for supplying a range of lens positions, which\n> > > has been previously estimated according to some pre-defined use cases,\n> > > to the AF algorithm in order restrict the AF search range to increase\n> > > the speed of a scan. Do you think it's a valid use case and it's worth a\n> > > control ?)\n> >\n> > I'd imagined doing this kind of thing with the AfSpeed and AfRange\n> > controls and by having the AF algorithm tuned for the module. I'm not\n> > convinced about passing in lens positions from outside the camera\n> > system. Where would they come from? Shouldn't the camera system know\n> > what positions to use?\n> >\n>\n> A possible use case, for controlled environments with low latency\n> requirements, is to run tests beforehand and for each 'scene' define\n> what is the ideal lens movement range. When the application starts an\n> AF scan it would tell the AF algorithm about the 'preferred' range.\n>\n> I understand this works backwards, as the lens position should be the\n> result of the AF algorithm run not one of its inputs :)\n\nIndeed, but as noted above, maybe there's an argument for an\nAfRangeCustom which could be user-settable? We've added \"Custom\"\nvalues for other controls in some places, IIRC.\n\n>\n> > In the Raspberry Pi world we would of course store this kind of\n> > information in the tuning file. I've noticed some discussion elsewhere\n> > about supplying configuration to the IPAs - it seems like this might\n> > be a case in point? I think that for real camera systems, in the end,\n> > there is no escape from \"configuration\" (aka. \"tuning\").\n> >\n> > >\n> > > > +\n> > > > + - AfRange:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + Control to set the range of focus distances that is scanned. An\n> > > > + implementation may choose not to implement all the options here.\n> > >\n> > > Only valid in Auto and Continuous mode, right ?\n> >\n> > Generally I've been expecting that you could change settings like this\n> > in Manual mode too, though it obviously can't have any effect until\n> > you go back to Auto/Continuous. Does that sound reasonable?\n>\n> I would rather assume thaat if a control is not valid in the current\n> context is simply ignored and not cached to be re-applied when the\n> context change ?\n\nPossibly, though I'm slightly nervous about how you'd do things like:\n\n- You're in Manual mode, but want to go to Auto and set the Range to Normal.\n- You're in Auto mode but were using a different range. You want to\nreset it to Normal before switching to Manual.\n\nCan you send the AfRange change and the new mode at the same time in\nboth cases or not? Or do you need two separate requests to send the\nchanges? It's not clear to me exactly what would work. If you say that\nthe AfRange always gets updated, then the behaviour is obvious, and\nyou can send both AfRange and AfMode in the same request which would\nbe easier to use.\n\n>\n> >\n> > >\n> > > (This makes me think that we could easily model Continuous by adding a\n> > > TriggerContinuous when in Auto mode. With a TriggerPause we could get rid of\n> > > AfPause... well, it's quite a redesign so I'll stop here unless\n> > > someone thinks it's a really good idea :)\n> >\n> > Actually I'm quite strongly against anything like this. I think Auto\n> > and Continuous should be quite separate. The reason is that Auto is\n> > (relatively) easy to implement, you do a scan over some range and\n> > choose the best spot. But Continuous AF is an absolute nightmare to do\n> > well. You have all these problems:\n> >\n> > - The algo doesn't realise it needs to rescan and so seems to be\n> > \"stuck\". (Watch people waving their hands in front of the lens to try\n> > and unstick it.) So you add ever more tests and arbitrary thresholds\n> > on different statistics to tell if a rescan is needed, and then the\n> > whole thing becomes untunable.\n> > - The algo rescans too often. A car in the distance goes by and the\n> > whole image see-saws in and out while a rescan happens, ending back at\n> > the same place. Yet more tests, thresholds...\n> > - You can never make all your tests and thresholds work at the same\n> > time but in different conditions. So you make them variable depending\n> > on things like exposure and gain. Now you are truly doomed.\n> > - The algo is generally not robust to camera shake, which frequently\n> > causes it to end up \"failing\" with a completely out-of-focus image.\n> > And then you can't get it to rescan... cue more hand-waving in front\n> > of the lens. You implement more special tests and timeouts... At this\n> > point you will wish for the earth to swallow you up.\n> > - When the algo rescans it starts off going the wrong way causing\n> > extra see-sawing.\n> > - You try to solve the last problem by doing really small initial\n> > steps. Only then it ends up statistically unstable.\n> >\n> > From the above you may detect that we have been there and have several\n> > T-shirts. Speaking for myself, I have no plans to implement Continuous\n> > AF without PDAF sensors, as PDAF is quite a big help.\n> >\n>\n> Yes, it feels like you know what you're talking about :)\n>\n> I understand you would like be cautious with CAF, but if one doesn't\n> want to implement CAF can't she simply not report TriggerContinuous ?\n>\n> Anyway, that was just an idea thrown in the mixer, don't worry about\n> it.\n>\n> > >\n> > > > + enum:\n> > > > + - name: AfRangeNormal\n> > > > + value: 0\n> > > > + description: |\n> > > > + A wide range of focus distances is scanned, all the way from\n> > > > + infinity down to close distances, though depending on the\n> > > > + implementation, possibly not including the very closest macro\n> > > > + positions.\n> > > > + - name: AfRangeMacro\n> > > > + value: 1\n> > > > + description: Only close distances are scanned.\n> > > > + - name: AfRangeFull\n> > > > + value: 2\n> > > > + description: |\n> > > > + The full range of focus distances is scanned just as with\n> > > > + AfRangeNormal but this time including the very closest macro\n> > > > + positions.\n> > > > +\n> > > > + - AfSpeed:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + Control that determines whether the AF algorithm is to move the lens\n> > > > + as quickly as possible or more steadily. For example, during video\n> > > > + recording it may be desirable not to move the lens too abruptly, but\n> > > > + when in a preview mode (waiting for a still capture) it may be\n> > > > + helpful to move the lens as quickly as is reasonably possible.\n> > > > + enum:\n> > > > + - name: AfSpeedNormal\n> > > > + value: 0\n> > > > + description: Move the lens at its usual speed.\n> > > > + - name: AfSpeedFast\n> > > > + value: 1\n> > > > + description: Move the lens more quickly.\n> > > > +\n> > > > + - AfWindows:\n> > > > + type: Rectangle\n> > > > + draft: true\n> > > > + description: |\n> > > > + Sets the focus windows used by the AF algorithm. The units used express\n> > > > + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> > > > + image), as u0.16 format numbers.\n> > > > +\n> > > > + In order to be activated, a rectangle must be programmed with non-zero\n> > > > + width and height. If no rectangles are programmed in this way, then the\n> > > > + system will choose its own single default window in the centre of the\n> > > > + image.\n> > > > +\n> > > > + The details of how the windows are used are platform dependent. We note\n> > > > + that when there is more than one AF window, a typical implementation\n> > > > + might find the optimal focus position for each one and finally select\n> > > > + the window closest to the camera.\n> > > > +\n> > > > + size: [platform dependent]\n> > >\n> > > I think this should just be\n> > >\n> > > size: [n]\n> >\n> > OK!\n> >\n> > > > +\n> > > > + - AfTrigger:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> > > > + and can also be used to terminate a scan early.\n> > > > +\n> > > > + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> > > > +\n> > > > + enum:\n> > > > + - name: AfTriggerStart\n> > > > + value: 0\n> > > > + description: Start an AF scan. Ignored if a scan is in progress.\n> > > > + - name: AfTriggerCancel\n> > > > + value: 1\n> > > > + description: Cancel an AF scan. This does not cause the lens to move\n> > > > + anywhere else. Ignored if no scan is in progress.\n> > > > +\n> > > > + - AfPause:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + This control has no effect except when in continuous autofocus mode\n> > > > + (AfModeContinuous). It can be used to pause any lens movements while\n> > > > + (for example) images are captured. The algorithm remains inactive\n> > > > + until it is instructed to resume.\n> > > > +\n> > > > + enum:\n> > > > + - name: AfPauseImmediate\n> > > > + value: 0\n> > > > + description: |\n> > > > + Pause the continuous autofocus algorithm immediately, whether or\n> > > > + not any kind of scan is underway. If the AfState was previously\n> > > > + reporting AfStateContinuousScanning it will now change to\n> > > > + AfStateContinuousScanningPaused, and similarly for\n> > > > + AfStateContinuousFocused and AfStateContinuousFailed.\n> > > > + - name AfPauseDeferred\n> > > > + value: 1\n> > > > + description: |\n> > > > + Pause the continuous autofocus algorithm as soon as it is no longer\n> > > > + scanning. If the AfState is currently reporting\n> > > > + AfStateContinuousFocused is will change to\n> > > > + AfStateContinuousFocusedPaused, and similarly in the case of\n> > > > + AfStateContinuousFailed.\n> > > > +\n> > > > + If AfState reports AfStateContinuousScanning it will change to\n> > > > + AfStateContinuousScanningPausing, and then move to one of\n> > > > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > > + when the scan completes.\n> > > > + - name: AfPauseResume\n> > > > + value: 2\n> > > > + description: |\n> > > > + Resume continous autofocus operation. The algorithm starts again\n> > > > + from exactly where it left off, and the AfState will drop the\n> > > > + 'Paused' or 'Pausing' part of the state.\n> > > > +\n> > > > + - LensPosition:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + Acts as a control to instruct the lens to move to a particular position\n> > > > + and also reports back the position of the lens for each frame.\n> > > > +\n> > > > + The units are determined by the lens driver.\n> > >\n> > > Ugh, we cannot do this in the control definition. Driver specificities\n> > > cannot surface to the controls definition, which should be platform\n> > > independent.\n> > >\n> > > I think we'll have to establish a numerical range (go ahead if you\n> > > have any idea) and translate to the actual driver values in the\n> > > CameraLens class ?\n> >\n> > I wasn't aware of this, but maybe it's a good thing. The question is\n>\n> Well, isn't it the same with \"Brightness\", for example ?\n\nIndeed, or ExposureTime where we explicitly want microseconds, not driver units.\n\n>\n> > what kind of range to use, and how do you know what limits apply for\n> > any given camera module? How about a dioptre-like scale, normalised\n> > for the hyperfocal distance H. It would go like this:\n> >\n> > The value n would mean being focused at distance H/n (n does not have\n> > to be an integer). So:\n> >\n> > The value 1 would mean hyperfocal position (focal distance H/1).\n> > The value 0 would mean infinity (focal distance H/0).\n> > Larger values would get progressively closer.\n> >\n> > Setting a control value outside the allowed range will clamp it to\n> > that end of the range.\n>\n> I sincerely don't know enough about this to tell if it's a good idea\n> or not. Hence, it seems like a good idea to me :)\n>\n> >\n> > In the Raspberry Pi world it would be sufficient for the tuning file\n> > to know the allowable range of values. More generally, I could imagine\n> > an application might want to say \"focus at precisely 1m\" in which case\n> > it would have to know the value of H from a property or something,\n> > though providing such information would be optional (some lenses\n> > simply aren't precisely calibrated). Perhaps an application might want\n> > to get the allowable range of (maybe approximate) focal distances too?\n> >\n> > >\n> > > > +\n> > > > + The LensPosition control is ignored unless the AfMode is set to\n> > > > + AfModeManual.\n> > > > +\n> > > > + Note that this control is distinct from AfLensPosition, which allows\n> > > > + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> > > > + than using lens driver units.\n> > > > +\n> > > > + - AfState:\n> > > > + type: int32_t\n> > > > + draft: true\n> > > > + description: |\n> > > > + Reports the current state of the AF algorithm. The possible state changes\n> > > > + are described below, although we add that the following general state\n> > > > + changes are also allowed.\n> > >\n> > > wow, there is a lot of states here :)\n> >\n> > Did you follow the link to the state transition diagram? I would hope\n> > that should go some way to explaining it! Here's the link again for\n> > convenience:\n> >\n> > https://docs.google.com/drawings/d/1rU2nS82dEC8Y2eY0lKPkQQhd67gKLOl7PvSeUcL1j_E/edit?usp=sharing\n> >\n>\n> Missed it. Thanks for providing it.\n>\n> > >\n> > > Let me copy them here\n> > >\n> > > - name: AfStateManual\n> > > - name: AfStateAuto\n> > > - name: AfStateAutoScanning\n> > > - name: AfStateAutoFocused\n> > > - name: AfStateAutoFailed\n> > > - name: AfStateContinuousScanning\n> > > - name: AfStateContinuousScanningPausing\n> > > - name: AfStateContinuousScanningPaused\n> > > - name: AfStateContinuousFocused\n> > > - name: AfStateContinuousFocusedPaused\n> > > - name: AfStateContinuousFailed\n> > > - name: AfStateContinuousFailedPaused\n> >\n> > I \"multiplied out\" all the different parts of the state to make a\n> > single long list. Actually I think that makes it more Android-like.\n> > But we could reduce this number if, for example, we're happy to report\n> > the state of AfMode separately on every frame. I don't mind - do we\n> > have a general preference for this kind of thing?\n> >\n>\n> Good question.\n>\n> More generally we need to establish a policy for metadata. For\n> controls we know that controls should be attached to a request only\n> when their value changes. If a control is not supplied with new\n> values, then the old (or default) one will stay valid.\n>\n> This implies the PH/IPA state machine is then stateful and 'remembers'\n> what was the last value a control was set to.\n>\n> What about metadata instead ? Can we assume the same stateful-ness in\n> applications too, or is it a burden we cannot impose on them ?\n>\n> Han-lin, do you know how does that work with Android ? Does it report\n> metadata even when their values do not change or do they assume that,\n> like controls, a metadata is reported only when its value has changed.\n>\n> What would the preference be for the RPi applications ?\n\nI don't think I have a very strong preference. I'm happy with either:\n\n- Report everything in the AfState on every frame.\n- Report the state in multiple metadata items (say, AfState and\nAfMode) on every frame.\n\nI'm less keen on having applications have to remember if they sent\nsomething and then wait for the request where it actually happened, it\nseems to me that would simply add annoying boilerplate code to all\napplications. What do you think?\n\n>\n> > >\n> > > (please note that AfStateAuto seems mis-aligned, unless I messed up\n> > > something while cutting&paste. Please check the compiled doc to see\n> > > how it looks like...)\n> >\n> > I will check.\n> >\n> > >\n> > > So we have three macro groups\n> > >\n> > > - Manual\n> > >\n> > > - Auto\n> > > - Scanning\n> > > - Focus\n> > > - Failed\n> > >\n> > > - Continuous\n> > >\n> > > - Scanning\n> > > - Pausing\n> > > - Paused\n> > > - Focused\n> > > - Paused\n> > > - Failed\n> > > - Paused\n> > >\n> > > The first group is AfMode (Manual, Auto, Continuous) and can be\n> > > deduced by inspecting the AfMode metadata.\n> > >\n> > > The second group is the actual state (Scannng, Focused, Failed)\n> > >\n> > > The third group is specific to continuous mode as it accepts a Pause\n> > > trigger.\n> > >\n> > > I understand why you have this, as expressing \"The AfState is set to\n> > > $something when in AfModeManual\" etc etc might get cumbersome, but I\n> > > think it might be worth a try\n> > >\n> > > 1) Manual mode (the bad)\n> > > As we all know there is a delay between when a lens is instructed\n> > > to move and when it reaches its final state.\n> > >\n> > > In Manual mode the lens is moved with LensPosition and\n> > > AfLensPosition.\n> > >\n> > > To know the current state:\n> > > - With LensPosition metadata can be inspected until it doesn't reach the\n> > > desired value.\n> > > - With AfLensPosition it is not clear to me how to know when the lens\n> > > as reched one of the pre-defined positions.\n> >\n> > It sounds like we're deleting AfLensPosition.\n> >\n> > >\n> > > Is it so wrong to report either (Scanning,Focus,Failed) even for\n> > > Manual mode ?\n> >\n> > I don't think it's wrong. If we have a single long list of states then\n> > it just makes for more states. If we don't then it's harmless, though\n> > I'm not aware of a use for it.\n> >\n> > >\n> > > 2) AutoMode (the good)\n> > > This seems the easy one, as (Scanning,Focus,Failed) are already\n> > > there\n> > >\n> > > 3) Continous (the ugly)\n> > > Continuous mode can be paused. That's a shame :)\n> > > Would that be that bad keeping a Paused or Pausing state which can\n> > > only be returned for Continuous mode.\n> >\n> > The whole paused/pausing thing rather complicates things, but it does\n> > seem to me that it's necessary. If you were in Continuous mode, you\n> > would rather the lens didn't set off again just as you want to capture\n> > a sequence of pictures.\n> >\n> > >\n> > > All in all, can we reduce the number of states to\n> > >\n> > > - Scanning\n> > >\n> > > The AF algorithm is scanning and has not yet reached a\n> > > stable result. When running in Auto mode an AF scan is\n> > > started by using the AfTrigger control. When in Manual mode\n> > > and a new lens position is requested either by sending a\n> > > LensPosition or AfLensPosition control, the AF state will\n> > > report \"Scanning\" until the lens doesn't reach the desired\n> > > position. Similarly, when running in continuous mode and the\n> > > AF algorithm decides to start a scan spontaneously the\n> > > AfState will report Scanning until a new stable state is\n> > > reached.\n> > >\n> > > - Focus\n> > >\n> > > The AF algorithm has reached a stable state and the image is\n> > > now focused. When running in Manual mode the lens has\n> > > reached the position requested using either LensPosition and\n> > > AfLensPosition.\n> > >\n> > > - Failed\n> > >\n> > > The AF algorithm has completed a scan and without finding a\n> > > good focus position. When running in manual mode, the camera\n> > > has failed moving the lens to the desired position.\n> > >\n> > > - Paused (copying your below text, please see comments below)\n> > >\n> > > The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > was scanning when AfPauseImmediate was sent. The scan will not\n> > > complete and the algorithm will remain in this state.\n> > >\n> > > - Pausing\n> > >\n> > > The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > was scanning when AfPauseDeferred was sent. The scan will complete\n> > > at which point the algorithm moves to the\n> > > AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > state.\n> >\n> > I think we could get by if we report the following for every frame:\n> >\n> > AfMode - Manual, Auto or Continuous\n> > AfState - Scanning, Focused or Failed\n> > AfPause - Resume (which means it's running), Immediate or Deferred\n> > (these states only apply during Continuous; the Deferred value can\n> > only occur when also Scanning).\n> >\n> > Or we could wrap AfState and AfPause together:\n> >\n> > AfState - Scanning, Focused. Failed, ScanningPausing, ScanningPaused,\n> > FocusedPaused, FailedPaused.\n> >\n> > That makes some of the interaction between pausing and scanning\n> > clearer, but then it's more states again. I don't know...\n> >\n>\n> Let's clarify what the policy on metadata is first.\n\nAgree!\n\n>\n> > >\n> > >\n> > > > +\n> > > > + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> > > > + may be set to AfModeAuto and the algorithm will move to the\n> > > > + AfStateAuto state. If AfTriggerStart is sent at the same time\n> > > > + then the algorithm will move to AfStateAutoScanning state.\n> > > > +\n> > > > + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> > > > + be set to AfModeContinuous and the algorithm will move to\n> > > > + AfStateContinuousScanning.\n> > > > +\n> > > > + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> > > > + be set to AfModeManual and the algorithm will move to\n> > > > + AfStateManual. The lens will not be moved and will be left where\n> > > > + it was at that moment.\n> > > > +\n> > > > + enum:\n> > > > + - name: AfStateManual\n> > > > + value: 0\n> > > > + description: |\n> > > > + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> > > > + and AfLensPosition controls can be used directly to move the lens.\n> > > > + - name: AfStateAuto\n> > > > + value: 1\n> > > > + description: |\n> > > > + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> > > > + been moved into that state, or a scan that was in progress has been\n> > > > + cancelled.\n> > > > + - name: AfStateAutoScanning\n> > > > + value: 2\n> > > > + description: |\n> > > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> > > > + started using the AfTrigger control. The scan can be cancelled by\n> > > > + sending AfTriggerCancel at which point the algorithm will either\n> > > > + move back to AfStateAuto or, if the scan actually completes\n> > > > + before the cancel request is processed, to one of\n> > > > + AfStateAutoFocused or AfStateAutoFailed.\n> > > > + - name: AfStateAutoFocused\n> > > > + value: 3\n> > > > + description: |\n> > > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > > > + completed with the result that the algorithm believes the image is\n> > > > + now in focus.\n> > > > + - name: AfStateAutoFailed\n> > > > + value: 4\n> > > > + description: |\n> > > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > > > + completed with the result that the algorithm did not find a good\n> > > > + focus position.\n> > > > + - name: AfStateContinuousScanning\n> > > > + value: 5\n> > > > + description: |\n> > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > + is currently scanning for a good focus position. This occurs when\n> > > > + the mode is first set to AfModeContinuous, or may happen\n> > > > + spontaneously when the algorithm believes a re-scan is needed.\n> > > > + - name: AfStateContinuousScanningPausing\n> > > > + value: 6\n> > > > + description: |\n> > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > + was scanning when AfPauseDeferred was sent. The scan will complete\n> > > > + at which point the algorithm moves to the\n> > > > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > > + state.\n> > > > + - name: AfStateContinuousScanningPaused\n> > > > + value: 7\n> > > > + description: |\n> > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > + was scanning when AfPauseImmediate was sent. The scan will not\n> > >\n> > > Why can't the algorithm be paused when it has reached a stable state\n> > > but only when scanning ? Won't a pause 'freeze' the algorithm in its\n> > > current state no matter what it is ? I understand with\n> > > \"AfPauseDeferred\" that it has to wait for AF to be locked before\n> > > stopping it, but with \"Immediate\" doesn't \"pause\" apply to all states ?\n> >\n> > Wasn't sure I understood this. It can be paused once it's reached\n> > Focused or Failed, and then both PauseImmediate and PauseDeferred will\n> > cause it to report FocusedPaused or FailedPaused. Was that the\n> > question?\n>\n> You're right, I have probably missed FocusedPaused and FailedPaused.\n> :)\n>\n> Thanks\n> j\n\nThanks for all the discussion!\n\nDavid\n\n>\n> >\n> > Anyway, thanks very much!\n> >\n> > David\n> >\n> > >\n> > > Thanks\n> > > j\n> > >\n> > > > + complete and the algorithm will remain in this state. The scan\n> > > > + may be resumed by sending AfPauseResume.\n> > > > + - name: AfStateContinuousFocused\n> > > > + value: 8\n> > > > + description: |\n> > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > + a scan has completed with the algorithm believing it has found a\n> > > > + good focus position.\n> > > > + - name: AfStateContinuousFocusedPaused\n> > > > + value: 9\n> > > > + description: |\n> > > > + The AF algorithm was in the AfStateContinuousFocused state and\n> > > > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > > > + or it was in the AfStateContinuousScanningPausing state and the\n> > > > + scan has completed successfully. The algorithm will now remain\n> > > > + in this state, and may be resumed by sending AfPauseResume.\n> > > > + - name: AfStateContinuousFailed\n> > > > + value: 10\n> > > > + description: |\n> > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > + a scan has completed without finding a good focus position.\n> > > > + - name: AfStateContinuousFailedPaused\n> > > > + value: 11\n> > > > + description: |\n> > > > + The AF algorithm was in the AfStateContinuousFailed state and\n> > > > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > > > + or it was in the AfStateContinuousScanningPausing state and the\n> > > > + scan has completed unsuccessfully. The algorithm will now remain\n> > > > + in this state, and may be resumed by sending AfPauseResume.\n> > > > +\n> > > > ...\n> > > > --\n> > > > 2.30.2\n> > > >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B0C6DC3256\n\tfor ;\n\tThu, 7 Apr 2022 09:05:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6B6DF65644;\n\tThu, 7 Apr 2022 11:05:46 +0200 (CEST)","from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com\n\t[IPv6:2a00:1450:4864:20::32e])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B49E961FBB\n\tfor ;\n\tThu, 7 Apr 2022 11:05:44 +0200 (CEST)","by mail-wm1-x32e.google.com with SMTP id\n\tv20-20020a05600c15d400b0038e9a88aee7so69263wmf.3\n\tfor ;\n\tThu, 07 Apr 2022 02:05:44 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649322346;\n\tbh=ATeYfpNoSxg35AD0GN5vhf4XRuJ86UMl3qVcZ3uGtYQ=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=Jj0IMY1Md6iYwPdQS11HZ7QtqaEfGBkWfevoY6dbAkTrAcZgSSo9Ofy4vUeiOsOh7\n\tZpK23ZBoRdFzUPwLRFAnND/LeCvkeHWbjfo/QvjmhPY0DVWvF5MCGdQd8blav9goEo\n\tEQdbMZwH9u0ZQR8gtMmwSQUPjhqRWrXxZrmNxjYNQ2HJrgKe3WIQUMCRHWMF140/al\n\tFrEj3Qpwjlimx3RvqxSejRb0AA2ZRitrz36U+RYeKts2+w3jaOXB63mcFwgR1r5i8g\n\tS9fa5OcCxvYZrYb7M7vxwtowo4W3fB7eC3krDBotzR4omNKtk1WQpElxCOZygApd7e\n\tfdpCcnJ75I6KQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=VfZwSd7Gtuw2GLYOqYhN38LcpD1I0Aqvo24nHgyW44M=;\n\tb=pozaofrKfSUeSGPpUKLph6yD2S+G1lsWACwX0y9CFmaV8RssqM7q/rAoOpyAbSX789\n\tpHZsgVtLC18zgjUNXivO3zEjhgKJvwz0xrYB+1MVVyfOpc0q5IKdAjizLcswryekfE8r\n\twQNQKgTZdwhrmlxO2Q2v4O/QdkcngoBNSIQLcOOOPSqHjsJa+zKZbWV6Tdhl+j0hC4uH\n\tNAQfCjiqCs6by+tP8p5HW03FytG025jOiwNc9PBZo59dE4WuWTcjPm/tMY4yjb9SFfjP\n\tRL2y3IpH1ZaANxHjaHBrEmvrzNiU5kL3YDES2KwRXJ35CPErRhUl5XAT4bFqVp4CqfT/\n\tdawg=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"pozaofrK\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=VfZwSd7Gtuw2GLYOqYhN38LcpD1I0Aqvo24nHgyW44M=;\n\tb=1wc11O6yFDjiYo8v5TD6CnHONltZBjv/K3VMA3BTGvMdo5d29ZuxwSN5f3EoiqKT5U\n\t4ItoReW1UAA+Vs9vFBj7sZeFwoYPxwLAjYigAm0ryAQNSlAKr7nCdOgI2XYXVfF8CMBD\n\tHXS6QHlKePpIuAoFIbrbTUpzKydUjD1/RkAjapAFmKg7jtZEDpxnJD0n6uronxirJxoO\n\tCN+quOaRex2t9NzvytUBseNjG7/H0EZb/PN0isSZAe6gjw91nXakAW+I8eyBPevljgiA\n\tuNAoj09221PK9ArmzLLF9Gi1CfkA8NcoZ0NxWPQL9Ta6KTAXxavQI4C798U+hR9stUT1\n\td2ag==","X-Gm-Message-State":"AOAM532dLB8K68bFUHoZcjIBlo3nOJrjiDGIxlWaTMtPg9rzAtjHsQ6x\n\t99r6dgtP+26vZFcfNRAvwVwsauAyW01iWufVJ+YiNw==","X-Google-Smtp-Source":"ABdhPJyZ0inkvZv2nzOw2xPrqsCkOrYUFtMyj2uVv3JM83LPLj3pTF1vhRMylZoKlmzz3CSx6f4qV1fEnyy5ys8KeP0=","X-Received":"by 2002:a05:600c:3495:b0:38e:6f95:e97 with SMTP id\n\ta21-20020a05600c349500b0038e6f950e97mr11343956wmq.60.1649322343745;\n\tThu, 07 Apr 2022 02:05:43 -0700 (PDT)","MIME-Version":"1.0","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t<20220405203610.peioc4aomqgtxb5w@uno.localdomain>\n\t\n\t<20220406163358.hhvc7vr5jzx6ml56@uno.localdomain>","In-Reply-To":"<20220406163358.hhvc7vr5jzx6ml56@uno.localdomain>","Date":"Thu, 7 Apr 2022 10:05:32 +0100","Message-ID":"","To":"Jacopo Mondi ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"David Plowman via libcamera-devel ","Reply-To":"David Plowman ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22645,"web_url":"https://patchwork.libcamera.org/comment/22645/","msgid":"","date":"2022-04-07T11:31:07","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"content":"Hi David and Jacopo,\n\nOn Thu, Apr 7, 2022 at 5:05 PM David Plowman \nwrote:\n\n> Hi Jacopo\n>\n> Thanks for the reply!\n>\n> On Wed, 6 Apr 2022 at 17:34, Jacopo Mondi wrote:\n> >\n> > Hi David,\n> > + Han-lin for an Android question on metadata\n> >\n> > On Wed, Apr 06, 2022 at 02:05:20PM +0100, David Plowman wrote:\n> > > Hi Jacopo\n> > >\n> > > Thanks for taking the time to read all this!\n> > >\n> > > On Tue, 5 Apr 2022 at 21:36, Jacopo Mondi wrote:\n> > > >\n> > > > Hi David,\n> > > > sorry for being late :)\n> > > >\n> > > > On Thu, Mar 31, 2022 at 04:17:47PM +0100, David Plowman via\n> libcamera-devel wrote:\n> > > > > This patch describes a series of controls that allow applications\n> to\n> > > > > drive AF algorithms:\n> > > > >\n> > > > > AfMode - manual, auto or continuous\n> > > > > AfLensPosition - set lens position to macro/hyperfocal/infinity\n> > > > > AfRange - full, macro or normal\n> > > > > AfSpeed - fast or slow\n> > > > > AfWindows - AF window locations\n> > > > > AfTrigger - start (trigger an AF scan) or cancel\n> > > > > AfPause - pause continuous AF\n> > > > > LensPosition - position of lens from lens driver\n> > > > > AfState - reports the mode, whether scanning/success/failure\n> > > > > ---\n> > > > > src/libcamera/control_ids.yaml | 373\n> +++++++++++++++++++++++++++------\n> > > > > 1 file changed, 313 insertions(+), 60 deletions(-)\n> > > > >\n> > > > > diff --git a/src/libcamera/control_ids.yaml\n> b/src/libcamera/control_ids.yaml\n> > > > > index 9d4638ae..23607368 100644\n> > > > > --- a/src/libcamera/control_ids.yaml\n> > > > > +++ b/src/libcamera/control_ids.yaml\n> > > > > @@ -406,27 +406,6 @@ controls:\n> > > > > The camera will cancel any active or completed\n> metering sequence.\n> > > > > The AE algorithm is reset to its initial state.\n> > > > >\n> > > > > - - AfTrigger:\n> > > > > - type: int32_t\n> > > > > - draft: true\n> > > > > - description: |\n> > > > > - Control for AF trigger. Currently identical to\n> > > > > - ANDROID_CONTROL_AF_TRIGGER.\n> > > > > -\n> > > > > - Whether the camera device will trigger autofocus for this\n> request.\n> > > > > - enum:\n> > > > > - - name: AfTriggerIdle\n> > > > > - value: 0\n> > > > > - description: The trigger is idle.\n> > > > > - - name: AfTriggerStart\n> > > > > - value: 1\n> > > > > - description: The AF routine is started by the camera.\n> > > > > - - name: AfTriggerCancel\n> > > > > - value: 2\n> > > > > - description: |\n> > > > > - The camera will cancel any active trigger and the AF\n> routine is\n> > > > > - reset to its initial state.\n> > > > > -\n> > > > > - NoiseReductionMode:\n> > > > > type: int32_t\n> > > > > draft: true\n> > > > > @@ -507,45 +486,6 @@ controls:\n> > > > > The AE algorithm has started a pre-capture metering\n> session.\n> > > > > \\sa AePrecaptureTrigger\n> > > > >\n> > > > > - - AfState:\n> > > > > - type: int32_t\n> > > > > - draft: true\n> > > > > - description: |\n> > > > > - Control to report the current AF algorithm state.\n> Currently identical to\n> > > > > - ANDROID_CONTROL_AF_STATE.\n> > > > > -\n> > > > > - Current state of the AF algorithm.\n> > > > > - enum:\n> > > > > - - name: AfStateInactive\n> > > > > - value: 0\n> > > > > - description: The AF algorithm is inactive.\n> > > > > - - name: AfStatePassiveScan\n> > > > > - value: 1\n> > > > > - description: |\n> > > > > - AF is performing a passive scan of the scene in\n> continuous\n> > > > > - auto-focus mode.\n> > > > > - - name: AfStatePassiveFocused\n> > > > > - value: 2\n> > > > > - description: |\n> > > > > - AF believes the scene is in focus, but might restart\n> scanning.\n> > > > > - - name: AfStateActiveScan\n> > > > > - value: 3\n> > > > > - description: |\n> > > > > - AF is performing a scan triggered by an AF trigger\n> request.\n> > > > > - \\sa AfTrigger\n> > > > > - - name: AfStateFocusedLock\n> > > > > - value: 4\n> > > > > - description: |\n> > > > > - AF believes has focused correctly and has locked\n> focus.\n> > > > > - - name: AfStateNotFocusedLock\n> > > > > - value: 5\n> > > > > - description: |\n> > > > > - AF has not been able to focus and has locked.\n> > > > > - - name: AfStatePassiveUnfocused\n> > > > > - value: 6\n> > > > > - description: |\n> > > > > - AF has completed a passive scan without finding focus.\n> > > > > -\n> > > > > - AwbState:\n> > > > > type: int32_t\n> > > > > draft: true\n> > > > > @@ -690,4 +630,317 @@ controls:\n> > > > > value. All of the custom test patterns will be static\n> (that is the\n> > > > > raw image must not vary from frame to frame).\n> > > > >\n> > > > > + - AfMode:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > >\n> > > > Was having all these controls as draft intentional ? The previous\n> > > > definitions where set to drafts as they mapped to the android\n> > > > definition, but now that we're defining our own ones...\n> > >\n> > > Not really... I only made them \"draft\" because it seemed like\n> > > everything was \"draft\" at first! But happy to change that. Are you\n> > > saying that these definitely should *not* be draft, then?\n> > >\n> >\n> > Yeah, \"draft\" was used to import controls from Android as they are for\n> > sake of CTS compliance. Let's drop draft from the ones we're defining\n>\n> Will do!\n>\n> >\n> > > >\n> > > > > + description: |\n> > > > > + Control to set the mode of the AF (autofocus) algorithm.\n> Applications\n> > > > > + are allowed to set a new mode, and to send additional\n> controls for\n> > > > > + that new mode, in the same request.\n> > > > > +\n> > > > > + An implementation may choose not to implement all the\n> modes.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfModeManual\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + The AF algorithm is in manual mode. In this mode it\n> will never\n> > > > > + perform any action nor move the lens of its own\n> accord, but an\n> > > > > + application can set controls to move the lens.\n> > > > > +\n> > > > > + In this mode the AfState will always report\n> AfStateManual.\n> > > > > + - name: AfModeAuto\n> > > > > + value: 1\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode. This means that the\n> algorithm\n> > > > > + will never move the lens or change state unless the\n> AfTrigger\n> > > > > + control is used. The AfTrigger control can be used to\n> initiate a\n> > > > > + focus scan, the results of which will also be\n> reported by AfState.\n> > > > > +\n> > > > > + If the autofocus algorithm is moved from AfModeAuto\n> to another\n> > > > > + mode while a scan is in progress, the scan is\n> cancelled\n> > > > > + immediately, without waiting for the scan to finish.\n> > > > > +\n> > > > > + When first entering this mode the AfState will report\n> > > > > + AfStateAuto. When a trigger control is sent, AfState\n> will\n> > > > > + report AfStateAutoScanning for a period before\n> spontaneously\n> > > > > + changing to AfStateAutoFocused or AfStateAutoFailed,\n> depending on\n> > > > > + the outcome of the scan. It will remain in this state\n> until\n> > > > > + another scan is initiated by the AfTrigger control.\n> If a scan is\n> > > > > + cancelled (without changing to another mode), AfState\n> will return\n> > > > > + to AfStateAuto.\n> > > > > + - name: AfModeContinuous\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode. This means\n> that the lens\n> > > > > + can re-start a scan spontaneously at any moment,\n> without any user\n> > > > > + intervention. The AfState still reports whether the\n> algorithm is\n> > > > > + currently scanning or not, though the application has\n> no ability\n> > > > > + to initiate or cancel scans (though it can \"pause\"\n> them), nor to\n> > > > > + move the lens for itself.\n> > > > > +\n> > > > > + When set to AfModeContinuous, the system will\n> immediately initiate\n> > > > > + a scan so AfState will report\n> AfStateContinuousScanning, and will\n> > > > > + settle on one of AfStateContinuousFocused or\n> AfStateContinuousFailed,\n> > > > > + depending on the scan result.\n> > > > > +\n> > > > > + - AfLensPosition:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Control to set the position of the lens to one of the\n> following\n> > > > > + predefined locations. This control only has any effect\n> when the AfMode\n> > > > > + is set to AfModeManual.\n> > > > > +\n> > > > > + This control is distinct from the LensPosition control,\n> which sets the\n> > > > > + lens position using the lens driver's units.\n> > > > > + enum:\n> > > > > + - name: AfLensPositionMacro\n> > > > > + value: 0\n> > > > > + description: The closest focal position of the lens.\n> > > > > + - name: AfLensPositionHyperfocal\n> > > > > + value: 1\n> > > > > + description: Hyperfocal position.\n> > > > > + - name: AfLensPositionInfinity\n> > > > > + value: 2\n> > > > > + description: The furthest focal position (usually\n> infinity).\n> > > >\n> > > > Interesting, bikeshedding on the name apart, is this some\n> 'simplified'\n> > > > LensState ? Do you think there is a use case for this ? Why would\n> > > > anyone set the AF to auto mode then use some coarse grained control\n> > > > like this one ? Wouldn't it make more sense as an 'hint' to the AF\n> > > > algorithm to restrict the search space ?\n> > >\n> > > In the light of your comments below about the LensPosition not using\n> > > driver units, then maybe this could be deleted, and we could select\n> > > some kind of canonical range for the lens?\n> > >\n> >\n> > Ah wait. What I was suggesting (\"An \"hint\" to the AF algorithm to\n> > restrict the search space\") is basically AfRange.\n>\n> I wonder if there's a case for a \"custom\" value for AfRange, and we\n> could then even have a control that sets the custom range?\n>\n> >\n> > > The most basic use case would be following on from the capture of an\n> > > image after an AF scan:\n> > >\n> > > - capture image\n> > > - AfMode to manual\n> > > - Lens to hyperfocal\n> > > - AfMode to auto\n> > > - now wait for the shutter button to be pressed before you start\n> > > another AF scan.\n> >\n> > Naive question:\n> > Wouldn't setting the lens to hyperfocal render the preview unusable ?\n>\n> Hyperfocal is the place where \"most stuff\" is normally in focus\n> (strictly speaking, has the greatest depth of field), so in the\n> absence of other information it's normally the best place to put the\n> lens. It's just for the preview, you'd typically run another scan for\n> a capture.\n>\n> (Some systems run Continuous AF during preview, of course. But you\n> know my anxiety about CAF algorithms... in my opinion just putting the\n> lens to hyperfocal is way less annoying than a not-very-good CAF\n> implementation!)\n>\n> >\n> > >\n> > > If your UI has a \"macro\" capture mode of some kind, maybe it's \"Lens\n> > > to macro\" instead of \"Lens to hyperfocal\".\n> >\n> > Why is this different than starting an AF scan (without going through\n> > Manual) with an AfRange that suggests in which range to scan ? Is this\n> > an optimization to forcefully move the lens closer to where it will\n> > likely end being moved to ?\n>\n> If you know you're doing a macro capture it would be helpful for the\n> preview to put the lens somewhere where macro things are likely to be\n> more in focus, instead of using hyperfocal which will likely make\n> close objects very blurry.\n>\n> >\n> > >\n> > > But as noted, if LensPosition allows us to select hyperfocal/macro\n> > > directly, then we can do without this one.\n> > >\n> >\n> > Providing my above understanding is correct, it would be a bit less\n> intuitive\n> > for applications to use a numerical value instead of some easier\n> > \"Macro\"/\"Hyperfocal\". Of course, wrappers like your one can simplify this\n> > for their consumers.\n>\n> True, though I'm now quite liking the \"dioptre-like\" canonical range\n> described below, where 1 = hyperfocal, 0 = infinity, large numbers =\n> close\n>\n> >\n> > > >\n> > > > (Btw I have a use case for supplying a range of lens positions, which\n> > > > has been previously estimated according to some pre-defined use\n> cases,\n> > > > to the AF algorithm in order restrict the AF search range to increase\n> > > > the speed of a scan. Do you think it's a valid use case and it's\n> worth a\n> > > > control ?)\n> > >\n> > > I'd imagined doing this kind of thing with the AfSpeed and AfRange\n> > > controls and by having the AF algorithm tuned for the module. I'm not\n> > > convinced about passing in lens positions from outside the camera\n> > > system. Where would they come from? Shouldn't the camera system know\n> > > what positions to use?\n> > >\n> >\n> > A possible use case, for controlled environments with low latency\n> > requirements, is to run tests beforehand and for each 'scene' define\n> > what is the ideal lens movement range. When the application starts an\n> > AF scan it would tell the AF algorithm about the 'preferred' range.\n> >\n> > I understand this works backwards, as the lens position should be the\n> > result of the AF algorithm run not one of its inputs :)\n>\n> Indeed, but as noted above, maybe there's an argument for an\n> AfRangeCustom which could be user-settable? We've added \"Custom\"\n> values for other controls in some places, IIRC.\n>\n> >\n> > > In the Raspberry Pi world we would of course store this kind of\n> > > information in the tuning file. I've noticed some discussion elsewhere\n> > > about supplying configuration to the IPAs - it seems like this might\n> > > be a case in point? I think that for real camera systems, in the end,\n> > > there is no escape from \"configuration\" (aka. \"tuning\").\n> > >\n> > > >\n> > > > > +\n> > > > > + - AfRange:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Control to set the range of focus distances that is\n> scanned. An\n> > > > > + implementation may choose not to implement all the\n> options here.\n> > > >\n> > > > Only valid in Auto and Continuous mode, right ?\n> > >\n> > > Generally I've been expecting that you could change settings like this\n> > > in Manual mode too, though it obviously can't have any effect until\n> > > you go back to Auto/Continuous. Does that sound reasonable?\n> >\n> > I would rather assume thaat if a control is not valid in the current\n> > context is simply ignored and not cached to be re-applied when the\n> > context change ?\n>\n> Possibly, though I'm slightly nervous about how you'd do things like:\n>\n> - You're in Manual mode, but want to go to Auto and set the Range to\n> Normal.\n> - You're in Auto mode but were using a different range. You want to\n> reset it to Normal before switching to Manual.\n>\n> Can you send the AfRange change and the new mode at the same time in\n> both cases or not? Or do you need two separate requests to send the\n> changes? It's not clear to me exactly what would work. If you say that\n> the AfRange always gets updated, then the behaviour is obvious, and\n> you can send both AfRange and AfMode in the same request which would\n> be easier to use.\n>\n> >\n> > >\n> > > >\n> > > > (This makes me think that we could easily model Continuous by adding\n> a\n> > > > TriggerContinuous when in Auto mode. With a TriggerPause we could\n> get rid of\n> > > > AfPause... well, it's quite a redesign so I'll stop here unless\n> > > > someone thinks it's a really good idea :)\n> > >\n> > > Actually I'm quite strongly against anything like this. I think Auto\n> > > and Continuous should be quite separate. The reason is that Auto is\n> > > (relatively) easy to implement, you do a scan over some range and\n> > > choose the best spot. But Continuous AF is an absolute nightmare to do\n> > > well. You have all these problems:\n> > >\n> > > - The algo doesn't realise it needs to rescan and so seems to be\n> > > \"stuck\". (Watch people waving their hands in front of the lens to try\n> > > and unstick it.) So you add ever more tests and arbitrary thresholds\n> > > on different statistics to tell if a rescan is needed, and then the\n> > > whole thing becomes untunable.\n> > > - The algo rescans too often. A car in the distance goes by and the\n> > > whole image see-saws in and out while a rescan happens, ending back at\n> > > the same place. Yet more tests, thresholds...\n> > > - You can never make all your tests and thresholds work at the same\n> > > time but in different conditions. So you make them variable depending\n> > > on things like exposure and gain. Now you are truly doomed.\n> > > - The algo is generally not robust to camera shake, which frequently\n> > > causes it to end up \"failing\" with a completely out-of-focus image.\n> > > And then you can't get it to rescan... cue more hand-waving in front\n> > > of the lens. You implement more special tests and timeouts... At this\n> > > point you will wish for the earth to swallow you up.\n> > > - When the algo rescans it starts off going the wrong way causing\n> > > extra see-sawing.\n> > > - You try to solve the last problem by doing really small initial\n> > > steps. Only then it ends up statistically unstable.\n> > >\n> > > From the above you may detect that we have been there and have several\n> > > T-shirts. Speaking for myself, I have no plans to implement Continuous\n> > > AF without PDAF sensors, as PDAF is quite a big help.\n> > >\n> >\n> > Yes, it feels like you know what you're talking about :)\n> >\n> > I understand you would like be cautious with CAF, but if one doesn't\n> > want to implement CAF can't she simply not report TriggerContinuous ?\n> >\n> > Anyway, that was just an idea thrown in the mixer, don't worry about\n> > it.\n> >\n> > > >\n> > > > > + enum:\n> > > > > + - name: AfRangeNormal\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + A wide range of focus distances is scanned, all the\n> way from\n> > > > > + infinity down to close distances, though depending on\n> the\n> > > > > + implementation, possibly not including the very\n> closest macro\n> > > > > + positions.\n> > > > > + - name: AfRangeMacro\n> > > > > + value: 1\n> > > > > + description: Only close distances are scanned.\n> > > > > + - name: AfRangeFull\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + The full range of focus distances is scanned just as\n> with\n> > > > > + AfRangeNormal but this time including the very\n> closest macro\n> > > > > + positions.\n> > > > > +\n> > > > > + - AfSpeed:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Control that determines whether the AF algorithm is to\n> move the lens\n> > > > > + as quickly as possible or more steadily. For example,\n> during video\n> > > > > + recording it may be desirable not to move the lens too\n> abruptly, but\n> > > > > + when in a preview mode (waiting for a still capture) it\n> may be\n> > > > > + helpful to move the lens as quickly as is reasonably\n> possible.\n> > > > > + enum:\n> > > > > + - name: AfSpeedNormal\n> > > > > + value: 0\n> > > > > + description: Move the lens at its usual speed.\n> > > > > + - name: AfSpeedFast\n> > > > > + value: 1\n> > > > > + description: Move the lens more quickly.\n> > > > > +\n> > > > > + - AfWindows:\n> > > > > + type: Rectangle\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Sets the focus windows used by the AF algorithm. The\n> units used express\n> > > > > + a proportion of the ScalerCrop control (or if\n> unavailable, of the entire\n> > > > > + image), as u0.16 format numbers.\n> > > > > +\n> > > > > + In order to be activated, a rectangle must be programmed\n> with non-zero\n> > > > > + width and height. If no rectangles are programmed in this\n> way, then the\n> > > > > + system will choose its own single default window in the\n> centre of the\n> > > > > + image.\n> > > > > +\n> > > > > + The details of how the windows are used are platform\n> dependent. We note\n> > > > > + that when there is more than one AF window, a typical\n> implementation\n> > > > > + might find the optimal focus position for each one and\n> finally select\n> > > > > + the window closest to the camera.\n> > > > > +\n> > > > > + size: [platform dependent]\n> > > >\n> > > > I think this should just be\n> > > >\n> > > > size: [n]\n> > >\n> > > OK!\n> > >\n> > > > > +\n> > > > > + - AfTrigger:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + This control starts an autofocus scan when AfMode is set\n> to AfModeAuto,\n> > > > > + and can also be used to terminate a scan early.\n> > > > > +\n> > > > > + It is ignored if AfMode is set to AfModeManual or\n> AfModeContinuous.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfTriggerStart\n> > > > > + value: 0\n> > > > > + description: Start an AF scan. Ignored if a scan is in\n> progress.\n> > > > > + - name: AfTriggerCancel\n> > > > > + value: 1\n> > > > > + description: Cancel an AF scan. This does not cause the\n> lens to move\n> > > > > + anywhere else. Ignored if no scan is in progress.\n> > > > > +\n> > > > > + - AfPause:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + This control has no effect except when in continuous\n> autofocus mode\n> > > > > + (AfModeContinuous). It can be used to pause any lens\n> movements while\n> > > > > + (for example) images are captured. The algorithm remains\n> inactive\n> > > > > + until it is instructed to resume.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfPauseImmediate\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + Pause the continuous autofocus algorithm immediately,\n> whether or\n> > > > > + not any kind of scan is underway. If the AfState was\n> previously\n> > > > > + reporting AfStateContinuousScanning it will now\n> change to\n> > > > > + AfStateContinuousScanningPaused, and similarly for\n> > > > > + AfStateContinuousFocused and AfStateContinuousFailed.\n> > > > > + - name AfPauseDeferred\n> > > > > + value: 1\n> > > > > + description: |\n> > > > > + Pause the continuous autofocus algorithm as soon as\n> it is no longer\n> > > > > + scanning. If the AfState is currently reporting\n> > > > > + AfStateContinuousFocused is will change to\n> > > > > + AfStateContinuousFocusedPaused, and similarly in the\n> case of\n> > > > > + AfStateContinuousFailed.\n> > > > > +\n> > > > > + If AfState reports AfStateContinuousScanning it will\n> change to\n> > > > > + AfStateContinuousScanningPausing, and then move to\n> one of\n> > > > > + AfStateContinuousFocusedPaused or\n> AfStateContinuousFailedPaused\n> > > > > + when the scan completes.\n> > > > > + - name: AfPauseResume\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + Resume continous autofocus operation. The algorithm\n> starts again\n> > > > > + from exactly where it left off, and the AfState will\n> drop the\n> > > > > + 'Paused' or 'Pausing' part of the state.\n> > > > > +\n> > > > > + - LensPosition:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Acts as a control to instruct the lens to move to a\n> particular position\n> > > > > + and also reports back the position of the lens for each\n> frame.\n> > > > > +\n> > > > > + The units are determined by the lens driver.\n> > > >\n> > > > Ugh, we cannot do this in the control definition. Driver\n> specificities\n> > > > cannot surface to the controls definition, which should be platform\n> > > > independent.\n> > > >\n> > > > I think we'll have to establish a numerical range (go ahead if you\n> > > > have any idea) and translate to the actual driver values in the\n> > > > CameraLens class ?\n> > >\n> > > I wasn't aware of this, but maybe it's a good thing. The question is\n> >\n> > Well, isn't it the same with \"Brightness\", for example ?\n>\n> Indeed, or ExposureTime where we explicitly want microseconds, not driver\n> units.\n>\n> >\n> > > what kind of range to use, and how do you know what limits apply for\n> > > any given camera module? How about a dioptre-like scale, normalised\n> > > for the hyperfocal distance H. It would go like this:\n> > >\n> > > The value n would mean being focused at distance H/n (n does not have\n> > > to be an integer). So:\n> > >\n> > > The value 1 would mean hyperfocal position (focal distance H/1).\n> > > The value 0 would mean infinity (focal distance H/0).\n> > > Larger values would get progressively closer.\n> > >\n> > > Setting a control value outside the allowed range will clamp it to\n> > > that end of the range.\n> >\n> > I sincerely don't know enough about this to tell if it's a good idea\n> > or not. Hence, it seems like a good idea to me :)\n> >\n> > >\n> > > In the Raspberry Pi world it would be sufficient for the tuning file\n> > > to know the allowable range of values. More generally, I could imagine\n> > > an application might want to say \"focus at precisely 1m\" in which case\n> > > it would have to know the value of H from a property or something,\n> > > though providing such information would be optional (some lenses\n> > > simply aren't precisely calibrated). Perhaps an application might want\n> > > to get the allowable range of (maybe approximate) focal distances too?\n> > >\n> > > >\n> > > > > +\n> > > > > + The LensPosition control is ignored unless the AfMode is\n> set to\n> > > > > + AfModeManual.\n> > > > > +\n> > > > > + Note that this control is distinct from AfLensPosition,\n> which allows\n> > > > > + the lens to be moved to its macro/hyperfocal/infinity\n> position, rather\n> > > > > + than using lens driver units.\n> > > > > +\n> > > > > + - AfState:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Reports the current state of the AF algorithm. The\n> possible state changes\n> > > > > + are described below, although we add that the following\n> general state\n> > > > > + changes are also allowed.\n> > > >\n> > > > wow, there is a lot of states here :)\n> > >\n> > > Did you follow the link to the state transition diagram? I would hope\n> > > that should go some way to explaining it! Here's the link again for\n> > > convenience:\n> > >\n> > >\n> https://docs.google.com/drawings/d/1rU2nS82dEC8Y2eY0lKPkQQhd67gKLOl7PvSeUcL1j_E/edit?usp=sharing\n> > >\n> >\n> > Missed it. Thanks for providing it.\n> >\n> > > >\n> > > > Let me copy them here\n> > > >\n> > > > - name: AfStateManual\n> > > > - name: AfStateAuto\n> > > > - name: AfStateAutoScanning\n> > > > - name: AfStateAutoFocused\n> > > > - name: AfStateAutoFailed\n> > > > - name: AfStateContinuousScanning\n> > > > - name: AfStateContinuousScanningPausing\n> > > > - name: AfStateContinuousScanningPaused\n> > > > - name: AfStateContinuousFocused\n> > > > - name: AfStateContinuousFocusedPaused\n> > > > - name: AfStateContinuousFailed\n> > > > - name: AfStateContinuousFailedPaused\n> > >\n> > > I \"multiplied out\" all the different parts of the state to make a\n> > > single long list. Actually I think that makes it more Android-like.\n> > > But we could reduce this number if, for example, we're happy to report\n> > > the state of AfMode separately on every frame. I don't mind - do we\n> > > have a general preference for this kind of thing?\n> > >\n> >\n> > Good question.\n> >\n> > More generally we need to establish a policy for metadata. For\n> > controls we know that controls should be attached to a request only\n> > when their value changes. If a control is not supplied with new\n> > values, then the old (or default) one will stay valid.\n> >\n> > This implies the PH/IPA state machine is then stateful and 'remembers'\n> > what was the last value a control was set to.\n> >\n> > What about metadata instead ? Can we assume the same stateful-ness in\n> > applications too, or is it a burden we cannot impose on them ?\n> >\n> > Han-lin, do you know how does that work with Android ? Does it report\n> > metadata even when their values do not change or do they assume that,\n> > like controls, a metadata is reported only when its value has changed.\n> >\n>\n\nI think Android metadata doesn't have the assumption, and is supposed to\nreturn with each frame.\nIn fact, with partial results supported, Android doesn't enforce a strict\norder of returning metadata among requests.\nIt's more like each metadata is independent of the previous one.\n\n> What would the preference be for the RPi applications ?\n>\n> I don't think I have a very strong preference. I'm happy with either:\n>\n> - Report everything in the AfState on every frame.\n> - Report the state in multiple metadata items (say, AfState and\n> AfMode) on every frame.\n>\n> I'm less keen on having applications have to remember if they sent\n> something and then wait for the request where it actually happened, it\n> seems to me that would simply add annoying boilerplate code to all\n> applications. What do you think?\n\n>\n> > > >\n> > > > (please note that AfStateAuto seems mis-aligned, unless I messed up\n> > > > something while cutting&paste. Please check the compiled doc to see\n> > > > how it looks like...)\n> > >\n> > > I will check.\n> > >\n> > > >\n> > > > So we have three macro groups\n> > > >\n> > > > - Manual\n> > > >\n> > > > - Auto\n> > > > - Scanning\n> > > > - Focus\n> > > > - Failed\n> > > >\n> > > > - Continuous\n> > > >\n> > > > - Scanning\n> > > > - Pausing\n> > > > - Paused\n> > > > - Focused\n> > > > - Paused\n> > > > - Failed\n> > > > - Paused\n> > > >\n> > > > The first group is AfMode (Manual, Auto, Continuous) and can be\n> > > > deduced by inspecting the AfMode metadata.\n> > > >\n> > > > The second group is the actual state (Scannng, Focused, Failed)\n> > > >\n> > > > The third group is specific to continuous mode as it accepts a Pause\n> > > > trigger.\n> > > >\n> > > > I understand why you have this, as expressing \"The AfState is set to\n> > > > $something when in AfModeManual\" etc etc might get cumbersome, but I\n> > > > think it might be worth a try\n> > > >\n> > > > 1) Manual mode (the bad)\n> > > > As we all know there is a delay between when a lens is instructed\n> > > > to move and when it reaches its final state.\n> > > >\n> > > > In Manual mode the lens is moved with LensPosition and\n> > > > AfLensPosition.\n> > > >\n> > > > To know the current state:\n> > > > - With LensPosition metadata can be inspected until it doesn't\n> reach the\n> > > > desired value.\n> > > > - With AfLensPosition it is not clear to me how to know when the\n> lens\n> > > > as reched one of the pre-defined positions.\n> > >\n> > > It sounds like we're deleting AfLensPosition.\n> > >\n> > > >\n> > > > Is it so wrong to report either (Scanning,Focus,Failed) even for\n> > > > Manual mode ?\n> > >\n> > > I don't think it's wrong. If we have a single long list of states then\n> > > it just makes for more states. If we don't then it's harmless, though\n> > > I'm not aware of a use for it.\n> > >\n> > > >\n> > > > 2) AutoMode (the good)\n> > > > This seems the easy one, as (Scanning,Focus,Failed) are already\n> > > > there\n> > > >\n> > > > 3) Continous (the ugly)\n> > > > Continuous mode can be paused. That's a shame :)\n> > > > Would that be that bad keeping a Paused or Pausing state which can\n> > > > only be returned for Continuous mode.\n> > >\n> > > The whole paused/pausing thing rather complicates things, but it does\n> > > seem to me that it's necessary. If you were in Continuous mode, you\n> > > would rather the lens didn't set off again just as you want to capture\n> > > a sequence of pictures.\n> > >\n> > > >\n> > > > All in all, can we reduce the number of states to\n> > > >\n> > > > - Scanning\n> > > >\n> > > > The AF algorithm is scanning and has not yet reached a\n> > > > stable result. When running in Auto mode an AF scan is\n> > > > started by using the AfTrigger control. When in Manual mode\n> > > > and a new lens position is requested either by sending a\n> > > > LensPosition or AfLensPosition control, the AF state will\n> > > > report \"Scanning\" until the lens doesn't reach the desired\n> > > > position. Similarly, when running in continuous mode and\n> the\n> > > > AF algorithm decides to start a scan spontaneously the\n> > > > AfState will report Scanning until a new stable state is\n> > > > reached.\n> > > >\n> > > > - Focus\n> > > >\n> > > > The AF algorithm has reached a stable state and the image\n> is\n> > > > now focused. When running in Manual mode the lens has\n> > > > reached the position requested using either LensPosition\n> and\n> > > > AfLensPosition.\n> > > >\n> > > > - Failed\n> > > >\n> > > > The AF algorithm has completed a scan and without finding a\n> > > > good focus position. When running in manual mode, the\n> camera\n> > > > has failed moving the lens to the desired position.\n> > > >\n> > > > - Paused (copying your below text, please see comments below)\n> > > >\n> > > > The AF algorithm is in continuous mode (AfModeContinuous)\n> and\n> > > > was scanning when AfPauseImmediate was sent. The scan will\n> not\n> > > > complete and the algorithm will remain in this state.\n> > > >\n> > > > - Pausing\n> > > >\n> > > > The AF algorithm is in continuous mode (AfModeContinuous)\n> and\n> > > > was scanning when AfPauseDeferred was sent. The scan will\n> complete\n> > > > at which point the algorithm moves to the\n> > > > AfStateContinuousFocusedPaused or\n> AfStateContinuousFailedPaused\n> > > > state.\n> > >\n> > > I think we could get by if we report the following for every frame:\n> > >\n> > > AfMode - Manual, Auto or Continuous\n> > > AfState - Scanning, Focused or Failed\n> > > AfPause - Resume (which means it's running), Immediate or Deferred\n> > > (these states only apply during Continuous; the Deferred value can\n> > > only occur when also Scanning).\n> > >\n> > > Or we could wrap AfState and AfPause together:\n> > >\n> > > AfState - Scanning, Focused. Failed, ScanningPausing, ScanningPaused,\n> > > FocusedPaused, FailedPaused.\n> > >\n> > > That makes some of the interaction between pausing and scanning\n> > > clearer, but then it's more states again. I don't know...\n> > >\n> >\n> > Let's clarify what the policy on metadata is first.\n>\n> Agree!\n>\n> >\n> > > >\n> > > >\n> > > > > +\n> > > > > + In any of the AfStateManual or AfStateContinuous states,\n> the AfMode\n> > > > > + may be set to AfModeAuto and the algorithm will move to\n> the\n> > > > > + AfStateAuto state. If AfTriggerStart is sent at the same\n> time\n> > > > > + then the algorithm will move to AfStateAutoScanning state.\n> > > > > +\n> > > > > + In any of the AfStateAuto or AfStateManual states, the\n> AfMode may\n> > > > > + be set to AfModeContinuous and the algorithm will move to\n> > > > > + AfStateContinuousScanning.\n> > > > > +\n> > > > > + In any of the AfStateAuto or AfStateContinuous states,\n> the AfMode may\n> > > > > + be set to AfModeManual and the algorithm will move to\n> > > > > + AfStateManual. The lens will not be moved and will be\n> left where\n> > > > > + it was at that moment.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfStateManual\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + The AF algorithm is in manual mode (AfModeManual).\n> The LensPosition\n> > > > > + and AfLensPosition controls can be used directly to\n> move the lens.\n> > > > > + - name: AfStateAuto\n> > > > > + value: 1\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and\n> has either just\n> > > > > + been moved into that state, or a scan that was in\n> progress has been\n> > > > > + cancelled.\n> > > > > + - name: AfStateAutoScanning\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and a\n> scan has been\n> > > > > + started using the AfTrigger control. The scan can be\n> cancelled by\n> > > > > + sending AfTriggerCancel at which point the algorithm\n> will either\n> > > > > + move back to AfStateAuto or, if the scan actually\n> completes\n> > > > > + before the cancel request is processed, to one of\n> > > > > + AfStateAutoFocused or AfStateAutoFailed.\n> > > > > + - name: AfStateAutoFocused\n> > > > > + value: 3\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and a\n> scan has\n> > > > > + completed with the result that the algorithm believes\n> the image is\n> > > > > + now in focus.\n> > > > > + - name: AfStateAutoFailed\n> > > > > + value: 4\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and a\n> scan has\n> > > > > + completed with the result that the algorithm did not\n> find a good\n> > > > > + focus position.\n> > > > > + - name: AfStateContinuousScanning\n> > > > > + value: 5\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode\n> (AfModeContinuous) and\n> > > > > + is currently scanning for a good focus position. This\n> occurs when\n> > > > > + the mode is first set to AfModeContinuous, or may\n> happen\n> > > > > + spontaneously when the algorithm believes a re-scan\n> is needed.\n> > > > > + - name: AfStateContinuousScanningPausing\n> > > > > + value: 6\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode\n> (AfModeContinuous) and\n> > > > > + was scanning when AfPauseDeferred was sent. The scan\n> will complete\n> > > > > + at which point the algorithm moves to the\n> > > > > + AfStateContinuousFocusedPaused or\n> AfStateContinuousFailedPaused\n> > > > > + state.\n> > > > > + - name: AfStateContinuousScanningPaused\n> > > > > + value: 7\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode\n> (AfModeContinuous) and\n> > > > > + was scanning when AfPauseImmediate was sent. The scan\n> will not\n> > > >\n> > > > Why can't the algorithm be paused when it has reached a stable state\n> > > > but only when scanning ? Won't a pause 'freeze' the algorithm in its\n> > > > current state no matter what it is ? I understand with\n> > > > \"AfPauseDeferred\" that it has to wait for AF to be locked before\n> > > > stopping it, but with \"Immediate\" doesn't \"pause\" apply to all\n> states ?\n> > >\n> > > Wasn't sure I understood this. It can be paused once it's reached\n> > > Focused or Failed, and then both PauseImmediate and PauseDeferred will\n> > > cause it to report FocusedPaused or FailedPaused. Was that the\n> > > question?\n> >\n> > You're right, I have probably missed FocusedPaused and FailedPaused.\n> > :)\n> >\n> > Thanks\n> > j\n>\n> Thanks for all the discussion!\n>\n> David\n>\n> >\n> > >\n> > > Anyway, thanks very much!\n> > >\n> > > David\n> > >\n> > > >\n> > > > Thanks\n> > > > j\n> > > >\n> > > > > + complete and the algorithm will remain in this state.\n> The scan\n> > > > > + may be resumed by sending AfPauseResume.\n> > > > > + - name: AfStateContinuousFocused\n> > > > > + value: 8\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode\n> (AfModeContinuous) and\n> > > > > + a scan has completed with the algorithm believing it\n> has found a\n> > > > > + good focus position.\n> > > > > + - name: AfStateContinuousFocusedPaused\n> > > > > + value: 9\n> > > > > + description: |\n> > > > > + The AF algorithm was in the AfStateContinuousFocused\n> state and\n> > > > > + has been paused (by either AfPauseImmediate or\n> AfPauseDeferred),\n> > > > > + or it was in the AfStateContinuousScanningPausing\n> state and the\n> > > > > + scan has completed successfully. The algorithm will\n> now remain\n> > > > > + in this state, and may be resumed by sending\n> AfPauseResume.\n> > > > > + - name: AfStateContinuousFailed\n> > > > > + value: 10\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode\n> (AfModeContinuous) and\n> > > > > + a scan has completed without finding a good focus\n> position.\n> > > > > + - name: AfStateContinuousFailedPaused\n> > > > > + value: 11\n> > > > > + description: |\n> > > > > + The AF algorithm was in the AfStateContinuousFailed\n> state and\n> > > > > + has been paused (by either AfPauseImmediate or\n> AfPauseDeferred),\n> > > > > + or it was in the AfStateContinuousScanningPausing\n> state and the\n> > > > > + scan has completed unsuccessfully. The algorithm will\n> now remain\n> > > > > + in this state, and may be resumed by sending\n> AfPauseResume.\n> > > > > +\n> > > > > ...\n> > > > > --\n> > > > > 2.30.2\n> > > > >\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0F536C3256\n\tfor ;\n\tThu, 7 Apr 2022 11:31:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6E8E865644;\n\tThu, 7 Apr 2022 13:31:23 +0200 (CEST)","from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com\n\t[IPv6:2607:f8b0:4864:20::22f])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AADE46563F\n\tfor ;\n\tThu, 7 Apr 2022 13:31:20 +0200 (CEST)","by mail-oi1-x22f.google.com with SMTP id q189so5281193oia.9\n\tfor ;\n\tThu, 07 Apr 2022 04:31:20 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649331083;\n\tbh=k3hmFbY68oF62yu5sY7KCrIz4L/LBNh1E1THToEsIOI=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=er824KB80n7NsNka5pfb2pT4MItXNrSCnfmcn/gnHA+e4GecaWXrjpD5j1bKejH/O\n\tKRec2qjdKUVR++SA00bF//IZgDVlUvgdJjgXHM5mIoVY/tQ3+abU7DxEb+nCq1bhil\n\tT/idRimO+oTHPMrJQfPobQwJl1pS0FV2ql9m/J9T4MTncp5g2d3ugxmqAoAxkN5BSA\n\t9m5SsiF7mzSHfpnkoSd9gU4Gk0CE5WfIq12FXk2URzQsRDBdqUZiOIwdKKyt0ABYBu\n\tGIW59A4RGfbgi+TLSkAZ0Nze52H7Mh/dpXyiLl7CN1CvErfKpI3AMmEYAz9yBLBGuv\n\tz0zuYCR4yo48g==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=YgHQdkKMRpJ33HlCzH0J9+cD0an+qIwyT8C+CuO45Kg=;\n\tb=hUPW9Sc2uGL6LhfaEXz+5Uiqa/405pZuQ+OcOPsoCw7RB9aVLN9RsFj3wZca/0G2Kx\n\tmUXagmzYT0kR3hGFVf2mUliu3bXlD6zfDIiSvaa8tZ8Ckl+YqmbcxloNZnTCZ02B3Q9A\n\trjKFfSXnxetyo4KNUmXRSXtGSu+o48YqImTj0="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"hUPW9Sc2\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=YgHQdkKMRpJ33HlCzH0J9+cD0an+qIwyT8C+CuO45Kg=;\n\tb=Z26xFwO+QB4Bs1shzan3rK5ULnmIS/98Kl4zOTCrFrSCf1hMvq0kx8jlgUoa8kUHYf\n\tABGVtOvWDpiD2kMwPycETcWxPvkraXqSY8iJv7ByEbqfmPXJxJ9JgMX1Dhub366JpPJ8\n\tj/1MU7izDBwUKkhEyAxx+3njJqm+TS9VcKWlK39ymnCtjZk9LSBwS+5GAZeJTyA6CfVE\n\tVLoP+In9XJ3NL8Mu1Eq+FfFKwQ4HUDWpnpclno3KCpxTDB/QuEsQo/+B+zN2kvpaG2Gj\n\t9qcw9uOsa2+PvdCkgV7Xo5nZ+UiBesnGjRz9CVpuEAYGLeRNjVCbVL6CT+DybkJE8nXl\n\tJ2Gg==","X-Gm-Message-State":"AOAM533475d/f4oytV7M7tfVAvAi4UDRLO7nxosRe16uDyzznblcpJPU\n\tBo7pGp8joD4CF9fBV6p+7TgsLM6UAtlf/lFD9GJazA==","X-Google-Smtp-Source":"ABdhPJyVzdXBYyRa8DvB2hBftSXsaP2+IsOjdxBVCaU0t01XHBOzMuhzWyZ8EGhghmmYiXxPEYlfhy0N+Xx9R2J1giw=","X-Received":"by 2002:a05:6808:1246:b0:2c9:efa5:7209 with SMTP id\n\to6-20020a056808124600b002c9efa57209mr5603830oiv.62.1649331078675;\n\tThu, 07 Apr 2022 04:31:18 -0700 (PDT)","MIME-Version":"1.0","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t<20220405203610.peioc4aomqgtxb5w@uno.localdomain>\n\t\n\t<20220406163358.hhvc7vr5jzx6ml56@uno.localdomain>\n\t","In-Reply-To":"","Date":"Thu, 7 Apr 2022 19:31:07 +0800","Message-ID":"","To":"David Plowman ","Content-Type":"multipart/alternative; boundary=\"0000000000000fd67e05dc0ed1e8\"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Hanlin Chen via libcamera-devel ","Reply-To":"Hanlin Chen ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22709,"web_url":"https://patchwork.libcamera.org/comment/22709/","msgid":"<20220414094428.xdao3vfbnpbggbsl@uno.localdomain>","date":"2022-04-14T09:44:28","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi David,\n\nOn Thu, Apr 07, 2022 at 10:05:32AM +0100, David Plowman wrote:\n> Hi Jacopo\n>\n> Thanks for the reply!\n>\n> On Wed, 6 Apr 2022 at 17:34, Jacopo Mondi wrote:\n> >\n> > Hi David,\n> > + Han-lin for an Android question on metadata\n> >\n> > On Wed, Apr 06, 2022 at 02:05:20PM +0100, David Plowman wrote:\n> > > Hi Jacopo\n> > >\n> > > Thanks for taking the time to read all this!\n> > >\n> > > On Tue, 5 Apr 2022 at 21:36, Jacopo Mondi wrote:\n> > > >\n> > > > Hi David,\n> > > > sorry for being late :)\n> > > >\n> > > > On Thu, Mar 31, 2022 at 04:17:47PM +0100, David Plowman via libcamera-devel wrote:\n> > > > > This patch describes a series of controls that allow applications to\n> > > > > drive AF algorithms:\n> > > > >\n> > > > > AfMode - manual, auto or continuous\n> > > > > AfLensPosition - set lens position to macro/hyperfocal/infinity\n> > > > > AfRange - full, macro or normal\n> > > > > AfSpeed - fast or slow\n> > > > > AfWindows - AF window locations\n> > > > > AfTrigger - start (trigger an AF scan) or cancel\n> > > > > AfPause - pause continuous AF\n> > > > > LensPosition - position of lens from lens driver\n> > > > > AfState - reports the mode, whether scanning/success/failure\n> > > > > ---\n> > > > > src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> > > > > 1 file changed, 313 insertions(+), 60 deletions(-)\n> > > > >\n> > > > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > > > index 9d4638ae..23607368 100644\n> > > > > --- a/src/libcamera/control_ids.yaml\n> > > > > +++ b/src/libcamera/control_ids.yaml\n> > > > > @@ -406,27 +406,6 @@ controls:\n> > > > > The camera will cancel any active or completed metering sequence.\n> > > > > The AE algorithm is reset to its initial state.\n> > > > >\n> > > > > - - AfTrigger:\n> > > > > - type: int32_t\n> > > > > - draft: true\n> > > > > - description: |\n> > > > > - Control for AF trigger. Currently identical to\n> > > > > - ANDROID_CONTROL_AF_TRIGGER.\n> > > > > -\n> > > > > - Whether the camera device will trigger autofocus for this request.\n> > > > > - enum:\n> > > > > - - name: AfTriggerIdle\n> > > > > - value: 0\n> > > > > - description: The trigger is idle.\n> > > > > - - name: AfTriggerStart\n> > > > > - value: 1\n> > > > > - description: The AF routine is started by the camera.\n> > > > > - - name: AfTriggerCancel\n> > > > > - value: 2\n> > > > > - description: |\n> > > > > - The camera will cancel any active trigger and the AF routine is\n> > > > > - reset to its initial state.\n> > > > > -\n> > > > > - NoiseReductionMode:\n> > > > > type: int32_t\n> > > > > draft: true\n> > > > > @@ -507,45 +486,6 @@ controls:\n> > > > > The AE algorithm has started a pre-capture metering session.\n> > > > > \\sa AePrecaptureTrigger\n> > > > >\n> > > > > - - AfState:\n> > > > > - type: int32_t\n> > > > > - draft: true\n> > > > > - description: |\n> > > > > - Control to report the current AF algorithm state. Currently identical to\n> > > > > - ANDROID_CONTROL_AF_STATE.\n> > > > > -\n> > > > > - Current state of the AF algorithm.\n> > > > > - enum:\n> > > > > - - name: AfStateInactive\n> > > > > - value: 0\n> > > > > - description: The AF algorithm is inactive.\n> > > > > - - name: AfStatePassiveScan\n> > > > > - value: 1\n> > > > > - description: |\n> > > > > - AF is performing a passive scan of the scene in continuous\n> > > > > - auto-focus mode.\n> > > > > - - name: AfStatePassiveFocused\n> > > > > - value: 2\n> > > > > - description: |\n> > > > > - AF believes the scene is in focus, but might restart scanning.\n> > > > > - - name: AfStateActiveScan\n> > > > > - value: 3\n> > > > > - description: |\n> > > > > - AF is performing a scan triggered by an AF trigger request.\n> > > > > - \\sa AfTrigger\n> > > > > - - name: AfStateFocusedLock\n> > > > > - value: 4\n> > > > > - description: |\n> > > > > - AF believes has focused correctly and has locked focus.\n> > > > > - - name: AfStateNotFocusedLock\n> > > > > - value: 5\n> > > > > - description: |\n> > > > > - AF has not been able to focus and has locked.\n> > > > > - - name: AfStatePassiveUnfocused\n> > > > > - value: 6\n> > > > > - description: |\n> > > > > - AF has completed a passive scan without finding focus.\n> > > > > -\n> > > > > - AwbState:\n> > > > > type: int32_t\n> > > > > draft: true\n> > > > > @@ -690,4 +630,317 @@ controls:\n> > > > > value. All of the custom test patterns will be static (that is the\n> > > > > raw image must not vary from frame to frame).\n> > > > >\n> > > > > + - AfMode:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > >\n> > > > Was having all these controls as draft intentional ? The previous\n> > > > definitions where set to drafts as they mapped to the android\n> > > > definition, but now that we're defining our own ones...\n> > >\n> > > Not really... I only made them \"draft\" because it seemed like\n> > > everything was \"draft\" at first! But happy to change that. Are you\n> > > saying that these definitely should *not* be draft, then?\n> > >\n> >\n> > Yeah, \"draft\" was used to import controls from Android as they are for\n> > sake of CTS compliance. Let's drop draft from the ones we're defining\n>\n> Will do!\n>\n> >\n> > > >\n> > > > > + description: |\n> > > > > + Control to set the mode of the AF (autofocus) algorithm. Applications\n> > > > > + are allowed to set a new mode, and to send additional controls for\n> > > > > + that new mode, in the same request.\n> > > > > +\n> > > > > + An implementation may choose not to implement all the modes.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfModeManual\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + The AF algorithm is in manual mode. In this mode it will never\n> > > > > + perform any action nor move the lens of its own accord, but an\n> > > > > + application can set controls to move the lens.\n> > > > > +\n> > > > > + In this mode the AfState will always report AfStateManual.\n> > > > > + - name: AfModeAuto\n> > > > > + value: 1\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode. This means that the algorithm\n> > > > > + will never move the lens or change state unless the AfTrigger\n> > > > > + control is used. The AfTrigger control can be used to initiate a\n> > > > > + focus scan, the results of which will also be reported by AfState.\n> > > > > +\n> > > > > + If the autofocus algorithm is moved from AfModeAuto to another\n> > > > > + mode while a scan is in progress, the scan is cancelled\n> > > > > + immediately, without waiting for the scan to finish.\n> > > > > +\n> > > > > + When first entering this mode the AfState will report\n> > > > > + AfStateAuto. When a trigger control is sent, AfState will\n> > > > > + report AfStateAutoScanning for a period before spontaneously\n> > > > > + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> > > > > + the outcome of the scan. It will remain in this state until\n> > > > > + another scan is initiated by the AfTrigger control. If a scan is\n> > > > > + cancelled (without changing to another mode), AfState will return\n> > > > > + to AfStateAuto.\n> > > > > + - name: AfModeContinuous\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode. This means that the lens\n> > > > > + can re-start a scan spontaneously at any moment, without any user\n> > > > > + intervention. The AfState still reports whether the algorithm is\n> > > > > + currently scanning or not, though the application has no ability\n> > > > > + to initiate or cancel scans (though it can \"pause\" them), nor to\n> > > > > + move the lens for itself.\n> > > > > +\n> > > > > + When set to AfModeContinuous, the system will immediately initiate\n> > > > > + a scan so AfState will report AfStateContinuousScanning, and will\n> > > > > + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> > > > > + depending on the scan result.\n> > > > > +\n> > > > > + - AfLensPosition:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Control to set the position of the lens to one of the following\n> > > > > + predefined locations. This control only has any effect when the AfMode\n> > > > > + is set to AfModeManual.\n> > > > > +\n> > > > > + This control is distinct from the LensPosition control, which sets the\n> > > > > + lens position using the lens driver's units.\n> > > > > + enum:\n> > > > > + - name: AfLensPositionMacro\n> > > > > + value: 0\n> > > > > + description: The closest focal position of the lens.\n> > > > > + - name: AfLensPositionHyperfocal\n> > > > > + value: 1\n> > > > > + description: Hyperfocal position.\n> > > > > + - name: AfLensPositionInfinity\n> > > > > + value: 2\n> > > > > + description: The furthest focal position (usually infinity).\n> > > >\n> > > > Interesting, bikeshedding on the name apart, is this some 'simplified'\n> > > > LensState ? Do you think there is a use case for this ? Why would\n> > > > anyone set the AF to auto mode then use some coarse grained control\n> > > > like this one ? Wouldn't it make more sense as an 'hint' to the AF\n> > > > algorithm to restrict the search space ?\n> > >\n> > > In the light of your comments below about the LensPosition not using\n> > > driver units, then maybe this could be deleted, and we could select\n> > > some kind of canonical range for the lens?\n> > >\n> >\n> > Ah wait. What I was suggesting (\"An \"hint\" to the AF algorithm to\n> > restrict the search space\") is basically AfRange.\n>\n> I wonder if there's a case for a \"custom\" value for AfRange, and we\n> could then even have a control that sets the custom range?\n>\n\nLet's start with fixed positions then eventually extend it ?\n\n> >\n> > > The most basic use case would be following on from the capture of an\n> > > image after an AF scan:\n> > >\n> > > - capture image\n> > > - AfMode to manual\n> > > - Lens to hyperfocal\n> > > - AfMode to auto\n> > > - now wait for the shutter button to be pressed before you start\n> > > another AF scan.\n> >\n> > Naive question:\n> > Wouldn't setting the lens to hyperfocal render the preview unusable ?\n>\n> Hyperfocal is the place where \"most stuff\" is normally in focus\n> (strictly speaking, has the greatest depth of field), so in the\n> absence of other information it's normally the best place to put the\n> lens. It's just for the preview, you'd typically run another scan for\n> a capture.\n>\n\nI see, thank for the clrification\n\n> (Some systems run Continuous AF during preview, of course. But you\n> know my anxiety about CAF algorithms... in my opinion just putting the\n> lens to hyperfocal is way less annoying than a not-very-good CAF\n> implementation!)\n>\n> >\n> > >\n> > > If your UI has a \"macro\" capture mode of some kind, maybe it's \"Lens\n> > > to macro\" instead of \"Lens to hyperfocal\".\n> >\n> > Why is this different than starting an AF scan (without going through\n> > Manual) with an AfRange that suggests in which range to scan ? Is this\n> > an optimization to forcefully move the lens closer to where it will\n> > likely end being moved to ?\n>\n> If you know you're doing a macro capture it would be helpful for the\n> preview to put the lens somewhere where macro things are likely to be\n> more in focus, instead of using hyperfocal which will likely make\n> close objects very blurry.\n>\n> >\n> > >\n> > > But as noted, if LensPosition allows us to select hyperfocal/macro\n> > > directly, then we can do without this one.\n> > >\n> >\n> > Providing my above understanding is correct, it would be a bit less intuitive\n> > for applications to use a numerical value instead of some easier\n> > \"Macro\"/\"Hyperfocal\". Of course, wrappers like your one can simplify this\n> > for their consumers.\n>\n> True, though I'm now quite liking the \"dioptre-like\" canonical range\n> described below, where 1 = hyperfocal, 0 = infinity, large numbers =\n> close\n>\n> >\n> > > >\n> > > > (Btw I have a use case for supplying a range of lens positions, which\n> > > > has been previously estimated according to some pre-defined use cases,\n> > > > to the AF algorithm in order restrict the AF search range to increase\n> > > > the speed of a scan. Do you think it's a valid use case and it's worth a\n> > > > control ?)\n> > >\n> > > I'd imagined doing this kind of thing with the AfSpeed and AfRange\n> > > controls and by having the AF algorithm tuned for the module. I'm not\n> > > convinced about passing in lens positions from outside the camera\n> > > system. Where would they come from? Shouldn't the camera system know\n> > > what positions to use?\n> > >\n> >\n> > A possible use case, for controlled environments with low latency\n> > requirements, is to run tests beforehand and for each 'scene' define\n> > what is the ideal lens movement range. When the application starts an\n> > AF scan it would tell the AF algorithm about the 'preferred' range.\n> >\n> > I understand this works backwards, as the lens position should be the\n> > result of the AF algorithm run not one of its inputs :)\n>\n> Indeed, but as noted above, maybe there's an argument for an\n> AfRangeCustom which could be user-settable? We've added \"Custom\"\n> values for other controls in some places, IIRC.\n>\n> >\n> > > In the Raspberry Pi world we would of course store this kind of\n> > > information in the tuning file. I've noticed some discussion elsewhere\n> > > about supplying configuration to the IPAs - it seems like this might\n> > > be a case in point? I think that for real camera systems, in the end,\n> > > there is no escape from \"configuration\" (aka. \"tuning\").\n> > >\n> > > >\n> > > > > +\n> > > > > + - AfRange:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Control to set the range of focus distances that is scanned. An\n> > > > > + implementation may choose not to implement all the options here.\n> > > >\n> > > > Only valid in Auto and Continuous mode, right ?\n> > >\n> > > Generally I've been expecting that you could change settings like this\n> > > in Manual mode too, though it obviously can't have any effect until\n> > > you go back to Auto/Continuous. Does that sound reasonable?\n> >\n> > I would rather assume thaat if a control is not valid in the current\n> > context is simply ignored and not cached to be re-applied when the\n> > context change ?\n>\n> Possibly, though I'm slightly nervous about how you'd do things like:\n>\n> - You're in Manual mode, but want to go to Auto and set the Range to Normal.\n> - You're in Auto mode but were using a different range. You want to\n> reset it to Normal before switching to Manual.\n>\n> Can you send the AfRange change and the new mode at the same time in\n> both cases or not? Or do you need two separate requests to send the\n> changes? It's not clear to me exactly what would work. If you say that\n> the AfRange always gets updated, then the behaviour is obvious, and\n> you can send both AfRange and AfMode in the same request which would\n> be easier to use.\n>\n\nI don't see why both controls cannot be specified together.\n\nEven if we allow AfRange to be specified in Manual mode and cache it\nto apply it when switching to Auto, AfRange will only be applied when\nAfMode==Auto is received, hence I don't see any optimization in\nspecifying it when in Manual mode ?\n\n> >\n> > >\n> > > >\n> > > > (This makes me think that we could easily model Continuous by adding a\n> > > > TriggerContinuous when in Auto mode. With a TriggerPause we could get rid of\n> > > > AfPause... well, it's quite a redesign so I'll stop here unless\n> > > > someone thinks it's a really good idea :)\n> > >\n> > > Actually I'm quite strongly against anything like this. I think Auto\n> > > and Continuous should be quite separate. The reason is that Auto is\n> > > (relatively) easy to implement, you do a scan over some range and\n> > > choose the best spot. But Continuous AF is an absolute nightmare to do\n> > > well. You have all these problems:\n> > >\n> > > - The algo doesn't realise it needs to rescan and so seems to be\n> > > \"stuck\". (Watch people waving their hands in front of the lens to try\n> > > and unstick it.) So you add ever more tests and arbitrary thresholds\n> > > on different statistics to tell if a rescan is needed, and then the\n> > > whole thing becomes untunable.\n> > > - The algo rescans too often. A car in the distance goes by and the\n> > > whole image see-saws in and out while a rescan happens, ending back at\n> > > the same place. Yet more tests, thresholds...\n> > > - You can never make all your tests and thresholds work at the same\n> > > time but in different conditions. So you make them variable depending\n> > > on things like exposure and gain. Now you are truly doomed.\n> > > - The algo is generally not robust to camera shake, which frequently\n> > > causes it to end up \"failing\" with a completely out-of-focus image.\n> > > And then you can't get it to rescan... cue more hand-waving in front\n> > > of the lens. You implement more special tests and timeouts... At this\n> > > point you will wish for the earth to swallow you up.\n> > > - When the algo rescans it starts off going the wrong way causing\n> > > extra see-sawing.\n> > > - You try to solve the last problem by doing really small initial\n> > > steps. Only then it ends up statistically unstable.\n> > >\n> > > From the above you may detect that we have been there and have several\n> > > T-shirts. Speaking for myself, I have no plans to implement Continuous\n> > > AF without PDAF sensors, as PDAF is quite a big help.\n> > >\n> >\n> > Yes, it feels like you know what you're talking about :)\n> >\n> > I understand you would like be cautious with CAF, but if one doesn't\n> > want to implement CAF can't she simply not report TriggerContinuous ?\n> >\n> > Anyway, that was just an idea thrown in the mixer, don't worry about\n> > it.\n> >\n> > > >\n> > > > > + enum:\n> > > > > + - name: AfRangeNormal\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + A wide range of focus distances is scanned, all the way from\n> > > > > + infinity down to close distances, though depending on the\n> > > > > + implementation, possibly not including the very closest macro\n> > > > > + positions.\n> > > > > + - name: AfRangeMacro\n> > > > > + value: 1\n> > > > > + description: Only close distances are scanned.\n> > > > > + - name: AfRangeFull\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + The full range of focus distances is scanned just as with\n> > > > > + AfRangeNormal but this time including the very closest macro\n> > > > > + positions.\n> > > > > +\n> > > > > + - AfSpeed:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Control that determines whether the AF algorithm is to move the lens\n> > > > > + as quickly as possible or more steadily. For example, during video\n> > > > > + recording it may be desirable not to move the lens too abruptly, but\n> > > > > + when in a preview mode (waiting for a still capture) it may be\n> > > > > + helpful to move the lens as quickly as is reasonably possible.\n> > > > > + enum:\n> > > > > + - name: AfSpeedNormal\n> > > > > + value: 0\n> > > > > + description: Move the lens at its usual speed.\n> > > > > + - name: AfSpeedFast\n> > > > > + value: 1\n> > > > > + description: Move the lens more quickly.\n> > > > > +\n> > > > > + - AfWindows:\n> > > > > + type: Rectangle\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Sets the focus windows used by the AF algorithm. The units used express\n> > > > > + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> > > > > + image), as u0.16 format numbers.\n> > > > > +\n> > > > > + In order to be activated, a rectangle must be programmed with non-zero\n> > > > > + width and height. If no rectangles are programmed in this way, then the\n> > > > > + system will choose its own single default window in the centre of the\n> > > > > + image.\n> > > > > +\n> > > > > + The details of how the windows are used are platform dependent. We note\n> > > > > + that when there is more than one AF window, a typical implementation\n> > > > > + might find the optimal focus position for each one and finally select\n> > > > > + the window closest to the camera.\n> > > > > +\n> > > > > + size: [platform dependent]\n> > > >\n> > > > I think this should just be\n> > > >\n> > > > size: [n]\n> > >\n> > > OK!\n> > >\n> > > > > +\n> > > > > + - AfTrigger:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> > > > > + and can also be used to terminate a scan early.\n> > > > > +\n> > > > > + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfTriggerStart\n> > > > > + value: 0\n> > > > > + description: Start an AF scan. Ignored if a scan is in progress.\n> > > > > + - name: AfTriggerCancel\n> > > > > + value: 1\n> > > > > + description: Cancel an AF scan. This does not cause the lens to move\n> > > > > + anywhere else. Ignored if no scan is in progress.\n> > > > > +\n> > > > > + - AfPause:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + This control has no effect except when in continuous autofocus mode\n> > > > > + (AfModeContinuous). It can be used to pause any lens movements while\n> > > > > + (for example) images are captured. The algorithm remains inactive\n> > > > > + until it is instructed to resume.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfPauseImmediate\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + Pause the continuous autofocus algorithm immediately, whether or\n> > > > > + not any kind of scan is underway. If the AfState was previously\n> > > > > + reporting AfStateContinuousScanning it will now change to\n> > > > > + AfStateContinuousScanningPaused, and similarly for\n> > > > > + AfStateContinuousFocused and AfStateContinuousFailed.\n> > > > > + - name AfPauseDeferred\n> > > > > + value: 1\n> > > > > + description: |\n> > > > > + Pause the continuous autofocus algorithm as soon as it is no longer\n> > > > > + scanning. If the AfState is currently reporting\n> > > > > + AfStateContinuousFocused is will change to\n> > > > > + AfStateContinuousFocusedPaused, and similarly in the case of\n> > > > > + AfStateContinuousFailed.\n> > > > > +\n> > > > > + If AfState reports AfStateContinuousScanning it will change to\n> > > > > + AfStateContinuousScanningPausing, and then move to one of\n> > > > > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > > > + when the scan completes.\n> > > > > + - name: AfPauseResume\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + Resume continous autofocus operation. The algorithm starts again\n> > > > > + from exactly where it left off, and the AfState will drop the\n> > > > > + 'Paused' or 'Pausing' part of the state.\n> > > > > +\n> > > > > + - LensPosition:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Acts as a control to instruct the lens to move to a particular position\n> > > > > + and also reports back the position of the lens for each frame.\n> > > > > +\n> > > > > + The units are determined by the lens driver.\n> > > >\n> > > > Ugh, we cannot do this in the control definition. Driver specificities\n> > > > cannot surface to the controls definition, which should be platform\n> > > > independent.\n> > > >\n> > > > I think we'll have to establish a numerical range (go ahead if you\n> > > > have any idea) and translate to the actual driver values in the\n> > > > CameraLens class ?\n> > >\n> > > I wasn't aware of this, but maybe it's a good thing. The question is\n> >\n> > Well, isn't it the same with \"Brightness\", for example ?\n>\n> Indeed, or ExposureTime where we explicitly want microseconds, not driver units.\n>\n> >\n> > > what kind of range to use, and how do you know what limits apply for\n> > > any given camera module? How about a dioptre-like scale, normalised\n> > > for the hyperfocal distance H. It would go like this:\n> > >\n> > > The value n would mean being focused at distance H/n (n does not have\n> > > to be an integer). So:\n> > >\n> > > The value 1 would mean hyperfocal position (focal distance H/1).\n> > > The value 0 would mean infinity (focal distance H/0).\n> > > Larger values would get progressively closer.\n> > >\n> > > Setting a control value outside the allowed range will clamp it to\n> > > that end of the range.\n> >\n> > I sincerely don't know enough about this to tell if it's a good idea\n> > or not. Hence, it seems like a good idea to me :)\n> >\n> > >\n> > > In the Raspberry Pi world it would be sufficient for the tuning file\n> > > to know the allowable range of values. More generally, I could imagine\n> > > an application might want to say \"focus at precisely 1m\" in which case\n> > > it would have to know the value of H from a property or something,\n> > > though providing such information would be optional (some lenses\n> > > simply aren't precisely calibrated). Perhaps an application might want\n> > > to get the allowable range of (maybe approximate) focal distances too?\n> > >\n\nSo this is a lens construction parameter that would need to be\nrecorded in the tuning file. In your experience, is this parameter\neasily available or can it be easily estimated ?\n\n> > > >\n> > > > > +\n> > > > > + The LensPosition control is ignored unless the AfMode is set to\n> > > > > + AfModeManual.\n> > > > > +\n> > > > > + Note that this control is distinct from AfLensPosition, which allows\n> > > > > + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> > > > > + than using lens driver units.\n> > > > > +\n> > > > > + - AfState:\n> > > > > + type: int32_t\n> > > > > + draft: true\n> > > > > + description: |\n> > > > > + Reports the current state of the AF algorithm. The possible state changes\n> > > > > + are described below, although we add that the following general state\n> > > > > + changes are also allowed.\n> > > >\n> > > > wow, there is a lot of states here :)\n> > >\n> > > Did you follow the link to the state transition diagram? I would hope\n> > > that should go some way to explaining it! Here's the link again for\n> > > convenience:\n> > >\n> > > https://docs.google.com/drawings/d/1rU2nS82dEC8Y2eY0lKPkQQhd67gKLOl7PvSeUcL1j_E/edit?usp=sharing\n> > >\n> >\n> > Missed it. Thanks for providing it.\n> >\n> > > >\n> > > > Let me copy them here\n> > > >\n> > > > - name: AfStateManual\n> > > > - name: AfStateAuto\n> > > > - name: AfStateAutoScanning\n> > > > - name: AfStateAutoFocused\n> > > > - name: AfStateAutoFailed\n> > > > - name: AfStateContinuousScanning\n> > > > - name: AfStateContinuousScanningPausing\n> > > > - name: AfStateContinuousScanningPaused\n> > > > - name: AfStateContinuousFocused\n> > > > - name: AfStateContinuousFocusedPaused\n> > > > - name: AfStateContinuousFailed\n> > > > - name: AfStateContinuousFailedPaused\n> > >\n> > > I \"multiplied out\" all the different parts of the state to make a\n> > > single long list. Actually I think that makes it more Android-like.\n> > > But we could reduce this number if, for example, we're happy to report\n> > > the state of AfMode separately on every frame. I don't mind - do we\n> > > have a general preference for this kind of thing?\n> > >\n> >\n> > Good question.\n> >\n> > More generally we need to establish a policy for metadata. For\n> > controls we know that controls should be attached to a request only\n> > when their value changes. If a control is not supplied with new\n> > values, then the old (or default) one will stay valid.\n> >\n> > This implies the PH/IPA state machine is then stateful and 'remembers'\n> > what was the last value a control was set to.\n> >\n> > What about metadata instead ? Can we assume the same stateful-ness in\n> > applications too, or is it a burden we cannot impose on them ?\n> >\n> > Han-lin, do you know how does that work with Android ? Does it report\n> > metadata even when their values do not change or do they assume that,\n> > like controls, a metadata is reported only when its value has changed.\n> >\n> > What would the preference be for the RPi applications ?\n>\n> I don't think I have a very strong preference. I'm happy with either:\n>\n> - Report everything in the AfState on every frame.\n> - Report the state in multiple metadata items (say, AfState and\n> AfMode) on every frame.\n\nAccording to Han-Lin reply, also Android seems to require to have\nmetadata in each completed request, hence we could have AfMode\nreported separately and AfState simplified.\n\nThe only drawback is that to reconstruct the actual state (mode +\nstate) application have to inspect two metadata instead of one, but I\nthink it's acceptable and simplifies things on the IPA side.\n\n>\n> I'm less keen on having applications have to remember if they sent\n> something and then wait for the request where it actually happened, it\n> seems to me that would simply add annoying boilerplate code to all\n> applications. What do you think?\n>\n\nWell, that's how it generally works. If you set your algorithm to auto\nor manual mode, you have to wait for a metadata that tells you when\nthe mode has actually been changed, being it reported through a\nseparate AfMode metadata or through a more expressive AfState which\nincorporates the mode.\n\n> >\n> > > >\n> > > > (please note that AfStateAuto seems mis-aligned, unless I messed up\n> > > > something while cutting&paste. Please check the compiled doc to see\n> > > > how it looks like...)\n> > >\n> > > I will check.\n> > >\n> > > >\n> > > > So we have three macro groups\n> > > >\n> > > > - Manual\n> > > >\n> > > > - Auto\n> > > > - Scanning\n> > > > - Focus\n> > > > - Failed\n> > > >\n> > > > - Continuous\n> > > >\n> > > > - Scanning\n> > > > - Pausing\n> > > > - Paused\n> > > > - Focused\n> > > > - Paused\n> > > > - Failed\n> > > > - Paused\n> > > >\n> > > > The first group is AfMode (Manual, Auto, Continuous) and can be\n> > > > deduced by inspecting the AfMode metadata.\n> > > >\n> > > > The second group is the actual state (Scannng, Focused, Failed)\n> > > >\n> > > > The third group is specific to continuous mode as it accepts a Pause\n> > > > trigger.\n> > > >\n> > > > I understand why you have this, as expressing \"The AfState is set to\n> > > > $something when in AfModeManual\" etc etc might get cumbersome, but I\n> > > > think it might be worth a try\n> > > >\n> > > > 1) Manual mode (the bad)\n> > > > As we all know there is a delay between when a lens is instructed\n> > > > to move and when it reaches its final state.\n> > > >\n> > > > In Manual mode the lens is moved with LensPosition and\n> > > > AfLensPosition.\n> > > >\n> > > > To know the current state:\n> > > > - With LensPosition metadata can be inspected until it doesn't reach the\n> > > > desired value.\n> > > > - With AfLensPosition it is not clear to me how to know when the lens\n> > > > as reched one of the pre-defined positions.\n> > >\n> > > It sounds like we're deleting AfLensPosition.\n> > >\n> > > >\n> > > > Is it so wrong to report either (Scanning,Focus,Failed) even for\n> > > > Manual mode ?\n> > >\n> > > I don't think it's wrong. If we have a single long list of states then\n> > > it just makes for more states. If we don't then it's harmless, though\n> > > I'm not aware of a use for it.\n> > >\n> > > >\n> > > > 2) AutoMode (the good)\n> > > > This seems the easy one, as (Scanning,Focus,Failed) are already\n> > > > there\n> > > >\n> > > > 3) Continous (the ugly)\n> > > > Continuous mode can be paused. That's a shame :)\n> > > > Would that be that bad keeping a Paused or Pausing state which can\n> > > > only be returned for Continuous mode.\n> > >\n> > > The whole paused/pausing thing rather complicates things, but it does\n> > > seem to me that it's necessary. If you were in Continuous mode, you\n> > > would rather the lens didn't set off again just as you want to capture\n> > > a sequence of pictures.\n> > >\n> > > >\n> > > > All in all, can we reduce the number of states to\n> > > >\n> > > > - Scanning\n> > > >\n> > > > The AF algorithm is scanning and has not yet reached a\n> > > > stable result. When running in Auto mode an AF scan is\n> > > > started by using the AfTrigger control. When in Manual mode\n> > > > and a new lens position is requested either by sending a\n> > > > LensPosition or AfLensPosition control, the AF state will\n> > > > report \"Scanning\" until the lens doesn't reach the desired\n> > > > position. Similarly, when running in continuous mode and the\n> > > > AF algorithm decides to start a scan spontaneously the\n> > > > AfState will report Scanning until a new stable state is\n> > > > reached.\n> > > >\n> > > > - Focus\n> > > >\n> > > > The AF algorithm has reached a stable state and the image is\n> > > > now focused. When running in Manual mode the lens has\n> > > > reached the position requested using either LensPosition and\n> > > > AfLensPosition.\n> > > >\n> > > > - Failed\n> > > >\n> > > > The AF algorithm has completed a scan and without finding a\n> > > > good focus position. When running in manual mode, the camera\n> > > > has failed moving the lens to the desired position.\n> > > >\n> > > > - Paused (copying your below text, please see comments below)\n> > > >\n> > > > The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > was scanning when AfPauseImmediate was sent. The scan will not\n> > > > complete and the algorithm will remain in this state.\n> > > >\n> > > > - Pausing\n> > > >\n> > > > The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > was scanning when AfPauseDeferred was sent. The scan will complete\n> > > > at which point the algorithm moves to the\n> > > > AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > > state.\n> > >\n> > > I think we could get by if we report the following for every frame:\n> > >\n> > > AfMode - Manual, Auto or Continuous\n> > > AfState - Scanning, Focused or Failed\n> > > AfPause - Resume (which means it's running), Immediate or Deferred\n> > > (these states only apply during Continuous; the Deferred value can\n> > > only occur when also Scanning).\n> > >\n> > > Or we could wrap AfState and AfPause together:\n> > >\n> > > AfState - Scanning, Focused. Failed, ScanningPausing, ScanningPaused,\n> > > FocusedPaused, FailedPaused.\n> > >\n> > > That makes some of the interaction between pausing and scanning\n> > > clearer, but then it's more states again. I don't know...\n> > >\n> >\n> > Let's clarify what the policy on metadata is first.\n>\n> Agree!\n\n\nMy preference would be to have three metadata, although it would\nrequire a bit more work on the application side and maybe also on the\nIPA side as correctness of the metadata combinations should be\nguaranteed (ie AfPause::Deferred can only be returned when\nAfState::Scanning according to your above comment). But it would\nsimplify the states greatly and it's worth it in my opinion.\n\nIt's a gut feeling, so if anyone feels there are reasons not to go\nthis way feel free to express it.\n\nThanks\n j\n\n>\n> >\n> > > >\n> > > >\n> > > > > +\n> > > > > + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> > > > > + may be set to AfModeAuto and the algorithm will move to the\n> > > > > + AfStateAuto state. If AfTriggerStart is sent at the same time\n> > > > > + then the algorithm will move to AfStateAutoScanning state.\n> > > > > +\n> > > > > + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> > > > > + be set to AfModeContinuous and the algorithm will move to\n> > > > > + AfStateContinuousScanning.\n> > > > > +\n> > > > > + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> > > > > + be set to AfModeManual and the algorithm will move to\n> > > > > + AfStateManual. The lens will not be moved and will be left where\n> > > > > + it was at that moment.\n> > > > > +\n> > > > > + enum:\n> > > > > + - name: AfStateManual\n> > > > > + value: 0\n> > > > > + description: |\n> > > > > + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> > > > > + and AfLensPosition controls can be used directly to move the lens.\n> > > > > + - name: AfStateAuto\n> > > > > + value: 1\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> > > > > + been moved into that state, or a scan that was in progress has been\n> > > > > + cancelled.\n> > > > > + - name: AfStateAutoScanning\n> > > > > + value: 2\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> > > > > + started using the AfTrigger control. The scan can be cancelled by\n> > > > > + sending AfTriggerCancel at which point the algorithm will either\n> > > > > + move back to AfStateAuto or, if the scan actually completes\n> > > > > + before the cancel request is processed, to one of\n> > > > > + AfStateAutoFocused or AfStateAutoFailed.\n> > > > > + - name: AfStateAutoFocused\n> > > > > + value: 3\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > > > > + completed with the result that the algorithm believes the image is\n> > > > > + now in focus.\n> > > > > + - name: AfStateAutoFailed\n> > > > > + value: 4\n> > > > > + description: |\n> > > > > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > > > > + completed with the result that the algorithm did not find a good\n> > > > > + focus position.\n> > > > > + - name: AfStateContinuousScanning\n> > > > > + value: 5\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > > + is currently scanning for a good focus position. This occurs when\n> > > > > + the mode is first set to AfModeContinuous, or may happen\n> > > > > + spontaneously when the algorithm believes a re-scan is needed.\n> > > > > + - name: AfStateContinuousScanningPausing\n> > > > > + value: 6\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > > + was scanning when AfPauseDeferred was sent. The scan will complete\n> > > > > + at which point the algorithm moves to the\n> > > > > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > > > > + state.\n> > > > > + - name: AfStateContinuousScanningPaused\n> > > > > + value: 7\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > > + was scanning when AfPauseImmediate was sent. The scan will not\n> > > >\n> > > > Why can't the algorithm be paused when it has reached a stable state\n> > > > but only when scanning ? Won't a pause 'freeze' the algorithm in its\n> > > > current state no matter what it is ? I understand with\n> > > > \"AfPauseDeferred\" that it has to wait for AF to be locked before\n> > > > stopping it, but with \"Immediate\" doesn't \"pause\" apply to all states ?\n> > >\n> > > Wasn't sure I understood this. It can be paused once it's reached\n> > > Focused or Failed, and then both PauseImmediate and PauseDeferred will\n> > > cause it to report FocusedPaused or FailedPaused. Was that the\n> > > question?\n> >\n> > You're right, I have probably missed FocusedPaused and FailedPaused.\n> > :)\n> >\n> > Thanks\n> > j\n>\n> Thanks for all the discussion!\n>\n> David\n>\n> >\n> > >\n> > > Anyway, thanks very much!\n> > >\n> > > David\n> > >\n> > > >\n> > > > Thanks\n> > > > j\n> > > >\n> > > > > + complete and the algorithm will remain in this state. The scan\n> > > > > + may be resumed by sending AfPauseResume.\n> > > > > + - name: AfStateContinuousFocused\n> > > > > + value: 8\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > > + a scan has completed with the algorithm believing it has found a\n> > > > > + good focus position.\n> > > > > + - name: AfStateContinuousFocusedPaused\n> > > > > + value: 9\n> > > > > + description: |\n> > > > > + The AF algorithm was in the AfStateContinuousFocused state and\n> > > > > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > > > > + or it was in the AfStateContinuousScanningPausing state and the\n> > > > > + scan has completed successfully. The algorithm will now remain\n> > > > > + in this state, and may be resumed by sending AfPauseResume.\n> > > > > + - name: AfStateContinuousFailed\n> > > > > + value: 10\n> > > > > + description: |\n> > > > > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > > > > + a scan has completed without finding a good focus position.\n> > > > > + - name: AfStateContinuousFailedPaused\n> > > > > + value: 11\n> > > > > + description: |\n> > > > > + The AF algorithm was in the AfStateContinuousFailed state and\n> > > > > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > > > > + or it was in the AfStateContinuousScanningPausing state and the\n> > > > > + scan has completed unsuccessfully. The algorithm will now remain\n> > > > > + in this state, and may be resumed by sending AfPauseResume.\n> > > > > +\n> > > > > ...\n> > > > > --\n> > > > > 2.30.2\n> > > > >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E7B2EC3256\n\tfor ;\n\tThu, 14 Apr 2022 09:44:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 319E065655;\n\tThu, 14 Apr 2022 11:44:36 +0200 (CEST)","from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net\n\t[217.70.183.193])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A2A96564E\n\tfor ;\n\tThu, 14 Apr 2022 11:44:35 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id BFD1D240010;\n\tThu, 14 Apr 2022 09:44:30 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649929476;\n\tbh=kNEX4tH3s8m5iKPyPhBKv0+uLYKhc+7jeImZ3T5MWmg=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=vJkVQnfSJHxptBo1hIPJ0iLIgmN2zrDdf93OoANU1jvUiIT8auFTNAgptj089Ydcr\n\tbzIh6hIqrZMekwoJuW0F1wuhvAwHG949gEXWlzROW3187DbF+lLJrHfcjnJbatNvEp\n\toe9UcIh+TfSAj1vVlSv+/wOM0u9y/ZKTW567PX0b/HDjoqkuwQtMge8jAxr9nlvmVu\n\t3mz7ULJsHkI7s2sEmfXtfUnG2BtZ8NGZEKhCHXIcu7RFimdzTM1ViLVN7b8tLixGVM\n\tXtbvnLO0vHk/zdIsDbiJUmehyDtBaIEYY/C6Iov1uMzQuTYux1b+PMAMDMjR3rK7DY\n\t+x+S6Wzat9BLg==","Date":"Thu, 14 Apr 2022 11:44:28 +0200","To":"David Plowman ","Message-ID":"<20220414094428.xdao3vfbnpbggbsl@uno.localdomain>","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t<20220405203610.peioc4aomqgtxb5w@uno.localdomain>\n\t\n\t<20220406163358.hhvc7vr5jzx6ml56@uno.localdomain>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22721,"web_url":"https://patchwork.libcamera.org/comment/22721/","msgid":"","date":"2022-04-15T08:21:57","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hello David,\n\nNow that I'm starting to play a bit with it, I see a few things we might \nwant to improve.\n\nOn 31/03/2022 17:17, David Plowman wrote:\n> This patch describes a series of controls that allow applications to\n> drive AF algorithms:\n> \n> AfMode - manual, auto or continuous\n> AfLensPosition - set lens position to macro/hyperfocal/infinity\n> AfRange - full, macro or normal\n> AfSpeed - fast or slow\n> AfWindows - AF window locations\n> AfTrigger - start (trigger an AF scan) or cancel\n> AfPause - pause continuous AF\n> LensPosition - position of lens from lens driver\n> AfState - reports the mode, whether scanning/success/failure\n> ---\n> src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> 1 file changed, 313 insertions(+), 60 deletions(-)\n> \n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index 9d4638ae..23607368 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -406,27 +406,6 @@ controls:\n> The camera will cancel any active or completed metering sequence.\n> The AE algorithm is reset to its initial state.\n> \n> - - AfTrigger:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control for AF trigger. Currently identical to\n> - ANDROID_CONTROL_AF_TRIGGER.\n> -\n> - Whether the camera device will trigger autofocus for this request.\n> - enum:\n> - - name: AfTriggerIdle\n> - value: 0\n> - description: The trigger is idle.\n> - - name: AfTriggerStart\n> - value: 1\n> - description: The AF routine is started by the camera.\n> - - name: AfTriggerCancel\n> - value: 2\n> - description: |\n> - The camera will cancel any active trigger and the AF routine is\n> - reset to its initial state.\n> -\n> - NoiseReductionMode:\n> type: int32_t\n> draft: true\n> @@ -507,45 +486,6 @@ controls:\n> The AE algorithm has started a pre-capture metering session.\n> \\sa AePrecaptureTrigger\n> \n> - - AfState:\n> - type: int32_t\n> - draft: true\n> - description: |\n> - Control to report the current AF algorithm state. Currently identical to\n> - ANDROID_CONTROL_AF_STATE.\n> -\n> - Current state of the AF algorithm.\n> - enum:\n> - - name: AfStateInactive\n> - value: 0\n> - description: The AF algorithm is inactive.\n> - - name: AfStatePassiveScan\n> - value: 1\n> - description: |\n> - AF is performing a passive scan of the scene in continuous\n> - auto-focus mode.\n> - - name: AfStatePassiveFocused\n> - value: 2\n> - description: |\n> - AF believes the scene is in focus, but might restart scanning.\n> - - name: AfStateActiveScan\n> - value: 3\n> - description: |\n> - AF is performing a scan triggered by an AF trigger request.\n> - \\sa AfTrigger\n> - - name: AfStateFocusedLock\n> - value: 4\n> - description: |\n> - AF believes has focused correctly and has locked focus.\n> - - name: AfStateNotFocusedLock\n> - value: 5\n> - description: |\n> - AF has not been able to focus and has locked.\n> - - name: AfStatePassiveUnfocused\n> - value: 6\n> - description: |\n> - AF has completed a passive scan without finding focus.\n> -\n> - AwbState:\n> type: int32_t\n> draft: true\n> @@ -690,4 +630,317 @@ controls:\n> value. All of the custom test patterns will be static (that is the\n> raw image must not vary from frame to frame).\n> \n> + - AfMode:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the mode of the AF (autofocus) algorithm. Applications\n> + are allowed to set a new mode, and to send additional controls for\n> + that new mode, in the same request.\n> +\n> + An implementation may choose not to implement all the modes.\n> +\n> + enum:\n> + - name: AfModeManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode. In this mode it will never\n> + perform any action nor move the lens of its own accord, but an\n> + application can set controls to move the lens.\n> +\n> + In this mode the AfState will always report AfStateManual.\n> + - name: AfModeAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode. This means that the algorithm\n> + will never move the lens or change state unless the AfTrigger\n> + control is used. The AfTrigger control can be used to initiate a\n> + focus scan, the results of which will also be reported by AfState.\n> +\n> + If the autofocus algorithm is moved from AfModeAuto to another\n> + mode while a scan is in progress, the scan is cancelled\n> + immediately, without waiting for the scan to finish.\n> +\n> + When first entering this mode the AfState will report\n> + AfStateAuto. When a trigger control is sent, AfState will\n> + report AfStateAutoScanning for a period before spontaneously\n> + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> + the outcome of the scan. It will remain in this state until\n> + another scan is initiated by the AfTrigger control. If a scan is\n> + cancelled (without changing to another mode), AfState will return\n> + to AfStateAuto.\n> + - name: AfModeContinuous\n> + value: 2\n> + description: |\n> + The AF algorithm is in continuous mode. This means that the lens\n> + can re-start a scan spontaneously at any moment, without any user\n> + intervention. The AfState still reports whether the algorithm is\n> + currently scanning or not, though the application has no ability\n> + to initiate or cancel scans (though it can \"pause\" them), nor to\n> + move the lens for itself.\n> +\n> + When set to AfModeContinuous, the system will immediately initiate\n> + a scan so AfState will report AfStateContinuousScanning, and will\n> + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> + depending on the scan result.\n> +\n> + - AfLensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the position of the lens to one of the following\n> + predefined locations. This control only has any effect when the AfMode\n> + is set to AfModeManual.\n> +\n> + This control is distinct from the LensPosition control, which sets the\n> + lens position using the lens driver's units.\n> + enum:\n> + - name: AfLensPositionMacro\n> + value: 0\n> + description: The closest focal position of the lens.\n> + - name: AfLensPositionHyperfocal\n> + value: 1\n> + description: Hyperfocal position.\n> + - name: AfLensPositionInfinity\n> + value: 2\n> + description: The furthest focal position (usually infinity).\n> +\n> + - AfRange:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control to set the range of focus distances that is scanned. An\n> + implementation may choose not to implement all the options here.\n> + enum:\n> + - name: AfRangeNormal\n> + value: 0\n> + description: |\n> + A wide range of focus distances is scanned, all the way from\n> + infinity down to close distances, though depending on the\n> + implementation, possibly not including the very closest macro\n> + positions.\n> + - name: AfRangeMacro\n> + value: 1\n> + description: Only close distances are scanned.\n> + - name: AfRangeFull\n> + value: 2\n> + description: |\n> + The full range of focus distances is scanned just as with\n> + AfRangeNormal but this time including the very closest macro\n> + positions.\n> +\n> + - AfSpeed:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Control that determines whether the AF algorithm is to move the lens\n> + as quickly as possible or more steadily. For example, during video\n> + recording it may be desirable not to move the lens too abruptly, but\n> + when in a preview mode (waiting for a still capture) it may be\n> + helpful to move the lens as quickly as is reasonably possible.\n> + enum:\n> + - name: AfSpeedNormal\n> + value: 0\n> + description: Move the lens at its usual speed.\n> + - name: AfSpeedFast\n> + value: 1\n> + description: Move the lens more quickly.\n> +\n> + - AfWindows:\n> + type: Rectangle\n> + draft: true\n> + description: |\n> + Sets the focus windows used by the AF algorithm. The units used express\n> + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> + image), as u0.16 format numbers.\n\nI don't mind using ScalerCrop, but I think it would probably make more \nsense to use the full pixel array ?\n\n> +\n> + In order to be activated, a rectangle must be programmed with non-zero\n> + width and height. If no rectangles are programmed in this way, then the\n> + system will choose its own single default window in the centre of the\n> + image.\n> +\n> + The details of how the windows are used are platform dependent. We note\n> + that when there is more than one AF window, a typical implementation\n> + might find the optimal focus position for each one and finally select\n> + the window closest to the camera.\n> +\n> + size: [platform dependent]\n\nAnd I would like to point you to this one (you probably already now it):\nhttps://developer.android.com/reference/android/hardware/Camera.Parameters#getFocusAreas()\n\nI really like the number of areas variable, and even more the weight per \narea. It makes sense, you might want to have more weight in the center, \nbut still have a larger region.\n\nI don't get why the corrdinates are between [-1000, 1000] though but... \nnot that important.\n\nWhat do you think ?\nJM\n\n> +\n> + - AfTrigger:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> + and can also be used to terminate a scan early.\n> +\n> + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> +\n> + enum:\n> + - name: AfTriggerStart\n> + value: 0\n> + description: Start an AF scan. Ignored if a scan is in progress.\n> + - name: AfTriggerCancel\n> + value: 1\n> + description: Cancel an AF scan. This does not cause the lens to move\n> + anywhere else. Ignored if no scan is in progress.\n> +\n> + - AfPause:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + This control has no effect except when in continuous autofocus mode\n> + (AfModeContinuous). It can be used to pause any lens movements while\n> + (for example) images are captured. The algorithm remains inactive\n> + until it is instructed to resume.\n> +\n> + enum:\n> + - name: AfPauseImmediate\n> + value: 0\n> + description: |\n> + Pause the continuous autofocus algorithm immediately, whether or\n> + not any kind of scan is underway. If the AfState was previously\n> + reporting AfStateContinuousScanning it will now change to\n> + AfStateContinuousScanningPaused, and similarly for\n> + AfStateContinuousFocused and AfStateContinuousFailed.\n> + - name AfPauseDeferred\n> + value: 1\n> + description: |\n> + Pause the continuous autofocus algorithm as soon as it is no longer\n> + scanning. If the AfState is currently reporting\n> + AfStateContinuousFocused is will change to\n> + AfStateContinuousFocusedPaused, and similarly in the case of\n> + AfStateContinuousFailed.\n> +\n> + If AfState reports AfStateContinuousScanning it will change to\n> + AfStateContinuousScanningPausing, and then move to one of\n> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> + when the scan completes.\n> + - name: AfPauseResume\n> + value: 2\n> + description: |\n> + Resume continous autofocus operation. The algorithm starts again\n> + from exactly where it left off, and the AfState will drop the\n> + 'Paused' or 'Pausing' part of the state.\n> +\n> + - LensPosition:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Acts as a control to instruct the lens to move to a particular position\n> + and also reports back the position of the lens for each frame.\n> +\n> + The units are determined by the lens driver.\n> +\n> + The LensPosition control is ignored unless the AfMode is set to\n> + AfModeManual.\n> +\n> + Note that this control is distinct from AfLensPosition, which allows\n> + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> + than using lens driver units.\n> +\n> + - AfState:\n> + type: int32_t\n> + draft: true\n> + description: |\n> + Reports the current state of the AF algorithm. The possible state changes\n> + are described below, although we add that the following general state\n> + changes are also allowed.\n> +\n> + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> + may be set to AfModeAuto and the algorithm will move to the\n> + AfStateAuto state. If AfTriggerStart is sent at the same time\n> + then the algorithm will move to AfStateAutoScanning state.\n> +\n> + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> + be set to AfModeContinuous and the algorithm will move to\n> + AfStateContinuousScanning.\n> +\n> + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> + be set to AfModeManual and the algorithm will move to\n> + AfStateManual. The lens will not be moved and will be left where\n> + it was at that moment.\n> +\n> + enum:\n> + - name: AfStateManual\n> + value: 0\n> + description: |\n> + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> + and AfLensPosition controls can be used directly to move the lens.\n> + - name: AfStateAuto\n> + value: 1\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> + been moved into that state, or a scan that was in progress has been\n> + cancelled.\n> + - name: AfStateAutoScanning\n> + value: 2\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> + started using the AfTrigger control. The scan can be cancelled by\n> + sending AfTriggerCancel at which point the algorithm will either\n> + move back to AfStateAuto or, if the scan actually completes\n> + before the cancel request is processed, to one of\n> + AfStateAutoFocused or AfStateAutoFailed.\n> + - name: AfStateAutoFocused\n> + value: 3\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm believes the image is\n> + now in focus.\n> + - name: AfStateAutoFailed\n> + value: 4\n> + description: |\n> + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> + completed with the result that the algorithm did not find a good\n> + focus position.\n> + - name: AfStateContinuousScanning\n> + value: 5\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + is currently scanning for a good focus position. This occurs when\n> + the mode is first set to AfModeContinuous, or may happen\n> + spontaneously when the algorithm believes a re-scan is needed.\n> + - name: AfStateContinuousScanningPausing\n> + value: 6\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseDeferred was sent. The scan will complete\n> + at which point the algorithm moves to the\n> + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> + state.\n> + - name: AfStateContinuousScanningPaused\n> + value: 7\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + was scanning when AfPauseImmediate was sent. The scan will not\n> + complete and the algorithm will remain in this state. The scan\n> + may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFocused\n> + value: 8\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed with the algorithm believing it has found a\n> + good focus position.\n> + - name: AfStateContinuousFocusedPaused\n> + value: 9\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFocused state and\n> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and the\n> + scan has completed successfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> + - name: AfStateContinuousFailed\n> + value: 10\n> + description: |\n> + The AF algorithm is in continuous mode (AfModeContinuous) and\n> + a scan has completed without finding a good focus position.\n> + - name: AfStateContinuousFailedPaused\n> + value: 11\n> + description: |\n> + The AF algorithm was in the AfStateContinuousFailed state and\n> + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> + or it was in the AfStateContinuousScanningPausing state and the\n> + scan has completed unsuccessfully. The algorithm will now remain\n> + in this state, and may be resumed by sending AfPauseResume.\n> +\n> ...","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9A75DC3256\n\tfor ;\n\tFri, 15 Apr 2022 08:22:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C8B3B65646;\n\tFri, 15 Apr 2022 10:22:01 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C374660207\n\tfor ;\n\tFri, 15 Apr 2022 10:22:00 +0200 (CEST)","from [IPV6:2a01:e0a:169:7140:db74:a7b2:c693:ed56] (unknown\n\t[IPv6:2a01:e0a:169:7140:db74:a7b2:c693:ed56])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4266E482;\n\tFri, 15 Apr 2022 10:22:00 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1650010921;\n\tbh=PIdSxse7Gxv4CcPT1KfhIPyq7smInKvKtJhQFBCovfc=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=wC4jQ9D6H/GkKPb2YpHQhlY45PUhYIAfD07bxYPx0iTqTE0WlvfQR6D9IjflIaQ4t\n\tVaoDX/S+v6PnV/QHsuHoQ0+hSceC0bR8xYb9JJQQ+PRYP4esFi5VS24eADsdaavTp1\n\tiaKjwHJ0YxRffdVe0k/uyew2FWVeLSVEYyMLYjohmqZGKwJIwfT+DkSvdkOanR182Q\n\t+vTk0keyw676dAm9/pwuiu8sTFKHLzFcUINa0f7KMJSkAR/59fMYX1bCmyssijMjsW\n\tmfn/PAnKJAs4NN/VXPa22hbM4yc2J52bCJ2o5ydWom97P+CIBChVLerjBH+Tq9j02Z\n\t/BvJcFRnydhlA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1650010920;\n\tbh=PIdSxse7Gxv4CcPT1KfhIPyq7smInKvKtJhQFBCovfc=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=vZWjkOou3ERJFtUGnlxEV22UbUnRmLpt5kscVtwOk0O0sWkDDQkC7NDsBuwJAzDOs\n\tEz5giuUzBnsiIue9DPXHbPt6W5rrZ6NaTELk5p/5D7NGa7OrzdS0+TGtxtFv02/jdP\n\tsLh3ish9WEiw3ADcLT0dTOFEFfXNcepe1R22KxXM="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"vZWjkOou\"; dkim-atps=neutral","Message-ID":"","Date":"Fri, 15 Apr 2022 10:21:57 +0200","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.7.0","Content-Language":"en-US","To":"David Plowman ,\n\tlibcamera-devel@lists.libcamera.org, hanlinchen@chromium.org,\n\thpa@redhat.com","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>","In-Reply-To":"<20220331151747.19458-2-david.plowman@raspberrypi.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jean-Michel Hautbois via libcamera-devel\n\t","Reply-To":"Jean-Michel Hautbois ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":22758,"web_url":"https://patchwork.libcamera.org/comment/22758/","msgid":"","date":"2022-04-20T13:02:10","subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi Jean-Michel\n\nThanks for raising these points, I think there are some things to\nfigure out here!\n\nOn Fri, 15 Apr 2022 at 09:22, Jean-Michel Hautbois\n wrote:\n>\n> Hello David,\n>\n> Now that I'm starting to play a bit with it, I see a few things we might\n> want to improve.\n>\n> On 31/03/2022 17:17, David Plowman wrote:\n> > This patch describes a series of controls that allow applications to\n> > drive AF algorithms:\n> >\n> > AfMode - manual, auto or continuous\n> > AfLensPosition - set lens position to macro/hyperfocal/infinity\n> > AfRange - full, macro or normal\n> > AfSpeed - fast or slow\n> > AfWindows - AF window locations\n> > AfTrigger - start (trigger an AF scan) or cancel\n> > AfPause - pause continuous AF\n> > LensPosition - position of lens from lens driver\n> > AfState - reports the mode, whether scanning/success/failure\n> > ---\n> > src/libcamera/control_ids.yaml | 373 +++++++++++++++++++++++++++------\n> > 1 file changed, 313 insertions(+), 60 deletions(-)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > index 9d4638ae..23607368 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -406,27 +406,6 @@ controls:\n> > The camera will cancel any active or completed metering sequence.\n> > The AE algorithm is reset to its initial state.\n> >\n> > - - AfTrigger:\n> > - type: int32_t\n> > - draft: true\n> > - description: |\n> > - Control for AF trigger. Currently identical to\n> > - ANDROID_CONTROL_AF_TRIGGER.\n> > -\n> > - Whether the camera device will trigger autofocus for this request.\n> > - enum:\n> > - - name: AfTriggerIdle\n> > - value: 0\n> > - description: The trigger is idle.\n> > - - name: AfTriggerStart\n> > - value: 1\n> > - description: The AF routine is started by the camera.\n> > - - name: AfTriggerCancel\n> > - value: 2\n> > - description: |\n> > - The camera will cancel any active trigger and the AF routine is\n> > - reset to its initial state.\n> > -\n> > - NoiseReductionMode:\n> > type: int32_t\n> > draft: true\n> > @@ -507,45 +486,6 @@ controls:\n> > The AE algorithm has started a pre-capture metering session.\n> > \\sa AePrecaptureTrigger\n> >\n> > - - AfState:\n> > - type: int32_t\n> > - draft: true\n> > - description: |\n> > - Control to report the current AF algorithm state. Currently identical to\n> > - ANDROID_CONTROL_AF_STATE.\n> > -\n> > - Current state of the AF algorithm.\n> > - enum:\n> > - - name: AfStateInactive\n> > - value: 0\n> > - description: The AF algorithm is inactive.\n> > - - name: AfStatePassiveScan\n> > - value: 1\n> > - description: |\n> > - AF is performing a passive scan of the scene in continuous\n> > - auto-focus mode.\n> > - - name: AfStatePassiveFocused\n> > - value: 2\n> > - description: |\n> > - AF believes the scene is in focus, but might restart scanning.\n> > - - name: AfStateActiveScan\n> > - value: 3\n> > - description: |\n> > - AF is performing a scan triggered by an AF trigger request.\n> > - \\sa AfTrigger\n> > - - name: AfStateFocusedLock\n> > - value: 4\n> > - description: |\n> > - AF believes has focused correctly and has locked focus.\n> > - - name: AfStateNotFocusedLock\n> > - value: 5\n> > - description: |\n> > - AF has not been able to focus and has locked.\n> > - - name: AfStatePassiveUnfocused\n> > - value: 6\n> > - description: |\n> > - AF has completed a passive scan without finding focus.\n> > -\n> > - AwbState:\n> > type: int32_t\n> > draft: true\n> > @@ -690,4 +630,317 @@ controls:\n> > value. All of the custom test patterns will be static (that is the\n> > raw image must not vary from frame to frame).\n> >\n> > + - AfMode:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control to set the mode of the AF (autofocus) algorithm. Applications\n> > + are allowed to set a new mode, and to send additional controls for\n> > + that new mode, in the same request.\n> > +\n> > + An implementation may choose not to implement all the modes.\n> > +\n> > + enum:\n> > + - name: AfModeManual\n> > + value: 0\n> > + description: |\n> > + The AF algorithm is in manual mode. In this mode it will never\n> > + perform any action nor move the lens of its own accord, but an\n> > + application can set controls to move the lens.\n> > +\n> > + In this mode the AfState will always report AfStateManual.\n> > + - name: AfModeAuto\n> > + value: 1\n> > + description: |\n> > + The AF algorithm is in auto mode. This means that the algorithm\n> > + will never move the lens or change state unless the AfTrigger\n> > + control is used. The AfTrigger control can be used to initiate a\n> > + focus scan, the results of which will also be reported by AfState.\n> > +\n> > + If the autofocus algorithm is moved from AfModeAuto to another\n> > + mode while a scan is in progress, the scan is cancelled\n> > + immediately, without waiting for the scan to finish.\n> > +\n> > + When first entering this mode the AfState will report\n> > + AfStateAuto. When a trigger control is sent, AfState will\n> > + report AfStateAutoScanning for a period before spontaneously\n> > + changing to AfStateAutoFocused or AfStateAutoFailed, depending on\n> > + the outcome of the scan. It will remain in this state until\n> > + another scan is initiated by the AfTrigger control. If a scan is\n> > + cancelled (without changing to another mode), AfState will return\n> > + to AfStateAuto.\n> > + - name: AfModeContinuous\n> > + value: 2\n> > + description: |\n> > + The AF algorithm is in continuous mode. This means that the lens\n> > + can re-start a scan spontaneously at any moment, without any user\n> > + intervention. The AfState still reports whether the algorithm is\n> > + currently scanning or not, though the application has no ability\n> > + to initiate or cancel scans (though it can \"pause\" them), nor to\n> > + move the lens for itself.\n> > +\n> > + When set to AfModeContinuous, the system will immediately initiate\n> > + a scan so AfState will report AfStateContinuousScanning, and will\n> > + settle on one of AfStateContinuousFocused or AfStateContinuousFailed,\n> > + depending on the scan result.\n> > +\n> > + - AfLensPosition:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control to set the position of the lens to one of the following\n> > + predefined locations. This control only has any effect when the AfMode\n> > + is set to AfModeManual.\n> > +\n> > + This control is distinct from the LensPosition control, which sets the\n> > + lens position using the lens driver's units.\n> > + enum:\n> > + - name: AfLensPositionMacro\n> > + value: 0\n> > + description: The closest focal position of the lens.\n> > + - name: AfLensPositionHyperfocal\n> > + value: 1\n> > + description: Hyperfocal position.\n> > + - name: AfLensPositionInfinity\n> > + value: 2\n> > + description: The furthest focal position (usually infinity).\n> > +\n> > + - AfRange:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control to set the range of focus distances that is scanned. An\n> > + implementation may choose not to implement all the options here.\n> > + enum:\n> > + - name: AfRangeNormal\n> > + value: 0\n> > + description: |\n> > + A wide range of focus distances is scanned, all the way from\n> > + infinity down to close distances, though depending on the\n> > + implementation, possibly not including the very closest macro\n> > + positions.\n> > + - name: AfRangeMacro\n> > + value: 1\n> > + description: Only close distances are scanned.\n> > + - name: AfRangeFull\n> > + value: 2\n> > + description: |\n> > + The full range of focus distances is scanned just as with\n> > + AfRangeNormal but this time including the very closest macro\n> > + positions.\n> > +\n> > + - AfSpeed:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Control that determines whether the AF algorithm is to move the lens\n> > + as quickly as possible or more steadily. For example, during video\n> > + recording it may be desirable not to move the lens too abruptly, but\n> > + when in a preview mode (waiting for a still capture) it may be\n> > + helpful to move the lens as quickly as is reasonably possible.\n> > + enum:\n> > + - name: AfSpeedNormal\n> > + value: 0\n> > + description: Move the lens at its usual speed.\n> > + - name: AfSpeedFast\n> > + value: 1\n> > + description: Move the lens more quickly.\n> > +\n> > + - AfWindows:\n> > + type: Rectangle\n> > + draft: true\n> > + description: |\n> > + Sets the focus windows used by the AF algorithm. The units used express\n> > + a proportion of the ScalerCrop control (or if unavailable, of the entire\n> > + image), as u0.16 format numbers.\n>\n> I don't mind using ScalerCrop, but I think it would probably make more\n> sense to use the full pixel array ?\n\nThe thing that bothers me a bit about the full pixel array is that, if\nyou use digital zoom, then you will have to update your AfWindows as\nthey might otherwise lie outside your final image! If it's defined as\nbeing a region of the scaler crop, then it automatically stays the\nsame (to the application, at least, the pipeline handler probably has\nsome work to do).\n\nI note that the Android link you give below says \"Focus areas are\nrelative to the current field of view (getZoom())\", which I guess is\npartly why they use that slightly arbitrary +/-1000 range. Maybe 1000\nis easier to work with than the u0.16 format that I specified?\n\n>\n> > +\n> > + In order to be activated, a rectangle must be programmed with non-zero\n> > + width and height. If no rectangles are programmed in this way, then the\n> > + system will choose its own single default window in the centre of the\n> > + image.\n> > +\n> > + The details of how the windows are used are platform dependent. We note\n> > + that when there is more than one AF window, a typical implementation\n> > + might find the optimal focus position for each one and finally select\n> > + the window closest to the camera.\n> > +\n> > + size: [platform dependent]\n>\n> And I would like to point you to this one (you probably already now it):\n> https://developer.android.com/reference/android/hardware/Camera.Parameters#getFocusAreas()\n>\n> I really like the number of areas variable, and even more the weight per\n> area. It makes sense, you might want to have more weight in the center,\n> but still have a larger region.\n>\n> I don't get why the corrdinates are between [-1000, 1000] though but...\n> not that important.\n>\n> What do you think ?\n\nInteresting, I'm not quite sure what to make of it. It feels rather\nlike they don't support traditional multi-spot AF where you focus on\nthe closest of the regions, not on some average of them. The former\n(traditional multi-spot) certainly seems more useful to me. (In fact I\nthink \"averaging\" focus regions is a bit misguided, because you have\nno a priori information how different the amount of detail might be in\ndifferent regions. So a slightly featureless face in the foreground\nwould get totally overwhelmed by that bush lurking in the background,\npretty much irrespective of the weights you specify.)\n\nIn my original proposal there was an AfMethod control. Perhaps we need\nto bring it back (with values AfMethodAverage and AfMethodMultiSpot)\nto give us both options? In the former case we'd obviously need to add\na per-region weight. Actually, I think we might be able to conjure up\nan interesting interpretation for the weights in the multi-spot case,\nbut perhaps we leave that as \"implementation defined\" for the moment.\n\nThoughts? I'm trying to put together the next version of the patch,\nbut opinions on this in the meantime would be helpful.\n\nThanks!\n\nDavid\n\n> JM\n>\n> > +\n> > + - AfTrigger:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + This control starts an autofocus scan when AfMode is set to AfModeAuto,\n> > + and can also be used to terminate a scan early.\n> > +\n> > + It is ignored if AfMode is set to AfModeManual or AfModeContinuous.\n> > +\n> > + enum:\n> > + - name: AfTriggerStart\n> > + value: 0\n> > + description: Start an AF scan. Ignored if a scan is in progress.\n> > + - name: AfTriggerCancel\n> > + value: 1\n> > + description: Cancel an AF scan. This does not cause the lens to move\n> > + anywhere else. Ignored if no scan is in progress.\n> > +\n> > + - AfPause:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + This control has no effect except when in continuous autofocus mode\n> > + (AfModeContinuous). It can be used to pause any lens movements while\n> > + (for example) images are captured. The algorithm remains inactive\n> > + until it is instructed to resume.\n> > +\n> > + enum:\n> > + - name: AfPauseImmediate\n> > + value: 0\n> > + description: |\n> > + Pause the continuous autofocus algorithm immediately, whether or\n> > + not any kind of scan is underway. If the AfState was previously\n> > + reporting AfStateContinuousScanning it will now change to\n> > + AfStateContinuousScanningPaused, and similarly for\n> > + AfStateContinuousFocused and AfStateContinuousFailed.\n> > + - name AfPauseDeferred\n> > + value: 1\n> > + description: |\n> > + Pause the continuous autofocus algorithm as soon as it is no longer\n> > + scanning. If the AfState is currently reporting\n> > + AfStateContinuousFocused is will change to\n> > + AfStateContinuousFocusedPaused, and similarly in the case of\n> > + AfStateContinuousFailed.\n> > +\n> > + If AfState reports AfStateContinuousScanning it will change to\n> > + AfStateContinuousScanningPausing, and then move to one of\n> > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > + when the scan completes.\n> > + - name: AfPauseResume\n> > + value: 2\n> > + description: |\n> > + Resume continous autofocus operation. The algorithm starts again\n> > + from exactly where it left off, and the AfState will drop the\n> > + 'Paused' or 'Pausing' part of the state.\n> > +\n> > + - LensPosition:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Acts as a control to instruct the lens to move to a particular position\n> > + and also reports back the position of the lens for each frame.\n> > +\n> > + The units are determined by the lens driver.\n> > +\n> > + The LensPosition control is ignored unless the AfMode is set to\n> > + AfModeManual.\n> > +\n> > + Note that this control is distinct from AfLensPosition, which allows\n> > + the lens to be moved to its macro/hyperfocal/infinity position, rather\n> > + than using lens driver units.\n> > +\n> > + - AfState:\n> > + type: int32_t\n> > + draft: true\n> > + description: |\n> > + Reports the current state of the AF algorithm. The possible state changes\n> > + are described below, although we add that the following general state\n> > + changes are also allowed.\n> > +\n> > + In any of the AfStateManual or AfStateContinuous states, the AfMode\n> > + may be set to AfModeAuto and the algorithm will move to the\n> > + AfStateAuto state. If AfTriggerStart is sent at the same time\n> > + then the algorithm will move to AfStateAutoScanning state.\n> > +\n> > + In any of the AfStateAuto or AfStateManual states, the AfMode may\n> > + be set to AfModeContinuous and the algorithm will move to\n> > + AfStateContinuousScanning.\n> > +\n> > + In any of the AfStateAuto or AfStateContinuous states, the AfMode may\n> > + be set to AfModeManual and the algorithm will move to\n> > + AfStateManual. The lens will not be moved and will be left where\n> > + it was at that moment.\n> > +\n> > + enum:\n> > + - name: AfStateManual\n> > + value: 0\n> > + description: |\n> > + The AF algorithm is in manual mode (AfModeManual). The LensPosition\n> > + and AfLensPosition controls can be used directly to move the lens.\n> > + - name: AfStateAuto\n> > + value: 1\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and has either just\n> > + been moved into that state, or a scan that was in progress has been\n> > + cancelled.\n> > + - name: AfStateAutoScanning\n> > + value: 2\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and a scan has been\n> > + started using the AfTrigger control. The scan can be cancelled by\n> > + sending AfTriggerCancel at which point the algorithm will either\n> > + move back to AfStateAuto or, if the scan actually completes\n> > + before the cancel request is processed, to one of\n> > + AfStateAutoFocused or AfStateAutoFailed.\n> > + - name: AfStateAutoFocused\n> > + value: 3\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > + completed with the result that the algorithm believes the image is\n> > + now in focus.\n> > + - name: AfStateAutoFailed\n> > + value: 4\n> > + description: |\n> > + The AF algorithm is in auto mode (AfModeAuto), and a scan has\n> > + completed with the result that the algorithm did not find a good\n> > + focus position.\n> > + - name: AfStateContinuousScanning\n> > + value: 5\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + is currently scanning for a good focus position. This occurs when\n> > + the mode is first set to AfModeContinuous, or may happen\n> > + spontaneously when the algorithm believes a re-scan is needed.\n> > + - name: AfStateContinuousScanningPausing\n> > + value: 6\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + was scanning when AfPauseDeferred was sent. The scan will complete\n> > + at which point the algorithm moves to the\n> > + AfStateContinuousFocusedPaused or AfStateContinuousFailedPaused\n> > + state.\n> > + - name: AfStateContinuousScanningPaused\n> > + value: 7\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + was scanning when AfPauseImmediate was sent. The scan will not\n> > + complete and the algorithm will remain in this state. The scan\n> > + may be resumed by sending AfPauseResume.\n> > + - name: AfStateContinuousFocused\n> > + value: 8\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + a scan has completed with the algorithm believing it has found a\n> > + good focus position.\n> > + - name: AfStateContinuousFocusedPaused\n> > + value: 9\n> > + description: |\n> > + The AF algorithm was in the AfStateContinuousFocused state and\n> > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > + or it was in the AfStateContinuousScanningPausing state and the\n> > + scan has completed successfully. The algorithm will now remain\n> > + in this state, and may be resumed by sending AfPauseResume.\n> > + - name: AfStateContinuousFailed\n> > + value: 10\n> > + description: |\n> > + The AF algorithm is in continuous mode (AfModeContinuous) and\n> > + a scan has completed without finding a good focus position.\n> > + - name: AfStateContinuousFailedPaused\n> > + value: 11\n> > + description: |\n> > + The AF algorithm was in the AfStateContinuousFailed state and\n> > + has been paused (by either AfPauseImmediate or AfPauseDeferred),\n> > + or it was in the AfStateContinuousScanningPausing state and the\n> > + scan has completed unsuccessfully. The algorithm will now remain\n> > + in this state, and may be resumed by sending AfPauseResume.\n> > +\n> > ...","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EF4AAC3256\n\tfor ;\n\tWed, 20 Apr 2022 13:02:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1407065643;\n\tWed, 20 Apr 2022 15:02:24 +0200 (CEST)","from mail-wm1-x332.google.com (mail-wm1-x332.google.com\n\t[IPv6:2a00:1450:4864:20::332])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 541F6604AB\n\tfor ;\n\tWed, 20 Apr 2022 15:02:22 +0200 (CEST)","by mail-wm1-x332.google.com with SMTP id\n\tay11-20020a05600c1e0b00b0038eb92fa965so3625106wmb.4\n\tfor ;\n\tWed, 20 Apr 2022 06:02:22 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1650459744;\n\tbh=z9oAZ66kN7IWbycBX+jFHmv4hMuVjGvuZshorMmac3E=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=UtdKEl+HVOgQrYzUtKkthj3RbbI2339fser55gm3qRvGbS7ukHW4OxqRMR6Tj49P3\n\t6Jgrlg7zKSCx9BpE6ytgZJCcuuucLk4ClZrxL9mFaLIsrbALsFytcxLb5Y+ZJWOmmu\n\txy4ogc4p/ZsELum/7OyPUTnwZ6OWiCwrl6eOt4/3cBHQvS0TcV/eACPepsgSSUwwJT\n\t6dfCt/b1vY+BeZ6aQ7vaWsXrsEFfOllqmwSkJ8owEVQG97jtv0Ge1xJxbPhtrd4eTG\n\ttkxd47p3YRGIM1i8lDqLtYf3FqrNKJg3sjS6hFqZpQRst42fusy3xELnrZYXeMHDFj\n\tR/p32/T1YcepQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=G15XzC6l1IACU36+qqTSXieXzHkIUIl+RQLipD6eXiM=;\n\tb=qKjonZGebLeeU9pHwBhXcBLrrqjmzqBxS3v3bUvJICuerqre+pZufLAB3bq4l44mY8\n\tupz9fifXX4bPlAAFUPaEKQdRLiTn/UxZL/iOINYxXYhJv0YLrJ9xZ1+0WrVcflh9RWE3\n\tRbPUFsR/ai8T8SCdApbNfI2hSVvCOCYqKH5/Q8qGlz/tlJzfrbtNWmN1MJ1XIV1+HN7m\n\tSkdlqv6VypbFwKRbE9UrQ5wEPBmKq9Azb8iV9hnG43oh1LIBwhhUIXCI9EKYJNhNIk8U\n\tSYFA7TUibDxfSrumkskzm+88kNqIbuxbZpaKjvsPu6L6Mbc3LRwjeEC8dM2Fb1DsXP05\n\tQYyw=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"qKjonZGe\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=G15XzC6l1IACU36+qqTSXieXzHkIUIl+RQLipD6eXiM=;\n\tb=wgSYjIu3i8wsnmxwbnSftlBCEyVpuhfAe9redZdp38jERqUtyQkVfDQ+zDzFcBO6M8\n\tjtHWGT4a9tbX/ktc4RbuJ9hW7wIeWxvt7qISZfX0bTPU0waBRs3l1IPenwZvjhMSCFkE\n\tG91uRAVetaMFYTHp13lk1xRAKWKM9I58tUk1guWvVwHKqJvactNOGnZuEBhMVmCXY4ex\n\t3q1FO1/H15c5C4qXBea5Ak1DGgWexFRiKxMPmB5UnQViI9ZEnZN2k0Fr7W1M8D5fDIAj\n\tDGUQKiaZ/FMBUzr/eF2okNqNzcR0pe/87Aj0K1e19k5kVdui4EZ6DFSasNbGxJ0xAEse\n\tYSzQ==","X-Gm-Message-State":"AOAM530vKN4UY0fqhPThvzjL48oY5E8FtFlAu89wl7rFBsHHd+JkWkDn\n\tHG2iAOUiYdX373vfCc78F8akNg3YwO/u8tZjorE2wNighujk0xSq","X-Google-Smtp-Source":"ABdhPJz+2NS+Mw5BtXs/7e576UkXvEdANnAreXiUQzAeig5qWPKjXnFkkWvpPLpb+BuFFrgMZjJ0hOQdTsdEj0bKeVM=","X-Received":"by 2002:a05:600c:54e:b0:38e:bbef:9115 with SMTP id\n\tk14-20020a05600c054e00b0038ebbef9115mr3594948wmc.60.1650459741543;\n\tWed, 20 Apr 2022 06:02:21 -0700 (PDT)","MIME-Version":"1.0","References":"<20220331151747.19458-1-david.plowman@raspberrypi.com>\n\t<20220331151747.19458-2-david.plowman@raspberrypi.com>\n\t","In-Reply-To":"","Date":"Wed, 20 Apr 2022 14:02:10 +0100","Message-ID":"","To":"Jean-Michel Hautbois ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2 1/1] libcamera: controls: Controls\n\tfor driving AF (autofocus) algorithms","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"David Plowman via libcamera-devel ","Reply-To":"David Plowman ","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]