From patchwork Mon May 16 16:12:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 15922 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D4338C326C for ; Mon, 16 May 2022 16:12:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 905B66565D; Mon, 16 May 2022 18:12:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1652717552; bh=0f/GoXPEk1GvVz2yEDOkzdQ7j7SbYpatwU0yMnblykA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=wizxBbyp1HvI9L6YsY284Cl6PSA1lPZRODsIShDVz3AlTchhmdNcyUOKJd/Sii+HK +3/C5b57NJj0Ss/Tj3DJbTuv2E6qqecgZSPHxR+yCnQC+0QDcWNH3l4dB2OzN9fOqA cQgqZNkkSrFoJ3OHwOUbDp2GvR2HOFbdBlftbtFdYsQCogV+1wmOzReKo7FcHYZQeH EhwtV15JbI2FnzazYk96o/o6qN4MEhMXC20X+ustaA596bechRSJSI95qnK9QLZ/XI BxJNuSWmmhhMGf07AdF7JlFxSRLv1SC5x4p6dBJUQLg211iaaCuZx8Ug8Myt3nDZZK ekeGm6rKBTsQg== Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E1BD665653 for ; Mon, 16 May 2022 18:12:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rMGCisTY"; dkim-atps=neutral Received: by mail-wm1-x333.google.com with SMTP id bd25-20020a05600c1f1900b0039485220e16so227630wmb.0 for ; Mon, 16 May 2022 09:12:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XrL/oVeM8AtWomWo/znShh4o33Jf6xBFbjHHDsMLi3Q=; b=rMGCisTY44Ow/8Js0Qwn7MnX2SRSG+KR+yccQuakUedniO4dYuFSlDajX/WziTqLRo +UT5ElD7EefQf4HfAz4VPt59+DvcU6+tBZgQ1PxEzd/NOSNvBhPMi2hgY6L6xZlIQm+T CbwQ0+b0UENStj3BeQE5WEAjOfXD9Pi972t/k7FWOmlhOP0IBjPVsYHHMO8y6FIQJYCR MtUXldB3kp26CZ6/Jj2DcZpHgaDRnp96HeBJ0/Tz8bdrdMP/FSHA87IWPizuDuh4431K 9LfQnoKBLwr9bI7QeWO9OU8CLVjg3Q07u6tTJXMg9Q/VFkPBhY8XqFYMkHpIot+v0dj0 Ld2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XrL/oVeM8AtWomWo/znShh4o33Jf6xBFbjHHDsMLi3Q=; b=U3HZFMpU4gk3ucZ8oLsyHRIIvTb7TtdkFz61Di5mThqgHR6+MM9vgNpcJuovcfnjEt ViY+g1Dd4S11NrRHLe/BWiKoEpPmymGaQcZK1TldeASmsyaVDkFoykRNyWEH+OWR5fgy MmcRZgK4q16lNGNo01vA5mU0QbVgUVT4mX/bd8Wdglu9Z5CvvF6BQyFZnZ2kC/6Anuqb BmiebhFZBeAkK5lffHwhhU6Py6XY3ZJg1WSeb1LwgL7u0wenyrOcRAaKFYWpW6mymv6a QaKzuDPw8m9vKwGBNCMqpIbBMsjDbOHPB8JN1ggZRDW/iM1vYOfkwm7pC/Pba9iZ73Kj +lMQ== X-Gm-Message-State: AOAM533S4BaBGWgd26HkfqoPiNpm22/JBb3Vad6BX118wMZAJNT+Uyf5 +yASois9ZBXmAtDTOceq+aGO4H5Xa27MDOz2 X-Google-Smtp-Source: ABdhPJxV6kmWD5DArMzTQ2C1mlIecI0KVrWXlozAPsPt0vjnM9696ydvJSqkkY9zoziAwUYgk74Xew== X-Received: by 2002:a7b:cb84:0:b0:382:a9b9:2339 with SMTP id m4-20020a7bcb84000000b00382a9b92339mr28505318wmi.91.1652717550130; Mon, 16 May 2022 09:12:30 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id n14-20020a05600c3b8e00b0039429bfebeasm349219wms.2.2022.05.16.09.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 May 2022 09:12:29 -0700 (PDT) To: libcamera-devel@lists.libcamera.org, Jacopo Mondi , Han-Lin Chen , Jean-Michel Hautbois , Kate Hsuan Date: Mon, 16 May 2022 17:12:24 +0100 Message-Id: <20220516161224.6712-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220516161224.6712-1-david.plowman@raspberrypi.com> References: <20220516161224.6712-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/1] libcamera: controls: Controls for driving AF (autofocus) algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: David Plowman via libcamera-devel From: David Plowman Reply-To: David Plowman Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch describes a series of controls that allow applications to drive AF algorithms: AfMode - manual, auto or continuous AfRange - full, macro or normal AfSpeed - fast or slow AfMetering - how to choose where to measure focus AfWindows - AF window locations AfTrigger - start (trigger) an AF scan or cancel AfPause - pause continuous AF LensPosition - set or retrieve position of lens AfState - reports whether scanning/success/failure AfPauseState - reports whether continuous AF paused or not Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Jean-Michel Hautbois --- src/libcamera/control_ids.yaml | 354 +++++++++++++++++++++++++++------ 1 file changed, 294 insertions(+), 60 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 9d4638ae..0e117de8 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -381,6 +381,300 @@ controls: \todo Define how the sensor timestamp has to be used in the reprocessing use case. + - AfMode: + type: int32_t + description: | + Control to set the mode of the AF (autofocus) algorithm. + + An implementation may choose not to implement all the modes. + + enum: + - name: AfModeManual + value: 0 + description: | + The AF algorithm is in manual mode. In this mode it will never + perform any action nor move the lens of its own accord, but an + application can specify the desired lens position using the + LensPosition control. + + In this mode the AfState will always report AfStateIdle. + - name: AfModeAuto + value: 1 + description: | + The AF algorithm is in auto mode. This means that the algorithm + will never move the lens or change state unless the AfTrigger + control is used. The AfTrigger control can be used to initiate a + focus scan, the results of which will be reported by AfState. + + If the autofocus algorithm is moved from AfModeAuto to another + mode while a scan is in progress, the scan is cancelled + immediately, without waiting for the scan to finish. + + When first entering this mode the AfState will report + AfStateIdle. When a trigger control is sent, AfState will + report AfStateScanning for a period before spontaneously + changing to AfStateFocused or AfStateFailed, depending on + the outcome of the scan. It will remain in this state until + another scan is initiated by the AfTrigger control. If a scan is + cancelled (without changing to another mode), AfState will return + to AfStateIdle. + - name: AfModeContinuous + value: 2 + description: | + The AF algorithm is in continuous mode. This means that the lens can + re-start a scan spontaneously at any moment, without any user + intervention. The AfState still reports whether the algorithm is + currently scanning or not, though the application has no ability to + initiate or cancel scans, nor to move the lens for itself. + + However, applications can pause the AF algorithm from continuously + scanning by using the AfPause control. This allows video or still + images to be captured whilst guaranteeing that the focus is fixed. + + When set to AfModeContinuous, the system will immediately initiate a + scan so AfState will report AfStateScanning, and will settle on one + of AfStateFocused or AfStateFailed, depending on the scan result. + + - AfRange: + type: int32_t + description: | + Control to set the range of focus distances that is scanned. An + implementation may choose not to implement all the options here. + enum: + - name: AfRangeNormal + value: 0 + description: | + A wide range of focus distances is scanned, all the way from + infinity down to close distances, though depending on the + implementation, possibly not including the very closest macro + positions. + - name: AfRangeMacro + value: 1 + description: Only close distances are scanned. + - name: AfRangeFull + value: 2 + description: | + The full range of focus distances is scanned just as with + AfRangeNormal but this time including the very closest macro + positions. + + - AfSpeed: + type: int32_t + description: | + Control that determines whether the AF algorithm is to move the lens + as quickly as possible or more steadily. For example, during video + recording it may be desirable not to move the lens too abruptly, but + when in a preview mode (waiting for a still capture) it may be + helpful to move the lens as quickly as is reasonably possible. + enum: + - name: AfSpeedNormal + value: 0 + description: Move the lens at its usual speed. + - name: AfSpeedFast + value: 1 + description: Move the lens more quickly. + + - AfMetering: + type: int32_t + description: | + Instruct the AF algorithm how it should decide which parts of the image + should be used to measure focus. + enum: + - name: AfMeteringAuto + value: 0 + description: The AF algorithm should decide for itself where it will + measure focus. + - name: AfMeteringWindows + value: 1 + description: The AF algorithm should use the rectangles defined by + the AfWindows control to measure focus. If no windows are specified + the behaviour is platform dependent. + + - AfWindows: + type: Rectangle + description: | + Sets the focus windows used by the AF algorithm when AfMetering is set + to AfMeteringWindows. The units used are pixels within the rectangle + returned by the ScalerCropMaximum property. + + In order to be activated, a rectangle must be programmed with non-zero + width and height. Internally, these rectangles are intersected with the + ScalerCropMaximum rectangle. If the window becomes empty after this + operation, then the window is ignored. If all the windows end up being + ignored, then the behaviour is platform dependent. + + On platforms that support the ScalerCrop control (for implementing + digital zoom, for example), no automatic recalculation or adjustment of + AF windows is performed internally if the ScalerCrop is changed. If any + window lies outside the output image after the scaler crop has been + applied, it is up to the application to recalculate them. + + The details of how the windows are used are platform dependent. We note + that when there is more than one AF window, a typical implementation + might find the optimal focus position for each one and finally select + the window where the focal distance for the objects shown in that part + of the image are closest to the camera. + + size: [n] + + - AfTrigger: + type: int32_t + description: | + This control starts an autofocus scan when AfMode is set to AfModeAuto, + and can also be used to terminate a scan early. + + It is ignored if AfMode is set to AfModeManual or AfModeContinuous. + + enum: + - name: AfTriggerStart + value: 0 + description: Start an AF scan. Ignored if a scan is in progress. + - name: AfTriggerCancel + value: 1 + description: Cancel an AF scan. This does not cause the lens to move + anywhere else. Ignored if no scan is in progress. + + - AfPause: + type: int32_t + description: | + This control has no effect except when in continuous autofocus mode + (AfModeContinuous). It can be used to pause any lens movements while + (for example) images are captured. The algorithm remains inactive + until it is instructed to resume. + + enum: + - name: AfPauseImmediate + value: 0 + description: | + Pause the continuous autofocus algorithm immediately, whether or not + any kind of scan is underway. AfPauseState will subsequently report + AfPauseStatePaused. AfState may report any of AfStateScanning, + AfStateFocused or AfStateFailed, depending on the algorithm's state + when it received this control. + - name: AfPauseDeferred + value: 1 + description: | + This is similar to AfPauseImmediate, and if the AfState is currently + reporting AfStateFocused or AfStateFailed it will remain in that + state and AfPauseState will report AfPauseStatePaused. + + However, if the algorithm is scanning (AfStateScanning), + AfPauseState will report AfPauseStatePausing until the scan is + finished, at which point AfState will report one of AfStateFocused + or AfStateFailed, and AfPauseState will change to + AfPauseStatePaused. + + - name: AfPauseResume + value: 2 + description: | + Resume continuous autofocus operation. The algorithm starts again + from exactly where it left off, and AfPauseState will report + AfPauseStateRunning. + + - LensPosition: + type: float + description: | + Acts as a control to instruct the lens to move to a particular position + and also reports back the position of the lens for each frame. + + The LensPosition control is ignored unless the AfMode is set to + AfModeManual, though the value is reported back unconditionally in all + modes. + + The units are dioptres divided by the hyperfocal distance. Non-integer + values are permitted. For example: + 0 moves the lens to infinity. + 0.5 moves the lens to twice the hyperfocal distance. + 1 moves the lens to the hyperfocal position. + And larger values will focus the lens ever closer. + + \todo Define a property to report the Hyperforcal distance of calibrated + lenses. + + \todo Define a property to report the maximum and minimum positions of + this lens. The minimum value will often be zero (meaning infinity). + + - AfState: + type: int32_t + description: | + Reports the current state of the AF algorithm in conjunction with the + reported AfMode value and (in continuous AF mode) the AfPauseState + value. The possible state changes are described below, though we note + the following state transitions that occur when the AfMode is changed. + + If the AfMode is set to AfModeManual, then the AfState will always + report AfStateIdle (even if the lens is subsequently moved). Changing to + the AfModeManual state does not initiate any lens movement. + + If the AfMode is set to AfModeAuto then the AfState will report + AfStateIdle. However, if AfModeAuto and AfTriggerStart are sent together + then AfState will omit AfStateIdle and move straight to AfStateScanning + (and start a scan). + + If the AfMode is set to AfModeContinuous then the AfState will initially + report AfStateScanning. + + enum: + - name: AfStateIdle + value: 0 + description: | + The AF algorithm is in manual mode (AfModeManual) or in auto mode + (AfModeAuto) and a scan has not yet been triggered, or an + in-progress scan was cancelled. + - name: AfStateScanning + value: 1 + description: | + The AF algorithm is in auto mode (AfModeAuto), and a scan has been + started using the AfTrigger control. The scan can be cancelled by + sending AfTriggerCancel at which point the algorithm will either + move back to AfStateIdle or, if the scan actually completes before + the cancel request is processed, to one of AfStateFocused or + AfStateFailed. + + Alternatively the AF algorithm could be in continuous mode + (AfModeContinuous) at which point it may enter this state + spontaneously whenever it determines that a rescan is needed. + - name: AfStateFocused + value: 2 + description: | + The AF algorithm is in auto (AfModeAuto) or continuous + (AfModeContinuous) mode and a scan has completed with the result + that the algorithm believes the image is now in focus. + - name: AfStateFailed + value: 3 + description: | + The AF algorithm is in auto (AfModeAuto) or continuous + (AfModeContinuous) mode and a scan has completed with the result + that the algorithm did not find a good focus position. + + - AfPauseState: + type: int32_t + description: | + Only applicable in continuous (AfModeContinuous) mode, this reports + whether the algorithm is currently running, paused or pausing (that is, + will pause as soon as any in-progress scan completes). + + Any change to AfMode will cause AfPauseStateRunning to be reported. + + enum: + - name: AfPauseStateRunning + value: 0 + description: | + Continuous AF is running and the algorithm may restart a scan + spontaneously. + - name: AfPauseStatePausing + value: 1 + description: | + Continuous AF has been sent an AfPauseDeferred control, and will + pause as soon as any in-progress scan completes (and then report + AfPauseStatePaused). No new scans will be start spontaneously until + the AfPauseResume control is sent. + - name: AfPauseStatePaused + value: 2 + description: | + Continuous AF is paused. No further state changes or lens movements + will occur until the AfPauseResume control is sent. + # ---------------------------------------------------------------------------- # Draft controls section @@ -406,27 +700,6 @@ controls: The camera will cancel any active or completed metering sequence. The AE algorithm is reset to its initial state. - - AfTrigger: - type: int32_t - draft: true - description: | - Control for AF trigger. Currently identical to - ANDROID_CONTROL_AF_TRIGGER. - - Whether the camera device will trigger autofocus for this request. - enum: - - name: AfTriggerIdle - value: 0 - description: The trigger is idle. - - name: AfTriggerStart - value: 1 - description: The AF routine is started by the camera. - - name: AfTriggerCancel - value: 2 - description: | - The camera will cancel any active trigger and the AF routine is - reset to its initial state. - - NoiseReductionMode: type: int32_t draft: true @@ -507,45 +780,6 @@ controls: The AE algorithm has started a pre-capture metering session. \sa AePrecaptureTrigger - - AfState: - type: int32_t - draft: true - description: | - Control to report the current AF algorithm state. Currently identical to - ANDROID_CONTROL_AF_STATE. - - Current state of the AF algorithm. - enum: - - name: AfStateInactive - value: 0 - description: The AF algorithm is inactive. - - name: AfStatePassiveScan - value: 1 - description: | - AF is performing a passive scan of the scene in continuous - auto-focus mode. - - name: AfStatePassiveFocused - value: 2 - description: | - AF believes the scene is in focus, but might restart scanning. - - name: AfStateActiveScan - value: 3 - description: | - AF is performing a scan triggered by an AF trigger request. - \sa AfTrigger - - name: AfStateFocusedLock - value: 4 - description: | - AF believes has focused correctly and has locked focus. - - name: AfStateNotFocusedLock - value: 5 - description: | - AF has not been able to focus and has locked. - - name: AfStatePassiveUnfocused - value: 6 - description: | - AF has completed a passive scan without finding focus. - - AwbState: type: int32_t draft: true