[{"id":23077,"web_url":"https://patchwork.libcamera.org/comment/23077/","msgid":"","date":"2022-05-18T22:19:21","subject":"Re: [libcamera-devel] [PATCH 3/3] libcamera: control_ids: Add\n\thigh-level documentation for AE","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nI'm going to hijack this patch to bring up a question to David :-)\n\nDavid, how do you envision the exposure time and gain controls to be\nhandled by the IPA when one of the two is in automatic mode and the\nother one in manual mode ? Let's take an example. The camera is running,\nwith the gain set to manual mode and value x1.0. The exposure time is\nautomatic, and the AGC algorithm has converged and computed a proper\nexposure value. At some point, in request N, the application modifies\nthe gain to x2.0. The frame captured for request N should have that gain\napplied. How about the exposure time ? Do you expect the IPA to halve\nthe exposure time exactly for that frame (neither earlier nor later), or\ndo you expect that the AGC could need one or a few frames to converge to\nthe new correct exposure time ?\n\nOn Wed, May 18, 2022 at 03:47:28PM +0200, Paul Elder via libcamera-devel wrote:\n> Add high-level documentation for AE, describing the states and the\n> transitions and how one would use the AE controls.\n> \n> Signed-off-by: Paul Elder \n> ---\n> src/libcamera/control_ids.cpp.in | 82 ++++++++++++++++++++++++++++++++\n> 1 file changed, 82 insertions(+)\n> \n> diff --git a/src/libcamera/control_ids.cpp.in b/src/libcamera/control_ids.cpp.in\n> index 5fb1c2c3..62451d7f 100644\n> --- a/src/libcamera/control_ids.cpp.in\n> +++ b/src/libcamera/control_ids.cpp.in\n> @@ -10,6 +10,88 @@\n> #include \n> #include \n> \n> +/**\n> + * \\page AE Auto-exposure Support\n> + *\n> + * libcamera offers the following controls related to exposure and gain:\n> + * - AnalogueGain\n> + * - AnalogueGainMode\n> + * - ExposureTime\n> + * - ExposureTimeMode\n> + * - AeState\n> + *\n> + * Auto-exposure and auto-gain can be enabled and disabled separately using the\n> + * ExposureTimeMode and AnalogueGainMode controls respectively. There is no\n> + * overarching AeEnable control.\n> + *\n> + * For each of exposure and gain, we can model it with three states: auto,\n> + * locked, and manual. Note that AnalogueGainMode and ExposureTimeMode only\n> + * have two values, as the locked state is simulated.\n> + *\n> + * /---------------------------------\\\n> + * | |\n> + * V |\n> + * +--------+ +--------+\n> + * | | ---------------------> | |\n> + * | Auto | +--------+ | Manual |\n> + * | | ----> | Locked | ----> | |\n> + * +--------+ +--------+ +--------+\n> + * ^ |\n> + * | |\n> + * \\----------------/\n> + *\n> + * Notice from the state diagram that locked to manual is a one-way state\n> + * change, as the reverse direction is nonsensical (see the design document for\n> + * more details on this topic).\n> + *\n> + * The exposure/gain is in the Auto state when\n> + * ExposureTimeMode/AnalogueGainMode is set to Auto. In this state, the value\n> + * that is computed by the AE algorithm is applied to the image sensor. Any\n> + * value that is supplied in the ExposureTime/AnalogueGain control is ignored\n> + * and is not retained.\n> + *\n> + * If ExposureTimeMode/AnalogueGainMode is set to Disabled, it can put us in\n> + * either the Locked or Manual state. The difference is in if\n> + * ExposureTime/AnalogueGain has been supplied. If it has not yet been\n> + * supplied, then we are in the Locked state. If it has been supplied, then we\n> + * are in the Manual state.\n> + *\n> + * In both the Locked state and the Manual state the exposure/gain value does\n> + * not come from the AE algorithm. In the Locked state the value comes from the\n> + * last value computed by the AE algorithm while the state was Auto, or if the\n> + * state was never Auto (e.g. we started in Locked, or the camera doesn't\n> + * support Auto), then the value should be a best-effort default value. In the\n> + * Manual state the value comes from the value supplied in the\n> + * ExposureTime/AnalogueGain control.\n> + *\n> + * To transition from the Locked state to the Manual state, a value needs to be\n> + * submitted in ExposureTime/AnalogueGain. Once the state has transitioned to\n> + * Manual, then this value will be retained, and doesn't need to be resubmitted\n> + * if it doesn't change.\n> + *\n> + * To transition to the Auto state, simply set\n> + * ExposureTimeMode/AnalogueGainMode to Auto.\n> + *\n> + *\n> + * The AeState metadata reports the state of the AE algorithm. As AE cannot\n> + * compute exposure and gain separately, the state of the AE component is\n> + * unified. There are three states: Inactive, Searching, and Converged.\n> + *\n> + * The state shall be Inactive if both ExposureTimeMode and AnalogueGainMode\n> + * are set to Disabled. If the camera only supports one of the two controls,\n> + * then the state shall be Inactive if that one control is set to Disabled. If\n> + * the camera does not support Disabled for at least one of the two controls,\n> + * then the state will never be Inactive, as AE will always be running.\n> + *\n> + * The state shall be Searching if at least one of exposure or gain calculated\n> + * by the AE algorithm is used (that is, at least one of the two modes is Auto),\n> + * *and* the value(s) have not converged yet.\n> + *\n> + * The state shall be Converged if at least one of exposure or gain calculated\n> + * by the AE algorithm is used (that is, at least one of the two modes is Auto),\n> + * *and* the value(s) have converged.\n> + */\n> +\n> /**\n> * \\file control_ids.h\n> * \\brief Camera control identifiers","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 32554C3256\n\tfor ;\n\tWed, 18 May 2022 22:19:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8577765659;\n\tThu, 19 May 2022 00:19:30 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A146B6041E\n\tfor ;\n\tThu, 19 May 2022 00:19:28 +0200 (CEST)","from pendragon.ideasonboard.com (unknown [45.131.31.124])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 372EE9E5;\n\tThu, 19 May 2022 00:19:28 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652912370;\n\tbh=tTdV8zDieBs+rzDbS5n4/i+Dp16PeltQGowydXNbYSY=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=uprAt1ZTQKjmUPpYag8hUAMtxhcSsdY98SKwFxXWayDyV0N4R/0x7bZW0/09q8Kwl\n\tEmX5UxHm1NeJykZjUhyoFSoMiEIEAyABr2sx35PfGw1uesN/qH8mVJ0ez057md/j21\n\tHE9aM+PK3fPEnffUKd/7rQgwXLTXU6aYOVzgA5kuO4oPXJuZ0VrcoX18688gEZv5aZ\n\tKJdTH063a6Lq8QAMIyKvG4JXsgu0Em0uwc4YDJsrS2VymNsNAXTx40R+38qxnMdWTi\n\tUj2LVtZ9V5pj0mY/XR8BoTvI9G3FaZbYsHwMUgpeHtL7MqOPLyak1ZajPp8XFF7GqG\n\ttLwrM80fPVZug==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652912368;\n\tbh=tTdV8zDieBs+rzDbS5n4/i+Dp16PeltQGowydXNbYSY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=DIjRjPX8kyK3gTeVGk/dd6OcgD9ey96ZVNy9GD1d+Yh54JjX8LZ/i//u2QzAPe3bI\n\tKgDxQK6T6IWLpeqK1+oVy4AiHjbYrStO1TG/SIgVfg8OrTP/+eALniUuTk/3tjiqCv\n\tLq+qOuPx7ordeHIv9zxRt0KnSs5ObI6GMWfjDmLo="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"DIjRjPX8\"; dkim-atps=neutral","Date":"Thu, 19 May 2022 01:19:21 +0300","To":"Paul Elder ,\n\tDavid Plowman ","Message-ID":"","References":"<20220518134728.777709-1-paul.elder@ideasonboard.com>\n\t<20220518134728.777709-4-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220518134728.777709-4-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 3/3] libcamera: control_ids: Add\n\thigh-level documentation for AE","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]