Patch Detail
Show a patch.
GET /api/patches/15179/?format=api
{ "id": 15179, "url": "https://patchwork.libcamera.org/api/patches/15179/?format=api", "web_url": "https://patchwork.libcamera.org/patch/15179/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20211221043610.2512334-2-paul.elder@ideasonboard.com>", "date": "2021-12-21T04:36:03", "name": "[libcamera-devel,v3,1/8] controls: Reorganize the AE-related controls", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "74b32d0743cd9be03478db85bc07a23f4a692e74", "submitter": { "id": 17, "url": "https://patchwork.libcamera.org/api/people/17/?format=api", "name": "Paul Elder", "email": "paul.elder@ideasonboard.com" }, "delegate": { "id": 17, "url": "https://patchwork.libcamera.org/api/users/17/?format=api", "username": "epaul", "first_name": "Paul", "last_name": "Elder", "email": "paul.elder@ideasonboard.com" }, "mbox": "https://patchwork.libcamera.org/patch/15179/mbox/", "series": [ { "id": 2849, "url": "https://patchwork.libcamera.org/api/series/2849/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=2849", "date": "2021-12-21T04:36:02", "name": "The Great AE Changes", "version": 3, "mbox": "https://patchwork.libcamera.org/series/2849/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/15179/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/15179/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C071EBE080\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue, 21 Dec 2021 04:36:23 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 361DC608F9;\n\tTue, 21 Dec 2021 05:36:23 +0100 (CET)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C438E60592\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 21 Dec 2021 05:36:20 +0100 (CET)", "from pyrite.mediacom.info (unknown\n\t[IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C2223FD2;\n\tTue, 21 Dec 2021 05:36:19 +0100 (CET)" ], "Authentication-Results": "lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"krON6oLD\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1640061380;\n\tbh=g52XoS8OdmFvRJ7mJBhylmfhTKgyygElo9/q5i1W9B0=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=krON6oLDzmsnrcmvRec9CPTs6g29dAX2F2bF59m/HGq5vTVqsyCA0sJyMiwGHXmWO\n\tRSQFkOkE6zwHN31Mm5DT1AenpQN4635TbhuqY1wESX8UWxUs5e2B05BA+oviUtmnpg\n\tfVla4cb92gjxWLQ+FvzjLy0umRM/PhoQFCXNNsik=", "From": "Paul Elder <paul.elder@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Date": "Mon, 20 Dec 2021 22:36:03 -0600", "Message-Id": "<20211221043610.2512334-2-paul.elder@ideasonboard.com>", "X-Mailer": "git-send-email 2.27.0", "In-Reply-To": "<20211221043610.2512334-1-paul.elder@ideasonboard.com>", "References": "<20211221043610.2512334-1-paul.elder@ideasonboard.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "Subject": "[libcamera-devel] [PATCH v3 1/8] controls: Reorganize the\n\tAE-related controls", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "We have multiple goals:\n- we need a lock of some sort, to instruct the AEGC to not update output\n results\n- we need manual modes, to override the values computed by the AEGC\n- we need to support seamless transitions from auto -> manual, and do so\n without flickering\n- we need custom minimum values for the manual controls, that is no\n magic values for enabling/disabling auto\n- all of these need to be done with AE sub-controls (exposure time,\n analogue gain)\n\nTo achieve these goals, we introduce mode controls for the AE\nsub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto\nstate, and a disabled state. The disabled state has an internal one-way\nstate change from locked to manual, triggered by the presence of the\nvalue-controls (ExposureTime and AnalogueGain).\n\nWe then remove the AeEnable control, as it is a redundant control in the\nface of these two mode controls.\n\nWe also remove AeLocked, as it is insufficient for reporting the AE\nstate, and we promote AeState to non-draft to fill its role. Notably,\nthe locked state is removed, since this information can be obtained from\nthe aforementioned mode controls.\n\nBug: https://bugs.libcamera.org/show_bug.cgi?id=42\nBug: https://bugs.libcamera.org/show_bug.cgi?id=43\nBug: https://bugs.libcamera.org/show_bug.cgi?id=47\nSigned-off-by: Paul Elder <paul.elder@ideasonboard.com>\n\n---\nChanges in v3:\n- improve wording of the control descriptions\n - make more succinct and clear\n- add description of how to do a flickerless transition\n\nChanges in v2:\n- No changes, just resubmitting at the head of this series so that it's\n together and so that /people will actually see it/\n\nInitial version:\nStill RFC as I haven't updated the users of the control yet, and I want\nto check that these are the controls and docs that we want.\n\nWe've decided that the \"master AE control\" will be implemented by a\nhelper... but looking at uvcvideo and the V4L2 controls I'm wondering if\nsuch helper should come earlier than later?\n---\n src/libcamera/control_ids.yaml | 239 ++++++++++++++++++++++++---------\n 1 file changed, 177 insertions(+), 62 deletions(-)", "diff": "diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\nindex 9d4638ae..84679317 100644\n--- a/src/libcamera/control_ids.yaml\n+++ b/src/libcamera/control_ids.yaml\n@@ -7,23 +7,63 @@\n # Unless otherwise stated, all controls are bi-directional, i.e. they can be\n # set through Request::controls() and returned out through Request::metadata().\n controls:\n- - AeEnable:\n- type: bool\n+ - AeState:\n+ type: int32_t\n description: |\n- Enable or disable the AE.\n+ Control to report the current AE algorithm state. The camera device can do\n+ several state transitions between two results, if it is allowed by the\n+ state transition table. For example, AeStateInactive may never\n+ actually be seen in a result.\n \n- \\sa ExposureTime AnalogueGain\n+ The state in the result is the state for this request. If AE state\n+ becomes AeStateConverged, then the image data associated with the\n+ result should be good to use.\n \n- - AeLocked:\n- type: bool\n- description: |\n- Report the lock status of a running AE algorithm.\n+ The state is still reported even if ExposureTimeMode or\n+ AnalogueGainMode is set to Disabled.\n \n- If the AE algorithm is locked the value shall be set to true, if it's\n- converging it shall be set to false. If the AE algorithm is not\n- running the control shall not be present in the metadata control list.\n+ \\sa AnalogueGain\n+ \\sa AnalogueGainMode\n+ \\sa ExposureTime\n+ \\sa ExposureTimeMode\n \n- \\sa AeEnable\n+ enum:\n+ - name: AeStateInactive\n+ value: 0\n+ description: |\n+ The AE algorithm is inactive.\n+ If the camera initiates an AE scan, the state shall go to\n+ AeStateSearching.\n+\n+ Unlike the analogous control in Android, enabling or disabling any\n+ of the AE-related mode controls (eg. AnalogueGainMode,\n+ ExposureTimeMode) does not require the state to be reset to to\n+ AeStateInactive.\n+ - name: AeStateSearching\n+ value: 1\n+ description: |\n+ The AE algorithm has not converged yet.\n+ If the camera finishes an AE scan, the state shall go to\n+ AeStateConverged. If the camera finishes an AE scan, but flash is\n+ required, the state shall go to AeStateFlashRequired.\n+ - name: AeStateConverged\n+ value: 2\n+ description: |\n+ The AE algorithm has converged.\n+ If the camera initiates an AE scan, the state shall go to\n+ AeStateSearching.\n+ - name: AeStateFlashRequired\n+ value: 3\n+ description: |\n+ The AE algorithm would need a flash for good results.\n+ If the camera initiates an AE scan, the state shall go to\n+ AeStateSearching.\n+ - name: AeStatePrecapture\n+ value: 4\n+ description: |\n+ The AE algorithm has started a pre-capture metering session.\n+ After the sequence is finished, the state shall go to AeStateConverged if\n+ \\sa AePrecaptureTrigger\n \n # AeMeteringMode needs further attention:\n # - Auto-generate max enum value.\n@@ -93,6 +133,13 @@ controls:\n how the desired total exposure is divided between the shutter time\n and the sensor's analogue gain. The exposure modes are platform\n specific, and not all exposure modes may be supported.\n+\n+ When one of AnalogueGainMode or ExposureTimeMode is set to Disabled,\n+ the fixed values will override any choices made by AeExposureMode.\n+\n+ \\sa AnalogueGainMode\n+ \\sa ExposureTimeMode\n+\n enum:\n - name: ExposureNormal\n value: 0\n@@ -111,13 +158,15 @@ controls:\n type: float\n description: |\n Specify an Exposure Value (EV) parameter. The EV parameter will only be\n- applied if the AE algorithm is currently enabled.\n+ applied if the AE algorithm is currently enabled, that is, at least one\n+ of AnalogueGainMode and ExposureTimeMode are auto.\n \n By convention EV adjusts the exposure as log2. For example\n EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment\n of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x].\n \n- \\sa AeEnable\n+ \\sa AnalogueGainMode\n+ \\sa ExposureTimeMode\n \n - ExposureTime:\n type: int32_t\n@@ -125,17 +174,65 @@ controls:\n Exposure time (shutter speed) for the frame applied in the sensor\n device. This value is specified in micro-seconds.\n \n- Setting this value means that it is now fixed and the AE algorithm may\n- not change it. Setting it back to zero returns it to the control of the\n- AE algorithm.\n+ This control will only take effect if ExposureTimeMode is Disabled. Its\n+ presence in a request acts as a trigger to switch to the internal\n+ manual mode within ExposureTimeModeDisabled.\n+\n+ When reported in metadata, this control indicates what exposure time\n+ was used for the current request, regardless of ExposureTimeMode.\n+ ExposureTimeMode will indicate the source of the exposure time value,\n+ whether it came from the AE algorithm or not.\n+\n+ \\sa AnalogueGain\n+ \\sa ExposureTimeMode\n \n- \\sa AnalogueGain AeEnable\n+ - ExposureTimeMode:\n+ type: int32_t\n+ description: |\n+ Controls how the frame exposure time is computed. When set to Auto, the\n+ AE algorithm computes the exposure time of the next frame and\n+ configures the image sensor accordingly. When set to Disabled, the AE\n+ algorithm stops updating the exposure time.\n \n- \\todo Document the interactions between AeEnable and setting a fixed\n- value for this control. Consider interactions with other AE features,\n- such as aperture and aperture/shutter priority mode, and decide if\n- control of which features should be automatically adjusted shouldn't\n- better be handled through a separate AE mode control.\n+ As the camera device has a pipeline of in-flight requests, there is a\n+ period of time between submitting a request with ExposureTimeMode set\n+ to Disabled and the exposure time component of the AE actually being\n+ disabled, during which the AE algorithm can still update the exposure\n+ time. If an application is switching from automatic and manual control\n+ and wishes to eliminate any flicker during the switch, the following\n+ procedure is recommended.\n+\n+ 1. Start with ExposureTimeMode set to Auto\n+\n+ 2. Set ExposureTimeMode to Disabled\n+\n+ 3. Wait for the first request to be output that has ExposureTimeMode\n+ set to Disabled\n+\n+ 4. Copy the value reported in ExposureTime into a new request, and\n+ submit it\n+\n+ 5. Proceed to run manual exposure time\n+\n+ \\sa ExposureTime\n+ enum:\n+ - name: ExposureTimeModeAuto\n+ value: 0\n+ description: |\n+ The exposure time will be calculated automatically and set by the\n+ AE algorithm.\n+ - name: ExposureTimeModeDisabled\n+ value: 1\n+ description: |\n+ The exposure time will not be updated by the AE algorithm. It will\n+ come from the last calculated value when the mode was Auto, or from\n+ the value specified in ExposureTime.\n+\n+ When transitioning from Auto to Disabled mode the last computed\n+ exposure value is used until a new value is specified through the\n+ ExposureTime control. If an ExposureTime value is specified in the\n+ same request where the ExposureTimeMode is set to Disabled, the\n+ provided ExposureTime is applied immediately.\n \n - AnalogueGain:\n type: float\n@@ -144,17 +241,65 @@ controls:\n The value of the control specifies the gain multiplier applied to all\n colour channels. This value cannot be lower than 1.0.\n \n- Setting this value means that it is now fixed and the AE algorithm may\n- not change it. Setting it back to zero returns it to the control of the\n- AE algorithm.\n+ This control will only take effect if ExposureTimeMode is Disabled. Its\n+ presence in a request acts as a trigger to switch to the internal\n+ manual mode within ExposureTimeModeDisabled.\n+\n+ When reported in metadata, this control indicates what exposure time\n+ was used for the current request, regardless of ExposureTimeMode.\n+ ExposureTimeMode will indicate the source of the exposure time value,\n+ whether it came from the AE algorithm or not.\n+\n+ \\sa ExposureTime\n+ \\sa AnalogueGainMode\n+\n+ - AnalogueGainMode:\n+ type: int32_t\n+ description: |\n+ Controls how the analogue gain is computed. When set to Auto, the\n+ AE algorithm computes the analogue gain of the next frame and\n+ configures the image sensor accordingly. When set to Disabled, the AE\n+ algorithm stops updating the analogue gain.\n+\n+ As the camera device has a pipeline of in-flight requests, there is a\n+ period of time between submitting a request with AnalogueGainMode set\n+ to Disabled and the analogue gain component of the AE actually being\n+ disabled, during which the AE algorithm can still update the analogue\n+ gain. If an application is switching from automatic and manual control\n+ and wishes to eliminate any flicker during the switch, the following\n+ procedure is recommended.\n+\n+ 1. Start with AnalogueGainMode set to Auto\n \n- \\sa ExposureTime AeEnable\n+ 2. Set AnalogueGainMode to Disabled\n \n- \\todo Document the interactions between AeEnable and setting a fixed\n- value for this control. Consider interactions with other AE features,\n- such as aperture and aperture/shutter priority mode, and decide if\n- control of which features should be automatically adjusted shouldn't\n- better be handled through a separate AE mode control.\n+ 3. Wait for the first request to be output that has AnalogueGainMode\n+ set to Disabled\n+\n+ 4. Copy the value reported in AnalogueGain into a new request, and\n+ submit it\n+\n+ 5. Proceed to run manual analogue gain\n+\n+ \\sa AnalogueGain\n+ enum:\n+ - name: AnalogueGainModeAuto\n+ value: 0\n+ description: |\n+ The analogue gain will be calculated automatically and set by the\n+ AE algorithm.\n+ - name: AnalogueGainModeDisabled\n+ value: 1\n+ description: |\n+ The analogue gain will not be updated by the AE algorithm. It will\n+ come from the last calculated value when the mode was Auto, or from\n+ the value specified in AnalogueGain.\n+\n+ When transitioning from Auto to Disabled mode the last computed\n+ gain value is used until a new value is specified through the\n+ AnalogueGain control. If an AnalogueGain value is specified in the\n+ same request where the AnalogueGainMode is set to Disabled, the\n+ provided AnalogueGain is applied immediately.\n \n - Brightness:\n type: float\n@@ -477,36 +622,6 @@ controls:\n High quality aberration correction which might reduce the frame\n rate.\n \n- - AeState:\n- type: int32_t\n- draft: true\n- description: |\n- Control to report the current AE algorithm state. Currently identical to\n- ANDROID_CONTROL_AE_STATE.\n-\n- Current state of the AE algorithm.\n- enum:\n- - name: AeStateInactive\n- value: 0\n- description: The AE algorithm is inactive.\n- - name: AeStateSearching\n- value: 1\n- description: The AE algorithm has not converged yet.\n- - name: AeStateConverged\n- value: 2\n- description: The AE algorithm has converged.\n- - name: AeStateLocked\n- value: 3\n- description: The AE algorithm is locked.\n- - name: AeStateFlashRequired\n- value: 4\n- description: The AE algorithm would need a flash for good results\n- - name: AeStatePrecapture\n- value: 5\n- description: |\n- The AE algorithm has started a pre-capture metering session.\n- \\sa AePrecaptureTrigger\n-\n - AfState:\n type: int32_t\n draft: true\n", "prefixes": [ "libcamera-devel", "v3", "1/8" ] }