From patchwork Thu Jan 9 00:09:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22474 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DCF0CC32F1 for ; Thu, 9 Jan 2025 00:10:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 26AE5684E2; Thu, 9 Jan 2025 01:10:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UtYlBlLR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E3D9A608AA for ; Thu, 9 Jan 2025 01:09:56 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9272FB63; Thu, 9 Jan 2025 01:09:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381343; bh=G3X6RMG2AaEwGiZQnzYwGnuLX6nucYTWDbKdN5hE76E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UtYlBlLR0CZfMVJOjTZdm8euYFBGJJ3ub4v6aQvGO2nO6SoRpycKA66400gcewd9z 0T5Ax2G1CiqRAuqWrB9m4Td1N7SAiMkGCZVi97ePmMvgOtlRLRAesnU9wGErLzwVuX 9wtBQDoYf5GqBEb+Pv8mqZk1lwaubIruuYZWaFzY= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Jacopo Mondi Subject: [PATCH v6 01/12] controls: Introduce AEGC-related controls Date: Wed, 8 Jan 2025 18:09:31 -0600 Message-Id: <20250109000942.1616565-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Introduce the AeState, ExposureTimeMode and AnalogueGainMode controls to model the AEGC algorithm block. The three controls allow applications to select the exposure time and analogue gain computation calculation mode (auto vs manual) independently from one another, while the AeState control reports the global state for the AEGC algorithm. The new controls are meant to replace the existing AeEnable and AeLocked controls, which are momentarily kept not to break compilation of platforms making use of them. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- No change in v6 Changes in v5: - improve wordings Changes in v4: - fix wordings - add mention on transition from manual to auto - add rules regarding the allowed combinations of control info Changes in v3: - recovered from 2-year-old bitrot --- src/libcamera/control_ids_core.yaml | 265 +++++++++++++++++++++++++-- src/libcamera/control_ids_draft.yaml | 30 --- 2 files changed, 245 insertions(+), 50 deletions(-) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index 1dfaee0c6..553095481 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -28,6 +28,71 @@ controls: \sa AeEnable + - AeState: + type: int32_t + direction: out + description: | + Report the AEGC algorithm state. + + The AEGC algorithm computes the exposure time and the analogue gain + to be applied to the image sensor. + + The AEGC algorithm behaviour is controlled by the ExposureTimeMode and + AnalogueGainMode controls, which allow applications to decide how + the exposure time and gain are computed, in Auto or Manual mode, + independently from one another. + + The AeState control reports the AEGC algorithm state through a single + value and describes it as a single computation block which computes + both the exposure time and the analogue gain values. + + When both the exposure time and analogue gain values are configured to + be in Manual mode, the AEGC algorithm is quiescent and does not actively + compute any value and the AeState control will report AeStateIdle. + + When at least the exposure time or analogue gain are configured to be + computed by the AEGC algorithm, the AeState control will report if the + algorithm has converged to stable values for all of the controls set + to be computed in Auto mode. + + \sa AnalogueGainMode + \sa ExposureTimeMode + + enum: + - name: AeStateIdle + value: 0 + description: | + The AEGC algorithm is inactive. + + This state is returned when both AnalogueGainMode and + ExposureTimeMode are set to Manual and the algorithm is not + actively computing any value. + - name: AeStateSearching + value: 1 + description: | + The AEGC algorithm is actively computing new values, for either the + exposure time or the analogue gain, but has not converged to a + stable result yet. + + This state is returned if at least one of AnalogueGainMode or + ExposureTimeMode is auto and the algorithm hasn't converged yet. + + The AEGC algorithm converges once stable values are computed for + all of the controls set to be computed in Auto mode. Once the + algorithm converges the state is moved to AeStateConverged. + - name: AeStateConverged + value: 2 + description: | + The AEGC algorithm has converged. + + This state is returned if at least one of AnalogueGainMode or + ExposureTimeMode is Auto, and the AEGC algorithm has converged to a + stable value. + + If the measurements move too far away from the convergence point + then the AEGC algorithm might start adjusting again, in which case + the state is moved to AeStateSearching. + # AeMeteringMode needs further attention: # - Auto-generate max enum value. # - Better handling of custom types. @@ -109,6 +174,13 @@ controls: The exposure modes specify how the desired total exposure is divided between the exposure time and the sensor's analogue gain. They are platform specific, and not all exposure modes may be supported. + + When one of AnalogueGainMode or ExposureTimeMode is set to Manual, + the fixed values will override any choices made by AeExposureMode. + + \sa AnalogueGainMode + \sa ExposureTimeMode + enum: - name: ExposureNormal value: 0 @@ -130,13 +202,15 @@ controls: Specify an Exposure Value (EV) parameter. The EV parameter will only be applied if the AE algorithm is currently - enabled. + enabled, that is, at least one of AnalogueGainMode and ExposureTimeMode + are in Auto mode. By convention EV adjusts the exposure as log2. For example EV = [-2, -1, -0.5, 0, 0.5, 1, 2] results in an exposure adjustment of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x]. - \sa AeEnable + \sa AnalogueGainMode + \sa ExposureTimeMode - ExposureTime: type: int32_t @@ -146,17 +220,108 @@ controls: This value is specified in micro-seconds. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Manual. If + this control is set when ExposureTimeMode is Auto, the value will be + ignored and will not be retained. + + When reported in metadata, this control indicates what exposure time + was used for the current frame, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. + + \sa AnalogueGain + \sa ExposureTimeMode + + - ExposureTimeMode: + type: int32_t + direction: inout + description: | + Controls the source of the exposure time that is applied to the image + sensor. + + When set to Auto, the AE algorithm computes the exposure time and + configures the image sensor accordingly. When set to Manual, the value + of the ExposureTime control is used. - \sa AnalogueGain AeEnable + When transitioning from Auto to Manual mode and no ExposureTime control + is provided by the application, the last value computed by the AE + algorithm when the mode was Auto will be used. If the ExposureTimeMode + was never set to Auto (either because the camera started in Manual mode, + or Auto is not supported by the camera), the camera should use a + best-effort default value. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + If ExposureTimeModeManual is supported, the ExposureTime control must + also be supported. + + Cameras that support manual control of the sensor shall support manual + mode for both ExposureTimeMode and AnalogueGainMode, and shall expose + the ExposureTime and AnalogueGain controls. If the camera also has an + AEGC implementation, both ExposureTimeMode and AnalogueGainMode shall + support both manual and auto mode. If auto mode is available, it shall + be the default mode. These rules do not apply to black box cameras + such as UVC cameras, where the available gain and exposure modes are + completely dependent on what the device exposes. + + \par Flickerless exposure mode transitions + + Applications that wish to transition from ExposureTimeModeAuto to direct + control of the exposure time without causing extra flicker can do so by + selecting an ExposureTime value as close as possible to the last value + computed by the auto exposure algorithm in order to avoid any visible + flickering. + + To select the correct value to use as ExposureTime value, applications + should accommodate the natural delay in applying controls caused by the + capture pipeline frame depth. + + When switching to manual exposure mode, applications should not + immediately specify an ExposureTime value in the same request where + ExposureTimeMode is set to Manual. They should instead wait for the + first Request where ExposureTimeMode is reported as + ExposureTimeModeManual in the Request metadata, and use the reported + ExposureTime to populate the control value in the next Request to be + queued to the Camera. + + The implementation of the auto-exposure algorithm should equally try to + minimize flickering and when transitioning from manual exposure mode to + auto exposure use the last value provided by the application as starting + point. + + 1. Start with ExposureTimeMode set to Auto + + 2. Set ExposureTimeMode to Manual + + 3. Wait for the first completed request that has ExposureTimeMode + set to Manual + + 4. Copy the value reported in ExposureTime into a new request, and + submit it + + 5. Proceed to run manual exposure time as desired + + \sa ExposureTime + enum: + - name: ExposureTimeModeAuto + value: 0 + description: | + The exposure time will be calculated automatically and set by the + AE algorithm. + + If ExposureTime is set while this mode is active, it will be + ignored, and its value will not be retained. + + When transitioning from Manual to Auto mode, the AEGC should start + its adjustments based on the last set manual ExposureTime value. + - name: ExposureTimeModeManual + value: 1 + description: | + The exposure time will not be updated by the AE algorithm. + + When transitioning from Auto to Manual mode, the last computed + exposure value is used until a new value is specified through the + ExposureTime control. If an ExposureTime value is specified in the + same request where the ExposureTimeMode is changed from Auto to + Manual, the provided ExposureTime is applied immediately. - AnalogueGain: type: float @@ -167,17 +332,77 @@ controls: The value of the control specifies the gain multiplier applied to all colour channels. This value cannot be lower than 1.0. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if AnalogueGainMode is Manual. If + this control is set when AnalogueGainMode is Auto, the value will be + ignored and will not be retained. + + When reported in metadata, this control indicates what analogue gain + was used for the current request, regardless of AnalogueGainMode. + AnalogueGainMode will indicate the source of the analogue gain value, + whether it came from the AEGC algorithm or not. - \sa ExposureTime AeEnable + \sa ExposureTime + \sa AnalogueGainMode + + - AnalogueGainMode: + type: int32_t + direction: inout + description: | + Controls the source of the analogue gain that is applied to the image + sensor. + + When set to Auto, the AEGC algorithm computes the analogue gain and + configures the image sensor accordingly. When set to Manual, the value + of the AnalogueGain control is used. + + When transitioning from Auto to Manual mode and no AnalogueGain control + is provided by the application, the last value computed by the AEGC + algorithm when the mode was Auto will be used. If the AnalogueGainMode + was never set to Auto (either because the camera started in Manual mode, + or Auto is not supported by the camera), the camera should use a + best-effort default value. + + If AnalogueGainModeManual is supported, the AnalogueGain control must + also be supported. + + For cameras where we have control over the ISP, both ExposureTimeMode + and AnalogueGainMode are expected to support manual mode, and both + controls (as well as ExposureTimeMode and AnalogueGain) are expected to + be present. If the camera also has an AEGC implementation, both + ExposureTimeMode and AnalogueGainMode shall support both manual and + auto mode. If auto mode is available, it shall be the default mode. + These rules do not apply to black box cameras such as UVC cameras, + where the available gain and exposure modes are completely dependent on + what the hardware exposes. + + The same procedure described for performing flickerless transitions in + the ExposureTimeMode control documentation can be applied to analogue + gain. + + \sa ExposureTimeMode + \sa AnalogueGain + enum: + - name: AnalogueGainModeAuto + value: 0 + description: | + The analogue gain will be calculated automatically and set by the + AEGC algorithm. + + If AnalogueGain is set while this mode is active, it will be + ignored, and it will also not be retained. + + When transitioning from Manual to Auto mode, the AEGC should start + its adjustments based on the last set manual AnalogueGain value. + - name: AnalogueGainModeManual + value: 1 + description: | + The analogue gain will not be updated by the AEGC algorithm. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + When transitioning from Auto to Manual mode, the last computed + gain value is used until a new value is specified through the + AnalogueGain control. If an AnalogueGain value is specified in the + same request where the AnalogueGainMode is changed from Auto to + Manual, the provided AnalogueGain is applied immediately. - AeFlickerMode: type: int32_t diff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml index 87e4e02db..03309eeac 100644 --- a/src/libcamera/control_ids_draft.yaml +++ b/src/libcamera/control_ids_draft.yaml @@ -80,36 +80,6 @@ controls: High quality aberration correction which might reduce the frame rate. - - AeState: - type: int32_t - direction: out - description: | - Control to report the current AE algorithm state. Currently identical to - ANDROID_CONTROL_AE_STATE. - - Current state of the AE algorithm. - enum: - - name: AeStateInactive - value: 0 - description: The AE algorithm is inactive. - - name: AeStateSearching - value: 1 - description: The AE algorithm has not converged yet. - - name: AeStateConverged - value: 2 - description: The AE algorithm has converged. - - name: AeStateLocked - value: 3 - description: The AE algorithm is locked. - - name: AeStateFlashRequired - value: 4 - description: The AE algorithm would need a flash for good results - - name: AeStatePrecapture - value: 5 - description: | - The AE algorithm has started a pre-capture metering session. - \sa AePrecaptureTrigger - - AwbState: type: int32_t direction: out From patchwork Thu Jan 9 00:09:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22475 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4CD5C3311 for ; Thu, 9 Jan 2025 00:10:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0CDE868500; Thu, 9 Jan 2025 01:10:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="r4hXAiGL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 94A3068500 for ; Thu, 9 Jan 2025 01:09:58 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AF3916F3; Thu, 9 Jan 2025 01:09:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381345; bh=bG87KFTuvfIXVf0ytUBkGQfGTrJI3PEV9WoDQfPcbj8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r4hXAiGLrexnwg4EGm2h9QgeBIDqtMtiWYwceZQfeM8TGWo2k2vF5v92eKeqA8b5y xNGVu/Aoe4mj5nxzae6kGtzDsq4eDW8OlONrIsAqFDIh0GUMGvIlUW3ohIaCNInxSG KnCV3fxBwquVh0+IfVBdWuQKv9QfmennQJbD7DJo= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Jacopo Mondi Subject: [PATCH v6 02/12] Documentation: design: ae: Document the design for AE controls Date: Wed, 8 Jan 2025 18:09:32 -0600 Message-Id: <20250109000942.1616565-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Document the design and rationale for the AE-related controls. Also add documentation for the controls. Signed-off-by: Paul Elder Signed-off-by: Jacopo Mondi Reviewed-by: Stefan Klug Reviewed-by: Laurent Pinchart --- No change in v6 Changes in v5: - improve wording Changes in v4: - fix wording - fix compilation - remove locked state in explanation Changes in v3: - merge the control documentation into the same document (including the patch) - this is because it was a bit unwieldy to put it in control_ids.cpp.in, now that it's used generically to generate control ids of all namespaces --- Documentation/design/ae.rst | 320 ++++++++++++++++++++++++++++++++++++ Documentation/index.rst | 4 +- Documentation/meson.build | 1 + 3 files changed, 324 insertions(+), 1 deletion(-) create mode 100644 Documentation/design/ae.rst diff --git a/Documentation/design/ae.rst b/Documentation/design/ae.rst new file mode 100644 index 000000000..f8b2c887c --- /dev/null +++ b/Documentation/design/ae.rst @@ -0,0 +1,320 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Design of Exposure and Gain controls +==================================== + +This document explains the design and rationale of the controls related to +exposure and gain. This includes the all-encompassing auto-exposure (AE), the +manual exposure control, and the manual gain control. + +Description of the problem +-------------------------- + +Sub controls +^^^^^^^^^^^^ + +There are more than one control that make up total exposure: exposure time, +gain, and aperture (though for now we will not consider aperture). We already +had individual controls for setting the values of manual exposure and manual +gain, but for switching between auto mode and manual mode we only had a +high-level boolean AeEnable control that would set *both* exposure and gain to +auto mode or manual mode; we had no way to set one to auto and the other to +manual. + +So, we need to introduce two new controls to act as "levers" to indicate +individually for exposure and gain if the value would come from AEGC or if it +would come from the manual control value. + +Aperture priority +^^^^^^^^^^^^^^^^^ + +We eventually may need to support aperture, and so whatever our solution is for +having only some controls on auto and the others on manual needs to be +extensible. + +Flickering when going from auto to manual +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When a manual exposure or gain value is requested by the application, it costs +a few frames worth of time for them to take effect. This means that during a +transition from auto to manual, there would be flickering in the control values +and the transition won't be smooth. + +Take for instance the following flow, where we start on auto exposure (which +for the purposes of the example increments by 1 each frame) and we want to +switch seamlessly to manual exposure, which involves copying the exposure value +computed by the auto exposure algorithm: + +:: + + +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ + | N | | N+1 | | N+2 | | N+3 | | N+4 | | N+5 | | N+6 | + +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ + + Mode requested: Auto Auto Auto Manual Manual Manual Manual + Exp requested: N/A N/A N/A 2 2 2 2 + Set in Frame: N+2 N+3 N+4 N+5 N+6 N+7 N+8 + + Mode used: Auto Auto Auto Auto Auto Manual Manual + Exp used: 0 1 2 3 4 2 2 + +As we can see, after frame N+2 completes, we copy the exposure value that was +used for frame N+2 (which was computed by AE algorithm), and queue that value +into request N+3 with manual mode on. However, as it takes two frames for the +exposure to be set, the exposure still changes since it is set by AE, and we +get a flicker in the exposure during the switch from auto to manual. + +A solution is to *not submit* any exposure value when manual mode is enabled, +and wait until the manual mode as been "applied" before copying the exposure +value: + +:: + + +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ + | N | | N+1 | | N+2 | | N+3 | | N+4 | | N+5 | | N+6 | + +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ + + Mode requested: Auto Auto Auto Manual Manual Manual Manual + Exp requested: N/A N/A N/A None None None 5 + Set in Frame: N+2 N+3 N+4 N+5 N+6 N+7 N+8 + + Mode used: Auto Auto Auto Auto Auto Manual Manual + Exp used: 0 1 2 3 4 5 5 + +In practice, this works. However, libcamera has a policy where once a control +is submitted, its value is saved and does not need to be resubmitted. If the +manual exposure value was set while auto mode was on, in theory the value would +be saved, so when manual mode is enabled, the exposure value that was +previously set would immediately be used. Clearly this solution isn't correct, +but it can serve as the basis for a proper solution, with some more rigorous +rules. + +Existing solutions +------------------ + +Raspberry Pi +^^^^^^^^^^^^ + +The Raspberry Pi IPA gets around the lack of individual AeEnable controls for +exposure and gain by using magic values. When AeEnable is false, if one of the +manual control values was set to 0 then the value computed by AEGC would be +used for just that control. This solution isn't desirable, as it prevents +that magic value from being used as a valid value. + +To get around the flickering issue, when AeEnable is false, the Raspberry Pi +AEGC simply stops updating the values to be set, without restoring the +previously set manual exposure time and gain. This works, but is not a proper +solution. + +Android +^^^^^^^ + +The Android HAL specification requires that exposure and gain (sensitivity) +must both be manual or both be auto. It cannot be that one is manual while the +other is auto, so they simply don't support sub controls. + +For the flickering issue, the Android HAL has an AeLock control. To transition +from auto to manual, the application would keep AE on auto, and turn on the +lock. Once the lock has propagated through, then the value can be copied from +the result into the request and the lock disabled and the mode set to manual. + +The problem with this solution is, besides the extra complexity, that it is +ambiguous what happens if there is a state transition from manual to locked +(even though it's a state transition that doesn't make sense). If locked is +defined to "use the last automatically computed values" then it could use the +values from the last time it AE was set to auto, or it would be undefined if AE +was never auto (eg. it started out as manual), or if AE is implemented to run +in the background it could just use the current values that are computed. If +locked is defined to "use the last value that was set" there would be less +ambiguity. Still, it's better if we can make it impossible to execute this +nonsensical state transition, and if we can reduce the complexity of having +this extra control or extra setting on a lever. + +Summary of goals +---------------- + +- We need a lock of some sort, to instruct the AEGC to not update output + results + +- We need manual modes, to override the values computed by the AEGC + +- We need to support seamless transitions from auto to manual, and do so + without flickering + +- We need custom minimum values for the manual controls; that is, no magic + values for enabling/disabling auto + +- All of these need to be done with AE sub-controls (exposure time, analogue + gain) and be extensible to aperture in the future + +Our solution +------------ + +A diagram of our solution: + +:: + + +----------------------------+-------------+------------------+-----------------+ + | INPUT | ALGORITHM | RESULT | OUTPUT | + +----------------------------+-------------+------------------+-----------------+ + + ExposureTimeMode ExposureTimeMode + ---------------------+----------------------------------------+-----------------> + 0: Auto | | + 1: Manual | V + | |\ + | | \ + | /----------------------------------> | 1| ExposureTime + | | +-------------+ exposure time | | --------------> + \--)--> | | --------------> | 0| + ExposureTime | | | | / + ------------------------+--> | | |/ + | | AeState + | AEGC | -----------------------------------> + AnalogueGain | | + ------------------------+--> | | |\ + | | | | \ + /--)--> | | --------------> | 0| AnalogueGain + | | +-------------+ analogue gain | | --------------> + | \----------------------------------> | 1| + | | / + | |/ + | ^ + AnalogueGainMode | | AnalogueGainMode + ---------------------+----------------------------------------+-----------------> + 0: Auto + 1: Manual + + +The diagram is divided in four sections horizontally: + +- Input: The values received from the request controls + +- Algorithm: The algorithm itself + +- Result: The values calculated by the algorithm + +- Output: The values reported in result metadata and applied to the device + +The four input controls are divided between manual values (ExposureTime and +AnalogueGain), and operation modes (ExposureTimeMode and AnalogueGainMode). The +former are the manual values, the latter control how they're applied. The two +modes are independent from each other, and each can take one of two values: + + - Auto (0): The AGC computes the value normally. The AGC result is applied + to the output. The manual value is ignored *and is not retained*. + + - Manual (1): The AGC uses the manual value internally. The corresponding + manual control from the request is applied to the output. The AGC result + is ignored. + +The AeState control reports the state of the unified AEGC block. If both +ExposureTimeMode and AnalogueGainMode are set to manual then it will report +Idle. If at least one of the two is set to auto, then AeState will report +if the AEGC has Converged or not (Searching). This control replaces the old +AeLocked control, as it was insufficient for reporting the AE state. + +There is a caveat to manual mode: the manual control value is not retained if +it is set during auto mode. This means that if manual mode is entered without +also setting the manual value, then it will enter a state similar to "locked", +where the last automatically computed value while the mode was auto will be +used. Once the manual value is set, then that will be used and retained as +usual. + +This simulates an auto -> locked -> manual or auto -> manual state transition, +and makes it impossible to do the nonsensical manual -> locked state +transition. + +We specifically do not have a "master AE control" like the old AeEnable. This +is because we have the individual mode controls, and if we had a master AE +control it would be a "control that sets other controls", which could easily +get out of control. + +With this solution, the earlier example would become: + +:: + + +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ + | N+2 | | N+3 | | N+4 | | N+5 | | N+6 | | N+7 | | N+8 | | N+9 | | N+10| + +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ + Mode requested: Auto Manual Manual Manual Manual Manual Manual Manual Manual + Exp requested: N/A None None None None 10 None 10 10 + Set in Frame: N+4 N+5 N+6 N+7 N+8 N+9 N+10 N+11 N+12 + + Mode used: Auto Auto Auto Manual Manual Manual Manual Manual Manual + Exp used: 2 3 4 5 5 5 5 10 10 + +This example is extended by a few frames to exhibit the simulated "locked" +state. At frame N+5 the application has confirmed that the manual mode has been +entered, but does not provide a manual value until request N+7. Thus, the value +that is used in requests N+5 and N+6 (where the mode is disabled), comes from +the last value that was used when the mode was auto, which comes from frame +N+4. + +Then, in N+7, a manual value of 10 is supplied. It takes until frame N+9 for +the exposure to be applied. N+8 does not supply a manual value, but the last +supplied value is retained, so a manual value of 10 is still used and set in +frame N+10. + +Although this behavior is the same as what we had with waiting for the manual +mode to propagate (in the section "Description of the problem"), this time it +is correct as we have defined specifically that if a manual value was specified +while the mode was auto, it will not be retained. + +Description of the controls +--------------------------- + +As described above, libcamera offers the following controls related to exposure +and gain: + +- AnalogueGain + +- AnalogueGainMode + +- ExposureTime + +- ExposureTimeMode + +- AeState + +Auto-exposure and auto-gain can be enabled and disabled separately using the +ExposureTimeMode and AnalogueGainMode controls respectively. There is no +overarching AeEnable control. + +When the respective mode is set to auto, the respective value that is computed +by the AEGC algorithm is applied to the image sensor. Any value that is +supplied in the manual ExposureTime/AnalogueGain control is ignored and not +retained. Another way to understand this is that when the mode transitions from +auto to manual, the internally stored control value is overwritten with the +last value computed by the auto algorithm. + +This means that when we transition from auto to manual without supplying a +manual control value, the last value that was set by the AEGC algorithm will +keep be used. This can be used to do a flickerless transition from auto to +manual as described earlier. If the camera started out in manual mode and no +corresponding value has been supplied yet, then a best-effort default value +shall be set. + +The manual control value can be set in the same request as setting the mode to +auto if the desired manual control value is already known. + +Transitioning from manual to auto shall be implicitly flickerless, as the AEGC +algorithms are expected to start running from the last manual value. + +The AeState metadata reports the state of the AE algorithm. As AE cannot +compute exposure and gain separately, the state of the AE component is +unified. There are three states: Idle, Searching, and Converged. + +The state shall be Idle if both ExposureTimeMode and AnalogueGainMode +are set to Manual. If the camera only supports one of the two controls, +then the state shall be Idle if that one control is set to Manual. If +the camera does not support Manual for at least one of the two controls, +then the state will never be Idle, as AE will always be running. + +The state shall be Searching if at least one of exposure or gain calculated +by the AE algorithm is used (that is, at least one of the two modes is Auto), +*and* the value(s) have not converged yet. + +The state shall be Converged if at least one of exposure or gain calculated +by the AE algorithm is used (that is, at least one of the two modes is Auto), +*and* the value(s) have converged. diff --git a/Documentation/index.rst b/Documentation/index.rst index bea406608..251112fbd 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -23,7 +23,9 @@ SoftwareISP Benchmarking Tracing guide + Design document: AE + .. toctree:: :hidden: - introduction \ No newline at end of file + introduction diff --git a/Documentation/meson.build b/Documentation/meson.build index 36ffae239..6158320e1 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -128,6 +128,7 @@ if sphinx.found() 'coding-style.rst', 'conf.py', 'contributing.rst', + 'design/ae.rst', 'documentation-contents.rst', 'environment_variables.rst', 'feature_requirements.rst', From patchwork Thu Jan 9 00:09:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 85DC9C326B for ; Thu, 9 Jan 2025 00:10:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 00BDE6851C; Thu, 9 Jan 2025 01:10:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="etClc2Zt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F250268506 for ; Thu, 9 Jan 2025 01:09:59 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ED681CE6; Thu, 9 Jan 2025 01:09:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381346; bh=1hxF9pkFGiPjQjZWnPUEqZ49YEmNr9MqCbryJwIfgeE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=etClc2Ztn9RyJgDi5sDlqUITkW0oi5A+Ufv/o0PoNsa7EYQHFhVgdK0eD9Kbebbmm /BRbwvPzvuUO9Ju5OPN3c7u4dzoJ8vza8+ykSOiKLJRrgolUIBp/9JqWJmfNvhXr/k Mrcaj37NEa/sLXMtcddALJWQYAMv/EuYIbsSqjqU= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Paul Elder Subject: [PATCH v6 03/12] libcamera: uvcvideo: Register ExposureTimeMode control Date: Wed, 8 Jan 2025 18:09:33 -0600 Message-Id: <20250109000942.1616565-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Port the UVC pipeline handler to use the new ExposureTimeMode control when processing Camera controls in place of the AeEnable control. The V4L2_CID_EXPOSURE_AUTO control allows 4 possible values, which map to ExposureTimeModeAuto and ExposureTimeModeManual. Signed-off-by: Jacopo Mondi Signed-off-by: Paul Elder Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- No change in v6 Changes in v5: - remove unnecessary intermediate variable No change in v3 --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 53 ++++++++++++++++++-- 1 file changed, 48 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 8c2c6baf3..7470b5627 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -298,7 +298,7 @@ int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id, cid = V4L2_CID_CONTRAST; else if (id == controls::Saturation) cid = V4L2_CID_SATURATION; - else if (id == controls::AeEnable) + else if (id == controls::ExposureTimeMode) cid = V4L2_CID_EXPOSURE_AUTO; else if (id == controls::ExposureTime) cid = V4L2_CID_EXPOSURE_ABSOLUTE; @@ -647,7 +647,7 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, id = &controls::Saturation; break; case V4L2_CID_EXPOSURE_AUTO: - id = &controls::AeEnable; + id = &controls::ExposureTimeMode; break; case V4L2_CID_EXPOSURE_ABSOLUTE: id = &controls::ExposureTime; @@ -660,6 +660,7 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, } /* Map the control info. */ + const std::vector &v4l2Values = v4l2Info.values(); int32_t min = v4l2Info.min().get(); int32_t max = v4l2Info.max().get(); int32_t def = v4l2Info.def().get(); @@ -697,10 +698,52 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, }; break; - case V4L2_CID_EXPOSURE_AUTO: - info = ControlInfo{ false, true, true }; + case V4L2_CID_EXPOSURE_AUTO: { + /* + * From the V4L2_CID_EXPOSURE_AUTO documentation: + * + * ------------------------------------------------------------ + * V4L2_EXPOSURE_AUTO: + * Automatic exposure time, automatic iris aperture. + * + * V4L2_EXPOSURE_MANUAL: + * Manual exposure time, manual iris. + * + * V4L2_EXPOSURE_SHUTTER_PRIORITY: + * Manual exposure time, auto iris. + * + * V4L2_EXPOSURE_APERTURE_PRIORITY: + * Auto exposure time, manual iris. + *------------------------------------------------------------- + * + * ExposureTimeModeAuto = { V4L2_EXPOSURE_AUTO, + * V4L2_EXPOSURE_APERTURE_PRIORITY } + * + * + * ExposureTimeModeManual = { V4L2_EXPOSURE_MANUAL, + * V4L2_EXPOSURE_SHUTTER_PRIORITY } + */ + std::array values{}; + + auto it = std::find_if(v4l2Values.begin(), v4l2Values.end(), + [&](const ControlValue &val) { + return (val.get() == V4L2_EXPOSURE_APERTURE_PRIORITY || + val.get() == V4L2_EXPOSURE_AUTO) ? true : false; + }); + if (it != v4l2Values.end()) + values.back() = static_cast(controls::ExposureTimeModeAuto); + + it = std::find_if(v4l2Values.begin(), v4l2Values.end(), + [&](const ControlValue &val) { + return (val.get() == V4L2_EXPOSURE_SHUTTER_PRIORITY || + val.get() == V4L2_EXPOSURE_MANUAL) ? true : false; + }); + if (it != v4l2Values.end()) + values.back() = static_cast(controls::ExposureTimeModeManual); + + info = ControlInfo{Span{values}, values[0]}; break; - + } case V4L2_CID_EXPOSURE_ABSOLUTE: /* * ExposureTime is in units of 1 µs, and UVC expects From patchwork Thu Jan 9 00:09:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22477 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A6CF5C3312 for ; Thu, 9 Jan 2025 00:10:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1745668545; Thu, 9 Jan 2025 01:10:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VlBbbup6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5D8C368516 for ; Thu, 9 Jan 2025 01:10:01 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 37AEF6F3; Thu, 9 Jan 2025 01:09:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381348; bh=FiB6dIMVE82zTD4ygQ5EE7hLiIQWlCW/iT+9TrFi1R8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VlBbbup6yX+XZ6emWCS/pTK+IhFTUNmwErW5YBbzlaXfyp0K106hiyrujE0f8y0li EgHT2sCgtXeKBtFPrJntyFBEX8HUjfNLOg2NmB7sS93TQ98J9xZFrqUFOcVinEjE8K 2Mt5+r5wA49GGSIGjQDCYINXRk1n5T8yevnNH0cM= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Paul Elder Subject: [PATCH v6 04/12] test: ipa_data_serialization: Use DebugMetadataEnable Date: Wed, 8 Jan 2025 18:09:34 -0600 Message-Id: <20250109000942.1616565-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Replace the deprecated AeEnable control with DebugMetadataEnable in ipa_data_serialization test. We use DebugMetadataEnable instead of one of the controls replacing AeEnable as they are not boolean controls. Signed-off-by: Jacopo Mondi Signed-off-by: Paul Elder Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- No change in v6 No change in v5 Changes in v4: - s/ExposureTimeMode/DebugMetadataEnable/ No change in v3 --- test/serialization/ipa_data_serializer_test.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp index aea63c731..afea93a6c 100644 --- a/test/serialization/ipa_data_serializer_test.cpp +++ b/test/serialization/ipa_data_serializer_test.cpp @@ -29,7 +29,7 @@ using namespace std; using namespace libcamera; static const ControlInfoMap Controls = ControlInfoMap({ - { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::DebugMetadataEnable, ControlInfo(false, true) }, { &controls::ExposureTime, ControlInfo(0, 999999) }, { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, From patchwork Thu Jan 9 00:09:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22478 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E0A3CC3313 for ; Thu, 9 Jan 2025 00:10:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E0566851F; Thu, 9 Jan 2025 01:10:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sw+hHDCf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1874268503 for ; Thu, 9 Jan 2025 01:10:03 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B3C02CE6; Thu, 9 Jan 2025 01:09:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381349; bh=re7XBdUAcDc1XhHzzZTYKUf8IKFjUw7Y9KzJxiCiThU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sw+hHDCfpNpTtGTLkq5rp1C+Xvyj1WScxp4xZekSfIeaAPdljJ/sHhlvV/9TNLR+R oXiDjU2O0TDp9SKmE2E0b4/0aKm5N5vBEwd5pL+vde1LD9TbbXlvN75XLEoYnGj+rn 5FTw63WxjuNs/vS/cDKpNtPAMEDsGgJkki6KtoLA= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Paul Elder Subject: [PATCH v6 05/12] ipa: raspberry: Port to the new AEGC controls Date: Wed, 8 Jan 2025 18:09:35 -0600 Message-Id: <20250109000942.1616565-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The newly introduced controls to drive the AEGC algorithm allow to control the computation of the exposure time and analogue gain separately. The RPi AEGC implementation already computes the shutter and gain values separately but does not expose separate functions to control them. Augment the AgcAlgorithm interface to allow pausing/resuming the shutter and gain automatic computations separately and plumb them to the newly introduced controls. Add safety checks to ignore ExposureTime and AnalogueGain values if the algorithms are not paused, and report the correct AeState value by checking if both algorithms have been paused or if they have converged. Signed-off-by: Jacopo Mondi Signed-off-by: Paul Elder Reviewed-by: Paul Elder --- No change in v6 No change in v5 Changes in v4: - s/shutter/exposure/ Changes in v3: - recovered from 2-year-old bitrot --- src/ipa/rpi/common/ipa_base.cpp | 74 +++++++++++++++++++--- src/ipa/rpi/controller/agc_algorithm.h | 8 ++- src/ipa/rpi/controller/rpi/agc.cpp | 52 +++++++++++++-- src/ipa/rpi/controller/rpi/agc.h | 8 ++- src/ipa/rpi/controller/rpi/agc_channel.cpp | 24 ++++++- src/ipa/rpi/controller/rpi/agc_channel.h | 8 ++- 6 files changed, 150 insertions(+), 24 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 0c8aee699..4334d22e1 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -55,8 +55,13 @@ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; /* List of controls handled by the Raspberry Pi IPA */ const ControlInfoMap::Map ipaControls{ - { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::ExposureTimeMode, + ControlInfo(static_cast(controls::ExposureTimeModeAuto), + static_cast(controls::ExposureTimeModeManual)) }, { &controls::ExposureTime, ControlInfo(0, 66666) }, + { &controls::AnalogueGainMode, + ControlInfo(static_cast(controls::AnalogueGainModeAuto), + static_cast(controls::AnalogueGainModeManual)) }, { &controls::AnalogueGain, ControlInfo(1.0f, 16.0f) }, { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, @@ -755,21 +760,22 @@ void IpaBase::applyControls(const ControlList &controls) << " = " << ctrl.second.toString(); switch (ctrl.first) { - case controls::AE_ENABLE: { + case controls::EXPOSURE_TIME_MODE: { RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); if (!agc) { LOG(IPARPI, Warning) - << "Could not set AE_ENABLE - no AGC algorithm"; + << "Could not set EXPOSURE_TIME_MODE - no AGC algorithm"; break; } - if (ctrl.second.get() == false) - agc->disableAuto(); + if (ctrl.second.get() == controls::ExposureTimeModeManual) + agc->disableAutoExposure(); else - agc->enableAuto(); + agc->enableAutoExposure(); - libcameraMetadata_.set(controls::AeEnable, ctrl.second.get()); + libcameraMetadata_.set(controls::ExposureTimeMode, + ctrl.second.get()); break; } @@ -782,6 +788,13 @@ void IpaBase::applyControls(const ControlList &controls) break; } + /* + * Ignore manual exposure time when the auto exposure + * algorithm is running. + */ + if (agc->autoExposureEnabled()) + break; + /* The control provides units of microseconds. */ agc->setFixedExposureTime(0, ctrl.second.get() * 1.0us); @@ -789,6 +802,25 @@ void IpaBase::applyControls(const ControlList &controls) break; } + case controls::ANALOGUE_GAIN_MODE: { + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.getAlgorithm("agc")); + if (!agc) { + LOG(IPARPI, Warning) + << "Could not set ANALOGUE_GAIN_MODE - no AGC algorithm"; + break; + } + + if (ctrl.second.get() == controls::AnalogueGainModeManual) + agc->disableAutoGain(); + else + agc->enableAutoGain(); + + libcameraMetadata_.set(controls::AnalogueGainMode, + ctrl.second.get()); + break; + } + case controls::ANALOGUE_GAIN: { RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); @@ -798,6 +830,13 @@ void IpaBase::applyControls(const ControlList &controls) break; } + /* + * Ignore manual analogue gain value when the auto gain + * algorithm is running. + */ + if (agc->autoGainEnabled()) + break; + agc->setFixedAnalogueGain(0, ctrl.second.get()); libcameraMetadata_.set(controls::AnalogueGain, @@ -854,6 +893,13 @@ void IpaBase::applyControls(const ControlList &controls) break; } + /* + * Ignore AE_EXPOSURE_MODE if the shutter or the gain + * are in auto mode. + */ + if (agc->autoExposureEnabled() || agc->autoGainEnabled()) + break; + int32_t idx = ctrl.second.get(); if (ExposureModeTable.count(idx)) { agc->setExposureMode(ExposureModeTable.at(idx)); @@ -1333,9 +1379,19 @@ void IpaBase::reportMetadata(unsigned int ipaContext) } AgcPrepareStatus *agcPrepareStatus = rpiMetadata.getLocked("agc.prepare_status"); - if (agcPrepareStatus) { - libcameraMetadata_.set(controls::AeLocked, agcPrepareStatus->locked); + if (agcPrepareStatus) libcameraMetadata_.set(controls::DigitalGain, agcPrepareStatus->digitalGain); + + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.getAlgorithm("agc")); + if (agc) { + if (!agc->autoExposureEnabled() && !agc->autoGainEnabled()) + libcameraMetadata_.set(controls::AeState, controls::AeStateIdle); + else if (agcPrepareStatus) + libcameraMetadata_.set(controls::AeState, + agcPrepareStatus->locked + ? controls::AeStateConverged + : controls::AeStateSearching); } LuxStatus *luxStatus = rpiMetadata.getLocked("lux.status"); diff --git a/src/ipa/rpi/controller/agc_algorithm.h b/src/ipa/rpi/controller/agc_algorithm.h index c97828577..fdaa10e6c 100644 --- a/src/ipa/rpi/controller/agc_algorithm.h +++ b/src/ipa/rpi/controller/agc_algorithm.h @@ -30,8 +30,12 @@ public: virtual void setMeteringMode(std::string const &meteringModeName) = 0; virtual void setExposureMode(std::string const &exposureModeName) = 0; virtual void setConstraintMode(std::string const &contraintModeName) = 0; - virtual void enableAuto() = 0; - virtual void disableAuto() = 0; + virtual void enableAutoExposure() = 0; + virtual void disableAutoExposure() = 0; + virtual bool autoExposureEnabled() const = 0; + virtual void enableAutoGain() = 0; + virtual void disableAutoGain() = 0; + virtual bool autoGainEnabled() const = 0; virtual void setActiveChannels(const std::vector &activeChannels) = 0; }; diff --git a/src/ipa/rpi/controller/rpi/agc.cpp b/src/ipa/rpi/controller/rpi/agc.cpp index c48fdf156..02bfdb4a5 100644 --- a/src/ipa/rpi/controller/rpi/agc.cpp +++ b/src/ipa/rpi/controller/rpi/agc.cpp @@ -74,22 +74,62 @@ int Agc::checkChannel(unsigned int channelIndex) const return 0; } -void Agc::disableAuto() +void Agc::disableAutoExposure() { - LOG(RPiAgc, Debug) << "disableAuto"; + LOG(RPiAgc, Debug) << "disableAutoExposure"; /* All channels are enabled/disabled together. */ for (auto &data : channelData_) - data.channel.disableAuto(); + data.channel.disableAutoExposure(); } -void Agc::enableAuto() +void Agc::enableAutoExposure() { - LOG(RPiAgc, Debug) << "enableAuto"; + LOG(RPiAgc, Debug) << "enableAutoExposure"; /* All channels are enabled/disabled together. */ for (auto &data : channelData_) - data.channel.enableAuto(); + data.channel.enableAutoExposure(); +} + +bool Agc::autoExposureEnabled() const +{ + LOG(RPiAgc, Debug) << "autoExposureEnabled"; + + /* + * We always have at least one channel, and since all channels are + * enabled and disabled together we can simply check the first one. + */ + return channelData_[0].channel.autoExposureEnabled(); +} + +void Agc::disableAutoGain() +{ + LOG(RPiAgc, Debug) << "disableAutoGain"; + + /* All channels are enabled/disabled together. */ + for (auto &data : channelData_) + data.channel.disableAutoGain(); +} + +void Agc::enableAutoGain() +{ + LOG(RPiAgc, Debug) << "enableAutoGain"; + + /* All channels are enabled/disabled together. */ + for (auto &data : channelData_) + data.channel.enableAutoGain(); +} + +bool Agc::autoGainEnabled() const +{ + LOG(RPiAgc, Debug) << "autoGainEnabled"; + + /* + * We always have at least one channel, and since all channels are + * enabled and disabled together we can simply check the first one. + */ + return channelData_[0].channel.autoGainEnabled(); } unsigned int Agc::getConvergenceFrames() const diff --git a/src/ipa/rpi/controller/rpi/agc.h b/src/ipa/rpi/controller/rpi/agc.h index 3aca000bb..c3a940bf6 100644 --- a/src/ipa/rpi/controller/rpi/agc.h +++ b/src/ipa/rpi/controller/rpi/agc.h @@ -40,8 +40,12 @@ public: void setMeteringMode(std::string const &meteringModeName) override; void setExposureMode(std::string const &exposureModeName) override; void setConstraintMode(std::string const &contraintModeName) override; - void enableAuto() override; - void disableAuto() override; + void enableAutoExposure() override; + void disableAutoExposure() override; + bool autoExposureEnabled() const override; + void enableAutoGain() override; + void disableAutoGain() override; + bool autoGainEnabled() const override; void switchMode(CameraMode const &cameraMode, Metadata *metadata) override; void prepare(Metadata *imageMetadata) override; void process(StatisticsPtr &stats, Metadata *imageMetadata) override; diff --git a/src/ipa/rpi/controller/rpi/agc_channel.cpp b/src/ipa/rpi/controller/rpi/agc_channel.cpp index 79c459735..e79184b7a 100644 --- a/src/ipa/rpi/controller/rpi/agc_channel.cpp +++ b/src/ipa/rpi/controller/rpi/agc_channel.cpp @@ -319,18 +319,36 @@ int AgcChannel::read(const libcamera::YamlObject ¶ms, return 0; } -void AgcChannel::disableAuto() +void AgcChannel::disableAutoExposure() { fixedExposureTime_ = status_.exposureTime; - fixedAnalogueGain_ = status_.analogueGain; } -void AgcChannel::enableAuto() +void AgcChannel::enableAutoExposure() { fixedExposureTime_ = 0s; +} + +bool AgcChannel::autoExposureEnabled() const +{ + return fixedExposureTime_ == 0s; +} + +void AgcChannel::disableAutoGain() +{ + fixedAnalogueGain_ = status_.analogueGain; +} + +void AgcChannel::enableAutoGain() +{ fixedAnalogueGain_ = 0; } +bool AgcChannel::autoGainEnabled() const +{ + return fixedAnalogueGain_ == 0; +} + unsigned int AgcChannel::getConvergenceFrames() const { /* diff --git a/src/ipa/rpi/controller/rpi/agc_channel.h b/src/ipa/rpi/controller/rpi/agc_channel.h index 734e5efd3..fa697e6fa 100644 --- a/src/ipa/rpi/controller/rpi/agc_channel.h +++ b/src/ipa/rpi/controller/rpi/agc_channel.h @@ -96,8 +96,12 @@ public: void setMeteringMode(std::string const &meteringModeName); void setExposureMode(std::string const &exposureModeName); void setConstraintMode(std::string const &contraintModeName); - void enableAuto(); - void disableAuto(); + void enableAutoExposure(); + void disableAutoExposure(); + bool autoExposureEnabled() const; + void enableAutoGain(); + void disableAutoGain(); + bool autoGainEnabled() const; void switchMode(CameraMode const &cameraMode, Metadata *metadata); void prepare(Metadata *imageMetadata); void process(StatisticsPtr &stats, DeviceStatus const &deviceStatus, Metadata *imageMetadata, From patchwork Thu Jan 9 00:09:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22479 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2E5BC3314 for ; Thu, 9 Jan 2025 00:10:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 622E06851D; Thu, 9 Jan 2025 01:10:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JhkYuRlt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 64B4668506 for ; Thu, 9 Jan 2025 01:10:04 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 51A9DB63; Thu, 9 Jan 2025 01:09:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381351; bh=rm1dc1DyZXC4DygiRRk1sVoAZ8pv1pFQsdP1hAH0GS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JhkYuRltkEdH1e+jCtgPYQjvDanfl8afuu+gv6rMljqaGuAPGud58+08Ypw2Z+k0h TLqrqYKsQGS9ohsQJH+vhIXtt2QQYWb4x4Kv1JThrBUpwVMDL+p2Hy1EgXBSE++pDR j8Pgvch2HyTGKmuc4SSP0LguJX4uYyjcdMWziHOQ= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com Subject: [PATCH v6 06/12] ipa: rkisp1: Port to the new AEGC controls Date: Wed, 8 Jan 2025 18:09:36 -0600 Message-Id: <20250109000942.1616565-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The newly introduced controls to drive the AEGC algorithm allow controlling the computation of the exposure time and analogue gain separately. Augument the RkISP1 AEGC implementation to handle the exposure and gain controls separately using the new controls. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- No change in v6 Changes in v5: - fix mixed manual-gain ae modes - improve documentation for mode change flags in the ipa context Changes in v4: - make auto the default value in the control infos - implement the expected behavior when transitioning between manual and auto modes New in v3 Back 2 years ago in v2 RkISP1 didn't yet support AeEnable properly yet so the AeEnable control was simply removed. --- src/ipa/rkisp1/algorithms/agc.cpp | 131 ++++++++++++++++++++++++------ src/ipa/rkisp1/ipa_context.cpp | 24 +++++- src/ipa/rkisp1/ipa_context.h | 8 +- 3 files changed, 133 insertions(+), 30 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 40e5a8f48..49c107c99 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -148,7 +148,14 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) if (ret) return ret; - context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true); + context.ctrlMap[&controls::ExposureTimeMode] = + ControlInfo(static_cast(controls::ExposureTimeModeAuto), + static_cast(controls::ExposureTimeModeManual), + static_cast(controls::ExposureTimeModeAuto)); + context.ctrlMap[&controls::AnalogueGainMode] = + ControlInfo(static_cast(controls::AnalogueGainModeAuto), + static_cast(controls::AnalogueGainModeManual), + static_cast(controls::AnalogueGainModeAuto)); context.ctrlMap.merge(controls()); return 0; @@ -169,7 +176,8 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) 10ms / context.configuration.sensor.lineDuration; context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; - context.activeState.agc.autoEnabled = !context.configuration.raw; + context.activeState.agc.autoExposureEnabled = !context.configuration.raw; + context.activeState.agc.autoGainEnabled = !context.configuration.raw; context.activeState.agc.constraintMode = static_cast(constraintModes().begin()->first); @@ -215,18 +223,47 @@ void Agc::queueRequest(IPAContext &context, auto &agc = context.activeState.agc; if (!context.configuration.raw) { - const auto &agcEnable = controls.get(controls::AeEnable); - if (agcEnable && *agcEnable != agc.autoEnabled) { - agc.autoEnabled = *agcEnable; + const auto &aeEnable = controls.get(controls::ExposureTimeMode); + if (aeEnable && + (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { + agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); LOG(RkISP1Agc, Debug) - << (agc.autoEnabled ? "Enabling" : "Disabling") - << " AGC"; + << (agc.autoExposureEnabled ? "Enabling" : "Disabling") + << " AGC (exposure)"; + + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + * + * \todo Check the previous frame at prepare() time + * instead of saving a flag here + */ + if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) + frameContext.agc.autoExposureModeChange = true; + } + + const auto &agEnable = controls.get(controls::AnalogueGainMode); + if (agEnable && + (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { + agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); + + LOG(RkISP1Agc, Debug) + << (agc.autoGainEnabled ? "Enabling" : "Disabling") + << " AGC (gain)"; + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + */ + if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) + frameContext.agc.autoGainModeChange = true; } } const auto &exposure = controls.get(controls::ExposureTime); - if (exposure && !agc.autoEnabled) { + if (exposure && !agc.autoExposureEnabled) { agc.manual.exposure = *exposure * 1.0us / context.configuration.sensor.lineDuration; @@ -235,18 +272,19 @@ void Agc::queueRequest(IPAContext &context, } const auto &gain = controls.get(controls::AnalogueGain); - if (gain && !agc.autoEnabled) { + if (gain && !agc.autoGainEnabled) { agc.manual.gain = *gain; LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; } - frameContext.agc.autoEnabled = agc.autoEnabled; + frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; + frameContext.agc.autoGainEnabled = agc.autoGainEnabled; - if (!frameContext.agc.autoEnabled) { + if (!frameContext.agc.autoExposureEnabled) frameContext.agc.exposure = agc.manual.exposure; + if (!frameContext.agc.autoGainEnabled) frameContext.agc.gain = agc.manual.gain; - } const auto &meteringMode = controls.get(controls::AeMeteringMode); if (meteringMode) { @@ -283,9 +321,26 @@ void Agc::queueRequest(IPAContext &context, void Agc::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) { - if (frameContext.agc.autoEnabled) { - frameContext.agc.exposure = context.activeState.agc.automatic.exposure; - frameContext.agc.gain = context.activeState.agc.automatic.gain; + uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; + double activeAutoGain = context.activeState.agc.automatic.gain; + + /* Populate exposure and gain in auto mode */ + if (frameContext.agc.autoExposureEnabled) + frameContext.agc.exposure = activeAutoExposure; + if (frameContext.agc.autoGainEnabled) + frameContext.agc.gain = activeAutoGain; + + /* + * Populate manual exposure and gain from the active auto values when + * transitioning from auto to manual + */ + if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { + context.activeState.agc.manual.exposure = activeAutoExposure; + frameContext.agc.exposure = activeAutoExposure; + } + if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { + context.activeState.agc.manual.gain = activeAutoGain; + frameContext.agc.gain = activeAutoGain; } if (frame > 0 && !frameContext.agc.updateMetering) @@ -333,7 +388,14 @@ void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, * frameContext.sensor.exposure; metadata.set(controls::AnalogueGain, frameContext.sensor.gain); metadata.set(controls::ExposureTime, exposureTime.get()); - metadata.set(controls::AeEnable, frameContext.agc.autoEnabled); + metadata.set(controls::ExposureTimeMode, + frameContext.agc.autoExposureEnabled + ? controls::ExposureTimeModeAuto + : controls::ExposureTimeModeManual); + metadata.set(controls::AnalogueGainMode, + frameContext.agc.autoGainEnabled + ? controls::AnalogueGainModeAuto + : controls::AnalogueGainModeManual); /* \todo Use VBlank value calculated from each frame exposure. */ uint32_t vTotal = context.configuration.sensor.size.height @@ -424,14 +486,35 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, [](uint32_t x) { return x >> 4; }); expMeans_ = { params->ae.exp_mean, context.hw->numAeCells }; - utils::Duration maxExposureTime = - std::clamp(frameContext.agc.maxFrameDuration, - context.configuration.sensor.minExposureTime, - context.configuration.sensor.maxExposureTime); - setLimits(context.configuration.sensor.minExposureTime, - maxExposureTime, - context.configuration.sensor.minAnalogueGain, - context.configuration.sensor.maxAnalogueGain); + /* + * Set the AGC limits using the fixed exposure time and/or gain in + * manual mode, or the sensor limits in auto mode. + */ + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + if (frameContext.agc.autoExposureEnabled) { + minExposureTime = context.configuration.sensor.minExposureTime; + maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, + context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime); + } else { + minExposureTime = context.configuration.sensor.lineDuration + * frameContext.agc.exposure; + maxExposureTime = minExposureTime; + } + + if (frameContext.agc.autoGainEnabled) { + minAnalogueGain = context.configuration.sensor.minAnalogueGain; + maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; + } else { + minAnalogueGain = frameContext.agc.gain; + maxAnalogueGain = frameContext.agc.gain; + } + + setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain); /* * The Agc algorithm needs to know the effective exposure value that was diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 80b99df8e..261c0472a 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -165,8 +165,11 @@ namespace libcamera::ipa::rkisp1 { * \var IPAActiveState::agc.automatic.gain * \brief Automatic analogue gain multiplier * - * \var IPAActiveState::agc.autoEnabled - * \brief Manual/automatic AGC state as set by the AeEnable control + * \var IPAActiveState::agc.autoExposureEnabled + * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control + * + * \var IPAActiveState::agc.autoGainEnabled + * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control * * \var IPAActiveState::agc.constraintMode * \brief Constraint mode as set by the AeConstraintMode control @@ -289,8 +292,11 @@ namespace libcamera::ipa::rkisp1 { * * The gain should be adapted to the sensor specific gain code before applying. * - * \var IPAFrameContext::agc.autoEnabled - * \brief Manual/automatic AGC state as set by the AeEnable control + * \var IPAFrameContext::agc.autoExposureEnabled + * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control + * + * \var IPAFrameContext::agc.autoGainEnabled + * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control * * \var IPAFrameContext::agc.constraintMode * \brief Constraint mode as set by the AeConstraintMode control @@ -306,6 +312,16 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAFrameContext::agc.updateMetering * \brief Indicate if new ISP AGC metering parameters need to be applied + * + * \var IPAFrameContext::agc.autoExposureModeChange + * \brief Indicate if autoExposureEnabled has changed from true in the previous + * frame to false in the current frame, and no manual exposure value has been + * supplied in the current frame. + * + * \var IPAFrameContext::agc.autoGainModeChange + * \brief Indicate if autoGainEnabled has changed from true in the previous + * frame to false in the current frame, and no manual gain value has been + * supplied in the current frame. */ /** diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index b83c18229..5d5b79fa6 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -79,7 +79,8 @@ struct IPAActiveState { double gain; } automatic; - bool autoEnabled; + bool autoExposureEnabled; + bool autoGainEnabled; controls::AeConstraintModeEnum constraintMode; controls::AeExposureModeEnum exposureMode; controls::AeMeteringModeEnum meteringMode; @@ -124,12 +125,15 @@ struct IPAFrameContext : public FrameContext { struct { uint32_t exposure; double gain; - bool autoEnabled; + bool autoExposureEnabled; + bool autoGainEnabled; controls::AeConstraintModeEnum constraintMode; controls::AeExposureModeEnum exposureMode; controls::AeMeteringModeEnum meteringMode; utils::Duration maxFrameDuration; bool updateMetering; + bool autoExposureModeChange; + bool autoGainModeChange; } agc; struct { From patchwork Thu Jan 9 00:09:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22480 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 18BCAC3315 for ; Thu, 9 Jan 2025 00:10:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3DD116851F; Thu, 9 Jan 2025 01:10:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UAJtI+aZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 058B96851D for ; Thu, 9 Jan 2025 01:10:06 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AC7096F3; Thu, 9 Jan 2025 01:09:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381352; bh=uNCIfLjnJEMjvYvpnI23JgUH5AWyRoPEpowk2v5ixJw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UAJtI+aZz6WYlkL0ntZGKpFM2Z8zHeGnm9DYPsfRNY0AOrsTADOcjVELX+ndqc+J+ 7qZX50xOyIQSSj0HVhzPJf+3y6XPuqwL4yjkFe0CXqTi8Gsg4WV8RWWTqCdnrMjL5g VKfZe4xpLvBMU9jW3kSQMhnAnkmSIXoI03i4F2TA= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Nicolas Nicolas Subject: [PATCH v6 07/12] gstreamer: Generate the new AEGC controls Date: Wed, 8 Jan 2025 18:09:37 -0600 Message-Id: <20250109000942.1616565-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Since AeEnable will be replaced with ExposureTimeMode and AnalogueGainMode so that the two can be set between auto/manual independently, update the gstreamer control ids generation to conform with this. Signed-off-by: Paul Elder Reviewed-by: Nicolas Nicolas Reviewed-by: Laurent Pinchart --- No change in v6 No change in v5 No change in v4 New in v3 --- utils/codegen/gen-gst-controls.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/utils/codegen/gen-gst-controls.py b/utils/codegen/gen-gst-controls.py index df0988266..07af7653b 100755 --- a/utils/codegen/gen-gst-controls.py +++ b/utils/codegen/gen-gst-controls.py @@ -19,8 +19,9 @@ from controls import Control exposed_controls = [ - 'AeEnable', 'AeMeteringMode', 'AeConstraintMode', 'AeExposureMode', - 'ExposureValue', 'ExposureTime', 'AnalogueGain', 'AeFlickerPeriod', + 'AeMeteringMode', 'AeConstraintMode', 'AeExposureMode', + 'ExposureValue', 'ExposureTime', 'ExposureTimeMode', + 'AnalogueGain', 'AnalogueGainMode', 'AeFlickerPeriod', 'Brightness', 'Contrast', 'AwbEnable', 'AwbMode', 'ColourGains', 'Saturation', 'Sharpness', 'ColourCorrectionMatrix', 'ScalerCrop', 'DigitalGain', 'AfMode', 'AfRange', 'AfSpeed', 'AfMetering', 'AfWindows', From patchwork Thu Jan 9 00:09:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22481 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 958E9C32F1 for ; Thu, 9 Jan 2025 00:10:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB5796854D; Thu, 9 Jan 2025 01:10:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aPTzYPTf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 821AD68506 for ; Thu, 9 Jan 2025 01:10:07 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4177ACE6; Thu, 9 Jan 2025 01:09:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381354; bh=r23aAdiOuONWkdCRuk4Z+mE6g9UTWfMIDltqsjsF14I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aPTzYPTfoTWj/yj18wCn5y/l3h6CXr7x7PS5Udx/tbQ4kZG9nwtRjYUXGuKVb2JoW 24T66ZmIjRX/FRM11kXdq/Z9BG21oWxoMr7cVC59s/RUPelOmtGrbk/HpZOZMLU2Tr wxOiTP2pAHrHKyv93T1PiIvv9ny38mLwCTlTKQWc= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, Paul Elder Subject: [PATCH v6 08/12] controls: Remove AeLocked Date: Wed, 8 Jan 2025 18:09:38 -0600 Message-Id: <20250109000942.1616565-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Now that the codebase has been ported to use the new AEGC controls remove the definition of AeLocked. AeEnable is not be removed as it will be redefined. Signed-off-by: Jacopo Mondi Signed-off-by: Paul Elder Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v6: - remove only AeLocked, as AeEnable will be redefined instead of removed - (used to be "controls: Remove AeEnable and AeLocked") No change in v5 No change in v4 No change in v3 --- src/libcamera/control_ids_core.yaml | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index 553095481..d88c1852e 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -16,18 +16,6 @@ controls: \sa ExposureTime AnalogueGain - - AeLocked: - type: bool - direction: out - description: | - Report the lock status of a running AE algorithm. - - If the AE algorithm is locked the value shall be set to true, if it's - converging it shall be set to false. If the AE algorithm is not - running the control shall not be present in the metadata control list. - - \sa AeEnable - - AeState: type: int32_t direction: out From patchwork Thu Jan 9 00:09:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22482 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4D917C3316 for ; Thu, 9 Jan 2025 00:10:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BE44368557; Thu, 9 Jan 2025 01:10:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uQaxDLV9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A8F5E68521 for ; Thu, 9 Jan 2025 01:10:08 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BBA52FA8; Thu, 9 Jan 2025 01:09:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381355; bh=mhWktRhCo484sgGQP0pxeSPRa0OTZP5UxpmCK3ddBaU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uQaxDLV9rI1wbkDFR9xyP0ce2AM0TcyDTvGzczNZ9jmJhKPzUpCH7Yvh8o0EynqF8 WDLIj7xwZUZ2RVU6ISIsQyeuOIyevaBaJ6S3izp4S2zu7eBGzkRGJHFxvGn0PiVna7 IIJVWcHhPaPbf8coeb62czKQvjOVNLI01xjT5K/s= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com Subject: [PATCH v6 09/12] controls: Redefine AeEnable Date: Wed, 8 Jan 2025 18:09:39 -0600 Message-Id: <20250109000942.1616565-10-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the redesign of the AE-related controls, the AeEnable control was intended to be removed in favor of more specific sub-controls for analogue gain and exposure time. However this will cause problems if an aperture sub-controls is introduced, and an application from a pre-aperture era uses a camera that supports aperture. If there is no AeEnable control, then a pre-aperture era application might set analogue gain and exposure time to manual, while aperture silently stays auto since that's the default mode. Thus aperture would be uncontrollable by the application. With an AeEnable control, then a pre-aperture era application can set AeEnable to manual, and under the hood all three of analogue gain and exposure time and aperture will be set to manual. The application won't be able to set the manual aperture, however. Although the above scenario is expected to be rare, as applications are expected to recompile on a libcamera version change, the scenario with an AeEnable control seems less detrimental. With an AeEnable control at least the aperture would be static at a reasonably usable value, whereas without an AeEnable the aperture would be more-or-less uncontrolable and could go to extreme values as the AEGC algorithm tries to compensate for the manual analogue gain and exposure time values. Thus we redefine the AeEnable control, available only as a control and not in metadata. It will be preprocessed by the Camera class so that the relevant sub-controls are set. No pipline handler nor IPA shall act on the AeEnable control. The IPA still has to report the control as available, however. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- New in v6 --- Documentation/design/ae.rst | 23 +++++++++++++++++------ src/libcamera/control_ids_core.yaml | 12 +++++++++--- 2 files changed, 26 insertions(+), 9 deletions(-) diff --git a/Documentation/design/ae.rst b/Documentation/design/ae.rst index f8b2c887c..bbb6bb1d1 100644 --- a/Documentation/design/ae.rst +++ b/Documentation/design/ae.rst @@ -185,6 +185,10 @@ A diagram of our solution: 0: Auto 1: Manual + AeEnable + - True -> ExposureTimeMode:Auto + AnalogueGainMode:Auto + - False -> ExposureTimeMode:Manual + AnalogueGainMode:Manual + The diagram is divided in four sections horizontally: @@ -225,10 +229,14 @@ This simulates an auto -> locked -> manual or auto -> manual state transition, and makes it impossible to do the nonsensical manual -> locked state transition. -We specifically do not have a "master AE control" like the old AeEnable. This -is because we have the individual mode controls, and if we had a master AE -control it would be a "control that sets other controls", which could easily -get out of control. +AeEnable still exists to allow applications to set the mode of all the +sub-controls at once. Besides being for convenience, this will also be useful +when we eventually implement an aperture control. This is because applications +there were made before aperture was available would still be able to set +aperture mode to auto or manual, as opposed to having the aperture stuck at +auto while the application really wanted manual. Although the aperture would +still be stuck at an uncontrollable value, at least it would be at a static +usable value as opposed to varying via the AEGC algorithm. With this solution, the earlier example would become: @@ -277,9 +285,12 @@ and gain: - AeState +- AeEnable + Auto-exposure and auto-gain can be enabled and disabled separately using the -ExposureTimeMode and AnalogueGainMode controls respectively. There is no -overarching AeEnable control. +ExposureTimeMode and AnalogueGainMode controls respectively. The AeEnable +control can also be used, as it sets both of the modes simultaneously. The +AeEnable control is not returned in metadata. When the respective mode is set to auto, the respective value that is computed by the AEGC algorithm is applied to the image sensor. Any value that is diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index d88c1852e..aa7448645 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -10,11 +10,17 @@ vendor: libcamera controls: - AeEnable: type: bool - direction: inout + direction: in description: | - Enable or disable the AE. + Enable or disable the AEGC algorithm. When this control is set to true, + both ExposureTimeMode and AnalogueGainMode are set to auto, and if this + control is set to false then both are set to manual. + + If ExposureTimeMode or AnalogueGainMode are also set in the same + request as AeEnable, then the modes supplied by ExposureTimeMode or + AnalogueGainMode will take precedence. - \sa ExposureTime AnalogueGain + \sa ExposureTimeMode AnalogueGainMode - AeState: type: int32_t From patchwork Thu Jan 9 00:09:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22483 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2E2BC3317 for ; Thu, 9 Jan 2025 00:10:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BFA8468553; Thu, 9 Jan 2025 01:10:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aMRxt3nX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 15D116854C for ; Thu, 9 Jan 2025 01:10:10 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E43B4B63; Thu, 9 Jan 2025 01:09:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381356; bh=jIa2rAbmyuIgxc74fBa49JmlbK4WRFcdmwsSokhGg6s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aMRxt3nXnn/KjJcQQncbTPTAuKeSGujZyzsg7r5HbaR50Iq4s+qvAF5O7UBUJRFvo gDycpfhGbFlb3SN4lGd3zKCV8eFBMYFwTRoWOP3E4A8LtwQmQRwZAKkNK+SqaOkRdx UbHHymI/HWQP2pbPz41ItDnchRx90sXP6QrYn63w= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com Subject: [PATCH v6 10/12] libcamera: camera: Pre-process AeEnable control Date: Wed, 8 Jan 2025 18:09:40 -0600 Message-Id: <20250109000942.1616565-11-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the AeEnable under-the-hood in the Camera class, such that AeEnable activates ExposureTimeMode and AnalogueGain together. This allows applications the convenience of setting auto/manual mode of all of the AE-related controls, as well as protecting applications against a nasty behavior change if an aperture control is added in the future. This also moves common handling code out of the IPA. While we also want to inject AeEnable in Camera::controls() so that IPAs don't have to report it, it is technically difficult at the moment as ControlInfoMaps are not easily modifiable. Signed-off-by: Paul Elder --- New in v6 --- src/libcamera/camera.cpp | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 69a7ee535..4e0f63930 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -1325,6 +1326,23 @@ int Camera::queueRequest(Request *request) } } + /* Pre-process AeEnable */ + ControlList &controls = request->controls(); + const auto &aeEnable = controls.get(controls::AeEnable); + if (aeEnable) { + if (!controls.contains(controls::AnalogueGainMode.id())) { + controls.set(controls::AnalogueGainMode, + *aeEnable ? controls::AnalogueGainModeAuto + : controls::AnalogueGainModeManual); + } + + if (!controls.contains(controls::ExposureTimeMode.id())) { + controls.set(controls::ExposureTimeMode, + *aeEnable ? controls::ExposureTimeModeAuto + : controls::ExposureTimeModeManual); + } + } + d->pipe_->invokeMethod(&PipelineHandler::queueRequest, ConnectionTypeQueued, request); From patchwork Thu Jan 9 00:09:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22484 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F2227C3318 for ; Thu, 9 Jan 2025 00:10:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6A17768521; Thu, 9 Jan 2025 01:10:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A1wIBK41"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4858868543 for ; Thu, 9 Jan 2025 01:10:11 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3D13ACE6; Thu, 9 Jan 2025 01:09:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381358; bh=FD/BGBvqUdXEdPZbAwXMJHAXe3+v3JBH6vxzf9VuhOQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A1wIBK413EbvDKQXYwJRHIvwmWsedhAEQNRfQUNor5wIJx9Y8ywoWqkSbt97TZ+iF OOmKBVFugk2lsZX62qcHmUu+DZ5c87Ysl/6ePbwceSMcHSvqJ5H/VT75cF3YUoZSwq gyzt1Iyq0HKFmvE4aV5IhLWBE0RMHoFdvCrA5fVQ= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com Subject: [PATCH v6 11/12] ipa: rkisp1: agc: Report new AeEnable control as available Date: Wed, 8 Jan 2025 18:09:41 -0600 Message-Id: <20250109000942.1616565-12-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Even though the new AeEnable control internally switches on and off the sub-controls (ExposureTimeMode and AnalogueGainMode), it still needs to be declared as available. Report this control as available in the rkisp1 IPA. Support for the control does not need to be added as it is handled by the Camera class. It does not need to be handled in metadata either as the new version of AeEnable is not returned in metadata. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- New in v6 --- src/ipa/rkisp1/algorithms/agc.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 49c107c99..3c0cb01c7 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -156,6 +156,8 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) ControlInfo(static_cast(controls::AnalogueGainModeAuto), static_cast(controls::AnalogueGainModeManual), static_cast(controls::AnalogueGainModeAuto)); + /* \todo Move this to the Camera class */ + context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true); context.ctrlMap.merge(controls()); return 0; From patchwork Thu Jan 9 00:09:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 22485 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 30D43C3319 for ; Thu, 9 Jan 2025 00:10:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 746706854F; Thu, 9 Jan 2025 01:10:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ve12ljxS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B395C68500 for ; Thu, 9 Jan 2025 01:10:12 +0100 (CET) Received: from pyrite.hamster-moth.ts.net (unknown [IPv6:2604:2d80:9e93:ad00:3d82:7e4f:ab9b:8a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8A0AC11F1; Thu, 9 Jan 2025 01:09:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1736381359; bh=QfykDUOINS/cQGLJPiSZ7a9QcoQb3YgBabHxTfBk0xo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ve12ljxSUwGlk6kKCiSQTuLT8EgQVKGUcTuM9PGCtFhqGeL1qMaT0l57hwT4yS+wB i/zMjWDuQMJVJA54ockJb1q0bVoUGcljZEn7ouB39J8OOf+/UEeMaBtgFzi/+pLuDE AZVR9v7IPJcfsLiPGgc+5MOZDkLFNo62BXZqhozo= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com Subject: [PATCH v6 12/12] ipa: raspberry: Report new AeEnable control as available Date: Wed, 8 Jan 2025 18:09:42 -0600 Message-Id: <20250109000942.1616565-13-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109000942.1616565-1-paul.elder@ideasonboard.com> References: <20250109000942.1616565-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Even though the new AeEnable control internally switches on and off the sub-controls (ExposureTimeMode and AnalogueGainMode), it still needs to be declared as available. Report this control as available in the rpi IPA. Support for the control does not need to be added as it is handled by the Camera class. It does not need to be handled in metadata either as the new version of AeEnable is not returned in metadata. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- New in v6 --- src/ipa/rpi/common/ipa_base.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 4334d22e1..44ce38caa 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -55,6 +55,8 @@ constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; /* List of controls handled by the Raspberry Pi IPA */ const ControlInfoMap::Map ipaControls{ + /* \todo Move this to the Camera class */ + { &controls::AeEnable, ControlInfo(false, true) }, { &controls::ExposureTimeMode, ControlInfo(static_cast(controls::ExposureTimeModeAuto), static_cast(controls::ExposureTimeModeManual)) },