From patchwork Fri Oct 1 10:33:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14016 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1992DBDC71 for ; Fri, 1 Oct 2021 10:33:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D5534691AC; Fri, 1 Oct 2021 12:33:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dlx1rHrH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A40E691AA for ; Fri, 1 Oct 2021 12:33:39 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 97781A16; Fri, 1 Oct 2021 12:33:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084418; bh=KJM5JdQPyPLEELRvHAwJqWh/zTOh9FW5iDIbNvPAehk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dlx1rHrHUcieRlzkujQMvdL9T3W2JzdREzwedqYak5Aig6SGzrelLJrrti6S9MhaG DI5xcZpF8NPYikUOBV9FxuSKMZgt1s5IiX5CIOAGLWzhfSsaXMwFlQTbb8zPZZ4C2I hJsbrPT4hBpNddrOG7D1PGB5U+XJtR/0zG9a2BaI= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:19 +0900 Message-Id: <20211001103325.1077590-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/7] controls: Reorganize the AE-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We have multiple goals: - we need a lock of some sort, to instruct the AEGC to not update output results - we need manual modes, to override the values computed by the AEGC - we need to support seamless transitions from auto -> manual, and do so without flickering - we need custom minimum values for the manual controls, that is no magic values for enabling/disabling auto - all of these need to be done with AE sub-controls (exposure time, analogue gain) To achieve these goals, we introduce mode controls for the AE sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto state, and a disabled state. The disabled state has an internal one-way state change from locked to manual, triggered by the presence of the value-controls (ExposureTime and AnalogueGain). We then remove the AeEnable control, as it is a redundant control in the face of these two mode controls. We also remove AeLocked, as it is insufficient for reporting the AE state, and we promote AeState to non-draft to fill its role. Notably, the locked state is removed, since this information can be obtained from the aforementioned mode controls. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- Changes in v2: - No changes, just resubmitting at the head of this series so that it's together and so that /people will actually see it/ Initial version: Still RFC as I haven't updated the users of the control yet, and I want to check that these are the controls and docs that we want. We've decided that the "master AE control" will be implemented by a helper... but looking at uvcvideo and the V4L2 controls I'm wondering if such helper should come earlier than later? --- src/libcamera/control_ids.yaml | 215 +++++++++++++++++++++++---------- 1 file changed, 150 insertions(+), 65 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 9d4638ae..18b186e8 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -7,23 +7,61 @@ # Unless otherwise stated, all controls are bi-directional, i.e. they can be # set through Request::controls() and returned out through Request::metadata(). controls: - - AeEnable: - type: bool - description: | - Enable or disable the AE. - - \sa ExposureTime AnalogueGain - - - AeLocked: - type: bool + - AeState: + type: int32_t description: | - Report the lock status of a running AE algorithm. - - If the AE algorithm is locked the value shall be set to true, if it's - converging it shall be set to false. If the AE algorithm is not - running the control shall not be present in the metadata control list. + Control to report the current AE algorithm state. Enabling or disabling + any of the AE-related mode controls (eg. AnalogueGainMode, + ExposureTimeMode) is not required to reset the state to + AeStateInactive. The camera device can do several state transitions + between two results, if it is allowed by the state transition table. + For example, AeStateInactive may never actually be seen in a result. + + The state in the result is the state for this image (in sync with this + image). If AE state becomes AeStateConverged, then the image data + associated with the result should be good to use. + + As some AE algorithms may still be running when any of the AE-related + controls are in manual mode, the states are valid even when the mode + controls are set to Disabled. To know if this image image used AE or + manual (or a mixture), check the relevant modes (eg. AnalogueGainMode, + ExposureTimeMode). + + \sa AnalogueGain + \sa AnalogueGainMode + \sa ExposureTime + \sa ExposureTimeMode - \sa AeEnable + enum: + - name: AeStateInactive + value: 0 + description: | + The AE algorithm is inactive. + If the camera initiates an AE scan, the state shall go to Searching. + - name: AeStateSearching + value: 1 + description: | + The AE algorithm has not converged yet. + If the camera finishes an AE scan, the state shall go to Converged. + If the camera finishes an AE scan, but flash is required, the state + shall go to FlashRequired. + - name: AeStateConverged + value: 2 + description: | + The AE algorithm has converged. + If the camera initiates an AE scan, the state shall go to Searching. + - name: AeStateFlashRequired + value: 3 + description: | + The AE algorithm would need a flash for good results + If the camera initiates an AE scan, the state shall go to Searching. + If AeLock is on, the state shall go to Locked. + - name: AeStatePrecapture + value: 4 + description: | + The AE algorithm has started a pre-capture metering session. + After the sequence is finished, the state shall go to Converged if + \sa AePrecaptureTrigger # AeMeteringMode needs further attention: # - Auto-generate max enum value. @@ -93,6 +131,17 @@ controls: how the desired total exposure is divided between the shutter time and the sensor's analogue gain. The exposure modes are platform specific, and not all exposure modes may be supported. + + When one of AnalogueGainMode or ExposureTimeMode is set to Disabled and + a corresponding manual control is provided, AeExposureMode may be + ignored. This is because, for example, if AeExposureMode is set to + ExposureLong but ExposureTimeMode is Disabled and a short ExposureTime + is provided (AnalogueGainMode set to Auto), then the AeExposureMode + doesn't make sense. + + \sa AnalogueGainMode + \sa ExposureTimeMode + enum: - name: ExposureNormal value: 0 @@ -111,13 +160,15 @@ controls: type: float description: | Specify an Exposure Value (EV) parameter. The EV parameter will only be - applied if the AE algorithm is currently enabled. + applied if the AE algorithm is currently enabled, that is, at least one + of AnalogueGainMode and ExposureTimeMode are auto. By convention EV adjusts the exposure as log2. For example EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x]. - \sa AeEnable + \sa AnalogueGainMode + \sa ExposureTimeMode - ExposureTime: type: int32_t @@ -125,17 +176,49 @@ controls: Exposure time (shutter speed) for the frame applied in the sensor device. This value is specified in micro-seconds. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. Its + presence in a request acts as a trigger to switch to the internal + manual mode within ExposureTimeModeDisabled. + + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. - \sa AnalogueGain AeEnable + \sa AnalogueGain + \sa ExposureTimeMode - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + - ExposureTimeMode: + type: int32_t + description: | + Control to set and report the source of the exposure time that will be + applied. + + \sa ExposureTime + enum: + - name: ExposureTimeModeAuto + value: 0 + description: | + The exposure time will be calculated automatically and set by the + AE algorithm. + - name: ExposureTimeModeDisabled + value: 1 + description: | + The exposure time will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in ExposureTime. + + This Disabled state has two internal states; locked and manual. + When the Disabled state is first entered, the internal state will + be locked, and the latest exposure time value set by the AE + algorithm will be used (or the default ExposureTime value, if the + mode started out as Disabled). When an ExposureTime is provided, + then the internal state will go to manual, and the provided value + will be used for the exposure time. Going from manual to locked is + not possible. If an ExposureTime value is provided in the same + request as going into the Disabled state, then the internal locked + state will be skipped, and the internal state will start out as + manual. - AnalogueGain: type: float @@ -144,17 +227,49 @@ controls: The value of the control specifies the gain multiplier applied to all colour channels. This value cannot be lower than 1.0. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. Its + presence in a request acts as a trigger to switch to the internal + manual mode within ExposureTimeModeDisabled. - \sa ExposureTime AeEnable + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. + + \sa ExposureTime + \sa AnalogueGainMode + + - AnalogueGainMode: + type: int32_t + description: | + Control to set and report the source of the analogue gain that will be + applied. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + \sa AnalogueGain + enum: + - name: AnalogueGainModeAuto + value: 0 + description: | + The analogue gain will be calculated automatically and set by the + AE algorithm. + - name: AnalogueGainModeDisabled + value: 1 + description: | + The analogue gain will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in AnalogueGain. + + This Disabled state has two internal states; locked and manual. + When the Disabled state is first entered, the internal state will + be locked, and the latest analogue gain value set by the AE + algorithm will be used (or the default AnalogueGain value, if the + mode started out as Disabled). When an AnalogueGain is provided, + then the internal state will go to manual, and the provided value + will be used for the analogue gain. Going from manual to locked is + not possible. If an AnalogueGain value is provided in the same + request as going into the Disabled state, then the internal locked + state will be skipped, and the internal state will start out as + manual. - Brightness: type: float @@ -477,36 +592,6 @@ controls: High quality aberration correction which might reduce the frame rate. - - AeState: - type: int32_t - draft: true - description: | - Control to report the current AE algorithm state. Currently identical to - ANDROID_CONTROL_AE_STATE. - - Current state of the AE algorithm. - enum: - - name: AeStateInactive - value: 0 - description: The AE algorithm is inactive. - - name: AeStateSearching - value: 1 - description: The AE algorithm has not converged yet. - - name: AeStateConverged - value: 2 - description: The AE algorithm has converged. - - name: AeStateLocked - value: 3 - description: The AE algorithm is locked. - - name: AeStateFlashRequired - value: 4 - description: The AE algorithm would need a flash for good results - - name: AeStatePrecapture - value: 5 - description: | - The AE algorithm has started a pre-capture metering session. - \sa AePrecaptureTrigger - - AfState: type: int32_t draft: true From patchwork Fri Oct 1 10:33:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14017 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 94703BDC71 for ; Fri, 1 Oct 2021 10:33:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 52610691B3; Fri, 1 Oct 2021 12:33:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PH2FAj6Q"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E2A51691B0 for ; Fri, 1 Oct 2021 12:33:40 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 71652596; Fri, 1 Oct 2021 12:33:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084420; bh=97M0rv3Sn0paTSe4Y0O7q9AzQaH/dcm29+dgj7mOcKc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PH2FAj6QwFVWVb0g+AQI2U7avDES5Ed9i0chEZxM90Wqa+rjIAd9GwmWP4Zanl7Gg bAJVrDpSWhrZ5ODDyDq99f8W8V94vrmfQ86pbb5qVED1oooZckZscL2Q3sTFDwR9jf 4pM7BIOmWT5Hg406wxxKxCn4q3cUVRTx5aXuk6jI= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:20 +0900 Message-Id: <20211001103325.1077590-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/7] libcamera: pipeline: uvcvideo: Support the new AE controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the new AE controls in the uvcvideo pipeline handler. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- Changes in v2: - fix the rebase error where some uvc stuff was in rasberrypi --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 36 +++++++++++++++++--- 1 file changed, 32 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 264f5370..3a9c3b8d 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -268,7 +268,9 @@ int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id, cid = V4L2_CID_CONTRAST; else if (id == controls::Saturation) cid = V4L2_CID_SATURATION; - else if (id == controls::AeEnable) + else if (id == controls::ExposureTimeMode) + cid = V4L2_CID_EXPOSURE_AUTO; + else if (id == controls::AnalogueGainMode) cid = V4L2_CID_EXPOSURE_AUTO; else if (id == controls::ExposureTime) cid = V4L2_CID_EXPOSURE_ABSOLUTE; @@ -302,9 +304,33 @@ int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id, } case V4L2_CID_EXPOSURE_AUTO: { - int32_t ivalue = value.get() + bool exposureSet = controls->contains(V4L2_CID_EXPOSURE_AUTO); + + /* \todo Make this nicer. */ + int32_t ivalue; + if (id == controls::ExposureTimeMode && exposureSet) { + int32_t exposureMode = controls->get(V4L2_CID_EXPOSURE_AUTO).get(); + ivalue = value.get() == controls::ExposureTimeModeAuto + ? (exposureMode == V4L2_EXPOSURE_SHUTTER_PRIORITY + ? V4L2_EXPOSURE_AUTO + : V4L2_EXPOSURE_APERTURE_PRIORITY) + : V4L2_EXPOSURE_MANUAL; + } else if (id == controls::ExposureTimeMode && !exposureSet) { + ivalue = value.get() == controls::ExposureTimeModeAuto ? V4L2_EXPOSURE_APERTURE_PRIORITY : V4L2_EXPOSURE_MANUAL; + } else if (id == controls::AnalogueGainMode && exposureSet) { + int32_t exposureMode = controls->get(V4L2_CID_EXPOSURE_AUTO).get(); + ivalue = value.get() == controls::AnalogueGainModeAuto + ? (exposureMode == V4L2_EXPOSURE_APERTURE_PRIORITY + ? V4L2_EXPOSURE_AUTO + : V4L2_EXPOSURE_SHUTTER_PRIORITY) + : V4L2_EXPOSURE_MANUAL; + } else if (id == controls::AnalogueGainMode && !exposureSet) { + ivalue = value.get() == controls::AnalogueGainModeAuto + ? V4L2_EXPOSURE_SHUTTER_PRIORITY + : V4L2_EXPOSURE_MANUAL; + } controls->set(V4L2_CID_EXPOSURE_AUTO, ivalue); break; } @@ -559,7 +585,7 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, id = &controls::Saturation; break; case V4L2_CID_EXPOSURE_AUTO: - id = &controls::AeEnable; + id = &controls::ExposureTimeMode; break; case V4L2_CID_EXPOSURE_ABSOLUTE: id = &controls::ExposureTime; @@ -610,7 +636,9 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, break; case V4L2_CID_EXPOSURE_AUTO: - info = ControlInfo{ false, true, true }; + info = ControlInfo{ { controls::ExposureTimeModeAuto, + controls::ExposureTimeModeDisabled }, + controls::ExposureTimeModeDisabled }; break; case V4L2_CID_EXPOSURE_ABSOLUTE: From patchwork Fri Oct 1 10:33:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14018 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9A14BDC71 for ; Fri, 1 Oct 2021 10:33:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A513B691B8; Fri, 1 Oct 2021 12:33:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rcT5H5DO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B337C691AA for ; Fri, 1 Oct 2021 12:33:42 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4C37F596; Fri, 1 Oct 2021 12:33:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084422; bh=FZZ3rKzhl/HjczljP766ilE3qTZbgXGbn4dkW1CfJuw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rcT5H5DOhhTePLH16K5BQtlpfEiiDU1KcVPrUAyXZKlMlcWoCxinHIudxyWW6V1gd L9LMaWR7ReM7TMxTYtVQP6K0khPBWqAX33+Zj8MoMb2srsbKRSRNYbrBgDkMQsJJ27 hwPHTli46+S4EQml69FUozsHpHSnV1WY74SRaUmU= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:21 +0900 Message-Id: <20211001103325.1077590-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/7] libcamera: pipeline: raspberrypi: Support the new AE controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the new AE controls in the raspberrypi pipeline handler, and in the IPA. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- Changes in v2: - fix the rebase error where some uvc stuff was in rasberrypi - i haven't yet taken in the comments to move the new Pause/Resume functions Initial versoin: This is very hacky. I wasn't sure what the best way was to plumb it into the raspberrypi IPA as it was a bit hairy... --- include/libcamera/ipa/raspberrypi.h | 3 +- src/ipa/raspberrypi/controller/rpi/agc.cpp | 18 +++++++++- src/ipa/raspberrypi/controller/rpi/agc.hpp | 5 +++ src/ipa/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++----- 4 files changed, 58 insertions(+), 10 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 521eaecd..363ea038 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -28,8 +28,9 @@ namespace RPi { * unsupported control is encountered. */ static const ControlInfoMap Controls({ - { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::ExposureTimeMode, ControlInfo(controls::ExposureTimeModeValues) }, { &controls::ExposureTime, ControlInfo(0, 999999) }, + { &controls::AnalogueGainMode, ControlInfo(controls::AnalogueGainModeValues) }, { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 289c1fce..b45ea454 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -203,14 +203,30 @@ bool Agc::IsPaused() const } void Agc::Pause() +{ +} + +void Agc::Resume() +{ +} + +void Agc::PauseExposure() { fixed_shutter_ = status_.shutter_time; +} + +void Agc::PauseGain() +{ fixed_analogue_gain_ = status_.analogue_gain; } -void Agc::Resume() +void Agc::ResumeExposure() { fixed_shutter_ = 0s; +} + +void Agc::ResumeGain() +{ fixed_analogue_gain_ = 0; } diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index 82063636..7ca3ca2f 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -92,6 +92,11 @@ public: void Prepare(Metadata *image_metadata) override; void Process(StatisticsPtr &stats, Metadata *image_metadata) override; + void PauseExposure(); + void PauseGain(); + void ResumeExposure(); + void ResumeGain(); + private: void updateLockStatus(DeviceStatus const &device_status); AgcConfig config_; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 047123ab..99935515 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -53,6 +53,8 @@ #include "sharpen_algorithm.hpp" #include "sharpen_status.h" +#include "controller/rpi/agc.hpp" + namespace libcamera { using namespace std::literals::chrono_literals; @@ -478,7 +480,10 @@ void IPARPi::reportMetadata() AgcStatus *agcStatus = rpiMetadata_.GetLocked("agc.status"); if (agcStatus) { - libcameraMetadata_.set(controls::AeLocked, agcStatus->locked); + libcameraMetadata_.set(controls::AeState, + agcStatus->locked ? + controls::AeStateConverged : + controls::AeStateSearching); libcameraMetadata_.set(controls::DigitalGain, agcStatus->digital_gain); } @@ -623,20 +628,22 @@ void IPARPi::queueRequest(const ControlList &controls) << " = " << ctrl.second.toString(); switch (ctrl.first) { - case controls::AE_ENABLE: { - RPiController::Algorithm *agc = controller_.GetAlgorithm("agc"); + case controls::EXPOSURE_TIME_MODE: { + RPiController::Algorithm *algo = controller_.GetAlgorithm("agc"); + RPiController::Agc *agc = reinterpret_cast(algo); if (!agc) { LOG(IPARPI, Warning) - << "Could not set AE_ENABLE - no AGC algorithm"; + << "Could not set EXPOSURE_TIME_MODE - no AGC algorithm"; break; } - if (ctrl.second.get() == false) - agc->Pause(); + if (ctrl.second.get() == controls::ExposureTimeModeDisabled) + agc->PauseExposure(); else - agc->Resume(); + agc->ResumeExposure(); - libcameraMetadata_.set(controls::AeEnable, ctrl.second.get()); + libcameraMetadata_.set(controls::ExposureTimeMode, + ctrl.second.get()); break; } @@ -656,6 +663,25 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::ANALOGUE_GAIN_MODE: { + RPiController::Algorithm *algo = controller_.GetAlgorithm("agc"); + RPiController::Agc *agc = reinterpret_cast(algo); + if (!agc) { + LOG(IPARPI, Warning) + << "Could not set ANALOGUE_GAIN_MODE - no AGC algorithm"; + break; + } + + if (ctrl.second.get() == controls::AnalogueGainModeDisabled) + agc->PauseGain(); + else + agc->ResumeGain(); + + libcameraMetadata_.set(controls::AnalogueGainMode, + ctrl.second.get()); + break; + } + case controls::ANALOGUE_GAIN: { RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.GetAlgorithm("agc")); From patchwork Fri Oct 1 10:33:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14019 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9926FBDC71 for ; Fri, 1 Oct 2021 10:33:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 635F1691B7; Fri, 1 Oct 2021 12:33:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cVA4RcLh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 89E64691AF for ; Fri, 1 Oct 2021 12:33:44 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 24DA8596; Fri, 1 Oct 2021 12:33:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084424; bh=iHgkFnCuKHqtAEaIVpDGSd+yrZgYpQ8xUTdWLWCpEVs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cVA4RcLhqoh1FEsEK0KznkMfyg3p0VjOdsnr+nXdmGr6N8NcSml3+jkRNoQ9Me0RE A2BMFjwZex8oaqnetk11d/cTVtzRjqKWTrWtpysOjhd3AodQk2QL4K8swMIrAUliFF w6j4hrOk4nF+RAPd7oVBYBk34ZgRM5vYYsdTfXwQ= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:22 +0900 Message-Id: <20211001103325.1077590-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/7] test: ipa_data_serializer: Remove controls::AeEnable X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove controls::AeEnable from the IPA data serializer test, as it is no longer an available control. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- No change in v2 --- test/serialization/ipa_data_serializer_test.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp index 5fcdcb8e..85ad4d04 100644 --- a/test/serialization/ipa_data_serializer_test.cpp +++ b/test/serialization/ipa_data_serializer_test.cpp @@ -33,7 +33,6 @@ using namespace std; using namespace libcamera; static const ControlInfoMap Controls = ControlInfoMap({ - { &controls::AeEnable, ControlInfo(false, true) }, { &controls::ExposureTime, ControlInfo(0, 999999) }, { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, From patchwork Fri Oct 1 10:33:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14020 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 07E68BDC71 for ; Fri, 1 Oct 2021 10:33:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BBDF7691B0; Fri, 1 Oct 2021 12:33:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lPUkJwOV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6823A691BB for ; Fri, 1 Oct 2021 12:33:46 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F3059596; Fri, 1 Oct 2021 12:33:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084426; bh=DGi34lE8u9nRpPGBnVrApqYb6dEBAYeELfDzkOu+lDU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lPUkJwOVO/dmJLSZKqHqH9/45vaedwcp5Y8ihl3G72kuN5YqvF/IaKHlwg0stTIb0 ScJlNHca7GlSwo6VST8/xYuwumtIHYNAw0Jvwjp+uveqFdEydY1h9QuFA4pklQlR6h hlOYFUAXR5GZEa/ixnv0b5NoHCTBvqctwcy23Dp4= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:23 +0900 Message-Id: <20211001103325.1077590-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/7] libcamera: pipeline: rkisp1: Support the new AE controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the new AE controls in the rkisp1 pipeline handler, and in the IPA. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- No change in v2 Initial version: It seems that the rkisp1 pipeline handler doesn't properly expose its control capabilities? And even though it exposed AeEnable with both true and false it didn't expose or handle AnalogueGain nor ExposureTime either? So I just simply replaced AeEnable with ExposureTimeMode for now. --- src/ipa/rkisp1/rkisp1.cpp | 18 ++++++++++-------- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++-- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index bf2c13b6..708adfc9 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -52,7 +52,7 @@ private: const rkisp1_stat_buffer *stats); void setControls(unsigned int frame); - void metadataReady(unsigned int frame, unsigned int aeState); + void metadataReady(unsigned int frame, int aeState); std::map buffers_; std::map mappedBuffers_; @@ -221,8 +221,9 @@ void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, memset(params, 0, sizeof(*params)); /* Auto Exposure on/off. */ - if (controls.contains(controls::AeEnable)) { - autoExposure_ = controls.get(controls::AeEnable); + if (controls.contains(controls::ExposureTimeMode)) { + autoExposure_ = (controls.get(controls::ExposureTimeMode) == + controls::ExposureTimeModeAuto); if (autoExposure_) params->module_ens = RKISP1_CIF_ISP_MODULE_AEC; @@ -239,7 +240,7 @@ void IPARkISP1::updateStatistics(unsigned int frame, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; - unsigned int aeState = 0; + int aeState = controls::AeStateInactive; if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) { const rkisp1_cif_isp_ae_stat *ae = ¶ms->ae; @@ -274,7 +275,9 @@ void IPARkISP1::updateStatistics(unsigned int frame, setControls(frame + 1); } - aeState = fabs(factor - 1.0f) < 0.05f ? 2 : 1; + aeState = fabs(factor - 1.0f) < 0.05f ? + controls::AeStateConverged : + controls::AeStateSearching; } metadataReady(frame, aeState); @@ -293,12 +296,11 @@ void IPARkISP1::setControls(unsigned int frame) queueFrameAction.emit(frame, op); } -void IPARkISP1::metadataReady(unsigned int frame, unsigned int aeState) +void IPARkISP1::metadataReady(unsigned int frame, int aeState) { ControlList ctrls(controls::controls); - if (aeState) - ctrls.set(controls::AeLocked, aeState == 2); + ctrls.set(controls::AeState, aeState); RkISP1Action op; op.op = ActionMetadata; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 98008862..1ed4522f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -935,8 +935,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) ControlInfoMap::Map ctrls; ctrls.emplace(std::piecewise_construct, - std::forward_as_tuple(&controls::AeEnable), - std::forward_as_tuple(false, true)); + std::forward_as_tuple(&controls::ExposureTimeMode), + std::forward_as_tuple(ControlInfo(controls::AeExposureModeValues))); data->controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); From patchwork Fri Oct 1 10:33:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14021 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 79B7EBDC71 for ; Fri, 1 Oct 2021 10:33:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 41197691B6; Fri, 1 Oct 2021 12:33:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G99FhBgo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E153691AA for ; Fri, 1 Oct 2021 12:33:48 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CC763A16; Fri, 1 Oct 2021 12:33:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084428; bh=2JdAS4NO+dDKWBHjAjtpumf0GTZhtUlaaonJgmNDkmI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G99FhBgoJWg85Q4gALp9rJjCzoYGXdDBLI50eFtGZInl7lEUz9KtG66CYVXrk4ESk j1OyDTsUHalCK2/WzMDQP7U+NJvQ0wwja7+P5aaQ0tcMTVZMc+0HrIQ1T1kgOpK9Z4 t+x+Yd3Xb2Y4AYbXgzLCgAvWN2shg/je8kckW37w= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:24 +0900 Message-Id: <20211001103325.1077590-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/7] android: Plumb all AE-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the AE-related controls reorganized in libcamera, with separate value and lever controls for exposure and gain, plumb these through the HAL layer (in order of appearence in the patch): - static metadata: available AE modes, AE lock available - manual template: add AE off (the other templates already have AE on) - request metadata: HAL -> libcamera controls conversion - result metadata: libcamera -> HAL controls conversion - result metadata: AE state We add class variables to CameraDevice to save the last set android AE controls, as that is how android controls function (no need to resubmit controls if they are unchanged). We also save libcamera's AE state in the request descriptor, as otherwise there is insufficient information from libcamera's result metadata alone to tell if the state is locked or manual (as they are an internal state in libcamera). Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- New in v2 --- src/android/camera_capabilities.cpp | 63 ++++++++++++++++--- src/android/camera_device.cpp | 98 +++++++++++++++++++++++++++-- src/android/camera_device.h | 16 +++++ 3 files changed, 165 insertions(+), 12 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index f7a6cda9..3fed3f83 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -830,12 +830,62 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_ANTIBANDING_MODES, aeAvailableAntiBandingModes); - std::vector aeAvailableModes = { - ANDROID_CONTROL_AE_MODE_ON, - }; + std::vector aeAvailableModes; + /* This will cause problems if one supports only on and the other only off */ + bool aeAddedOn = false; + bool aeAddedOff = false; + + const auto &analogGainModeInfo = controlsInfo.find(&controls::AnalogueGainMode); + if (analogGainModeInfo != controlsInfo.end()) { + for (const auto &value : analogGainModeInfo->second.values()) { + switch (value.get()) { + case controls::AnalogueGainModeAuto: + if (!aeAddedOn) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_ON); + aeAddedOn = true; + break; + case controls::AnalogueGainModeDisabled: + if (!aeAddedOff) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_OFF); + aeAddedOff = true; + break; + default: + break; + } + } + } + + const auto &exposureTimeModeInfo = controlsInfo.find(&controls::ExposureTimeMode); + if (exposureTimeModeInfo != controlsInfo.end()) { + for (const auto &value : exposureTimeModeInfo->second.values()) { + switch (value.get()) { + case controls::ExposureTimeModeAuto: + if (!aeAddedOn) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_ON); + aeAddedOn = true; + break; + case controls::ExposureTimeModeDisabled: + if (!aeAddedOff) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_OFF); + aeAddedOff = true; + break; + default: + break; + } + } + } + + if (aeAvailableModes.empty()) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_ON); staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES, aeAvailableModes); + /* In libcamera, turning off AE is equivalient to locking. */ + uint8_t aeLockAvailable = aeAddedOff ? ANDROID_CONTROL_AE_LOCK_AVAILABLE_TRUE + : ANDROID_CONTROL_AE_LOCK_AVAILABLE_FALSE; + staticMetadata_->addEntry(ANDROID_CONTROL_AE_LOCK_AVAILABLE, + aeLockAvailable); + int64_t minFrameDurationNsec = -1; int64_t maxFrameDurationNsec = -1; const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits); @@ -946,10 +996,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_SCENE_MODE_OVERRIDES, sceneModesOverride); - uint8_t aeLockAvailable = ANDROID_CONTROL_AE_LOCK_AVAILABLE_FALSE; - staticMetadata_->addEntry(ANDROID_CONTROL_AE_LOCK_AVAILABLE, - aeLockAvailable); - uint8_t awbLockAvailable = ANDROID_CONTROL_AWB_LOCK_AVAILABLE_FALSE; staticMetadata_->addEntry(ANDROID_CONTROL_AWB_LOCK_AVAILABLE, awbLockAvailable); @@ -1358,6 +1404,9 @@ std::unique_ptr CameraCapabilities::requestTemplateManual() cons if (!manualTemplate) return nullptr; + uint8_t aeMode = ANDROID_CONTROL_AE_MODE_OFF; + manualTemplate->addEntry(ANDROID_CONTROL_AE_MODE, aeMode); + return manualTemplate; } diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index ef4fbab8..d5027ec5 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -263,7 +263,9 @@ CameraDevice::Camera3RequestDescriptor::Camera3RequestDescriptor( CameraDevice::CameraDevice(unsigned int id, std::shared_ptr camera) : id_(id), state_(State::Stopped), camera_(std::move(camera)), - facing_(CAMERA_FACING_FRONT), orientation_(0) + facing_(CAMERA_FACING_FRONT), orientation_(0), + aeOn_(true), aeLocked_(false), lastExposureTime_(0), + lastAnalogueGain_(1.0f), lastDigitalGain_(1.0f) { camera_->requestCompleted.connect(this, &CameraDevice::requestComplete); @@ -857,6 +859,62 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::draft::TestPatternMode, testPatternMode); } + /* + * \todo Can we trust that we won't receive values that we didn't + * report supporting? + */ + if (settings.getEntry(ANDROID_CONTROL_AE_LOCK, &entry)) + aeLocked_ = *entry.data.u8 != ANDROID_CONTROL_AE_LOCK_OFF; + + if (settings.getEntry(ANDROID_CONTROL_AE_MODE, &entry)) + aeOn_ = *entry.data.u8 != ANDROID_CONTROL_AE_MODE_OFF; + + AeMode aeMode; + if (aeLocked_ && aeOn_) + aeMode = AeMode::Lock; + else if (aeLocked_ && !aeOn_) + aeMode = AeMode::Manual; + else if (!aeLocked_ && aeOn_) + aeMode = AeMode::Auto; + else /* !aeLocked_ && !aeOn_ */ + aeMode = AeMode::Manual; + + /* Save this so that we can recover it in the result */ + descriptor->aeMode_ = aeMode; + + const auto &eInfo = camera_->controls().find(&controls::ExposureTimeMode); + if (eInfo != camera_->controls().end()) { + controls.set(controls::ExposureTimeMode, + aeMode == AeMode::Auto ? + controls::ExposureTimeModeAuto : + controls::ExposureTimeModeDisabled); + } + + const auto &gInfo = camera_->controls().find(&controls::AnalogueGainMode); + if (gInfo != camera_->controls().end()) { + controls.set(controls::AnalogueGainMode, + aeMode == AeMode::Auto ? + controls::AnalogueGainModeAuto : + controls::AnalogueGainModeDisabled); + } + + if (settings.getEntry(ANDROID_SENSOR_EXPOSURE_TIME, &entry)) { + const auto &info = camera_->controls().find(&controls::ExposureTime); + if (info != camera_->controls().end()) { + lastExposureTime_ = (*entry.data.i64) / 1000; + /* Don't disable libcamera's internal AeMode::Lock */ + if (aeMode != AeMode::Lock) + controls.set(controls::ExposureTime, lastExposureTime_); + } + } + + /* Trigger libcamera's locked -> manual state change */ + if (aeMode == AeMode::Manual && !settings.hasEntry(ANDROID_SENSOR_EXPOSURE_TIME)) { + const auto &info = camera_->controls().find(&controls::ExposureTime); + if (info != camera_->controls().end()) + controls.set(controls::ExposureTime, lastExposureTime_); + } + return 0; } @@ -1345,11 +1403,16 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_CONTROL_AE_EXPOSURE_COMPENSATION, value32); - value = ANDROID_CONTROL_AE_LOCK_OFF; - resultMetadata->addEntry(ANDROID_CONTROL_AE_LOCK, value); + uint8_t aeLock = (descriptor.aeMode_ == AeMode::Lock) + ? ANDROID_CONTROL_AE_LOCK_ON + : ANDROID_CONTROL_AE_LOCK_OFF; + resultMetadata->addEntry(ANDROID_CONTROL_AE_LOCK, aeLock); - value = ANDROID_CONTROL_AE_MODE_ON; - resultMetadata->addEntry(ANDROID_CONTROL_AE_MODE, value); + /* Locked means auto + locked in android */ + uint8_t aeMode = (descriptor.aeMode_ != AeMode::Manual) + ? ANDROID_CONTROL_AE_MODE_ON + : ANDROID_CONTROL_AE_MODE_OFF; + resultMetadata->addEntry(ANDROID_CONTROL_AE_MODE, aeMode); if (settings.getEntry(ANDROID_CONTROL_AE_TARGET_FPS_RANGE, &entry)) /* @@ -1366,6 +1429,31 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_CONTROL_AE_PRECAPTURE_TRIGGER, value); value = ANDROID_CONTROL_AE_STATE_CONVERGED; + if (metadata.contains(controls::AeState)) { + switch (metadata.get(controls::AeState)) { + case controls::AeStateInactive: + value = ANDROID_CONTROL_AE_STATE_INACTIVE; + break; + case controls::AeStateSearching: + value = ANDROID_CONTROL_AE_STATE_SEARCHING; + break; + case controls::AeStateConverged: + value = ANDROID_CONTROL_AE_STATE_CONVERGED; + break; + case controls::AeStateFlashRequired: + value = ANDROID_CONTROL_AE_STATE_FLASH_REQUIRED; + break; + case controls::AeStatePrecapture: + value = ANDROID_CONTROL_AE_STATE_PRECAPTURE; + break; + default: + if (descriptor.aeMode_ == AeMode::Lock) { + value = ANDROID_CONTROL_AE_STATE_LOCKED; + break; + } + LOG(HAL, Error) << "Invalid AeState, setting converged"; + } + } resultMetadata->addEntry(ANDROID_CONTROL_AE_STATE, value); value = ANDROID_CONTROL_AF_MODE_OFF; diff --git a/src/android/camera_device.h b/src/android/camera_device.h index b7d774fe..f693cdbc 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -73,6 +73,12 @@ private: CameraDevice(unsigned int id, std::shared_ptr camera); + enum class AeMode { + Auto, + Lock, + Manual, + }; + struct Camera3RequestDescriptor { enum class Status { Pending, @@ -96,6 +102,9 @@ private: camera3_capture_result_t captureResult_ = {}; Status status_ = Status::Pending; + + /* The libcamera internal AE state for this request */ + AeMode aeMode_ = AeMode::Auto; }; enum class State { @@ -146,6 +155,13 @@ private: int facing_; int orientation_; + /* Track the last-set android AE controls */ + bool aeOn_; + bool aeLocked_; + int32_t lastExposureTime_; + float lastAnalogueGain_; + float lastDigitalGain_; + CameraMetadata lastSettings_; }; From patchwork Fri Oct 1 10:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14022 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD7B3BDC71 for ; Fri, 1 Oct 2021 10:33:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AB748691BB; Fri, 1 Oct 2021 12:33:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jkvv5wqh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1CD65691AC for ; Fri, 1 Oct 2021 12:33:50 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A74F1596; Fri, 1 Oct 2021 12:33:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084429; bh=K9auD2iBnOpNG4ZMLIaruFtpf06EYn/2zB6+cFeCFGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jkvv5wqh03b48sSkc6TyQECEo0V5/+QZniIsUHS4ZP9SjQ0qUucOZTQQG57IdNRvR 1hg+iLj4iZUPkH+8Jx5vhdHYsX6bZvCa9NDM+eM7GFzFO6ORn+7IHwbOAmB9s7MagJ cKhBs4Vy+hwFCflQDwnMQFStGe6p6tE5wSJzAS4A= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:25 +0900 Message-Id: <20211001103325.1077590-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 7/7] android: Plumb all sensitivity-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Plumb through the HAL the following three controls: - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) - ANDROID_SENSOR_SENSITIVITY (request, result) Also add a minISO to the HAL config, and add appropriate capability detection. The sensitivity range comes from: - min: hardcode to 100, or take minISO from the HAL config - max: set to minISO * max(AnalogueGain) * max(DigitalGain) The max analog sensitivity comes from: - hardcode to minISO * max(AnalogueGain) The request sensitivity is mapped to requested ISO / minISO, and split between analog and digital gain. The result sensitivity is mapped from minISO * analog gain * digital gain. Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- No changes in v2 - I'm moving this into the AE-related series, because this is AE-related --- src/android/camera_capabilities.cpp | 61 ++++++++++++++++++++--- src/android/camera_capabilities.h | 3 +- src/android/camera_device.cpp | 76 ++++++++++++++++++++++++++++- src/android/camera_device.h | 1 + src/android/camera_hal_config.cpp | 10 +++- src/android/camera_hal_config.h | 1 + 6 files changed, 142 insertions(+), 10 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 3fed3f83..8e84a736 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -233,6 +233,15 @@ bool CameraCapabilities::validateManualSensorCapability() return false; } + /* + * Checking the sensitivity range is sufficient, as it also covers the + * max analog sensitivity and the sensor sensitivity request/result key + */ + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE)) { + LOG(HAL, Info) << noMode << "missing sensitivity range"; + return false; + } + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE)) { LOG(HAL, Info) << noMode << "missing exposure time range"; return false; @@ -401,11 +410,12 @@ void CameraCapabilities::computeHwLevel( } int CameraCapabilities::initialize(std::shared_ptr camera, - int orientation, int facing) + int orientation, int facing, int minISO) { camera_ = camera; orientation_ = orientation; facing_ = facing; + minISO_ = minISO; rawStreamAvailable_ = false; /* Acquire the camera and initialize available stream configurations. */ @@ -721,7 +731,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, ANDROID_SENSOR_INFO_PHYSICAL_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, ANDROID_SENSOR_ORIENTATION, ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, @@ -1077,11 +1086,49 @@ int CameraCapabilities::initializeStaticMetadata() data); } - int32_t sensitivityRange[] = { - 32, 2400, - }; - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, - sensitivityRange); + /* + * The following three controls are only necessary for FULL + * (specifically the manual sensor capability): + * - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) + * - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) + * - ANDROID_SENSOR_SENSITIVITY (request, result) + * + * The first and second depend on analog gain, and the third depend on + * both, so if analog gain is not available then we can cross out all + * of them. We'll default to 100 minimum sensitivity if it isn't + * specified in the HAL configuration file. + * + * Digital gain is optional; if it's unavailable then the max analog + * sensitivity will be equal to the max sensitivity range. + * + * The minimum sensitivity must be at most 100, while the maximum must + * be at least 800. + */ + const auto &analogGainInfo = controlsInfo.find(&controls::AnalogueGain); + if (analogGainInfo != controlsInfo.end()) { + float maxAnalogGain = analogGainInfo->second.max().get(); + + const auto &digitalGainInfo = controlsInfo.find(&controls::DigitalGain); + float maxDigitalGain = digitalGainInfo == controlsInfo.end() ? + 1.0f : + digitalGainInfo->second.max().get(); + + int32_t maxISO = minISO_ * maxAnalogGain * maxDigitalGain; + int32_t sensitivityRange[] = { + minISO_, + maxISO < 800 ? 800 : maxISO, + }; + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, + sensitivityRange); + + staticMetadata_->addEntry(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY, + minISO_ * maxAnalogGain); + + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE); + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY); + availableRequestKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + availableResultKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + } /* Report the color filter arrangement if the camera reports it. */ if (properties.contains(properties::draft::ColorFilterArrangement)) { diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index a1259699..b1eee37f 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -26,7 +26,7 @@ public: CameraCapabilities() = default; int initialize(std::shared_ptr camera, - int orientation, int facing); + int orientation, int facing, int minISO); CameraMetadata *staticMetadata() const { return staticMetadata_.get(); } libcamera::PixelFormat toPixelFormat(int format) const; @@ -68,6 +68,7 @@ private: int facing_; int orientation_; + int minISO_; bool rawStreamAvailable_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; std::set capabilities_; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index d5027ec5..8854a960 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -393,7 +393,15 @@ int CameraDevice::initialize(const CameraConfigData *cameraConfigData) orientation_ = 0; } - return capabilities_.initialize(camera_, orientation_, facing_); + if (cameraConfigData && cameraConfigData->minISO != -1) { + minISO_ = cameraConfigData->minISO; + } else { + LOG(HAL, Error) + << "Minimum ISO not in configuration file. Using 100."; + minISO_ = 100; + } + + return capabilities_.initialize(camera_, orientation_, facing_, minISO_); } /* @@ -908,6 +916,51 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) } } + if (settings.getEntry(ANDROID_SENSOR_SENSITIVITY, &entry)) { + const auto &info = camera_->controls().find(&controls::AnalogueGain); + if (info != camera_->controls().end()) { + const auto &dInfo = + camera_->controls().find(&controls::DigitalGain); + bool digitalGainAvailable = + dInfo == camera_->controls().end(); + + float maxAnalogGain = info->second.max().get(); + float maxDigitalGain = digitalGainAvailable ? + dInfo->second.max().get() : 1.0f; + + /* target ISO / min ISO = gain */ + float gain = *entry.data.i32 / minISO_; + + /* + * Max out analog gain before applying digital gain, if + * digital gain is available. + */ + bool setDigital = false; + if (gain <= maxAnalogGain) { + lastAnalogueGain_ = gain; + } else if (gain <= maxAnalogGain * maxDigitalGain) { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) { + setDigital = true; + lastDigitalGain_ = gain / maxAnalogGain; + } + } else { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) { + setDigital = true; + lastDigitalGain_ = maxDigitalGain; + } + } + + /* Don't disable libcamera's internal AeMode::Lock */ + if (aeMode != AeMode::Lock) + controls.set(controls::AnalogueGain, lastAnalogueGain_); + + if (setDigital && (aeMode != AeMode::Lock)) + controls.set(controls::DigitalGain, lastDigitalGain_); + } + } + /* Trigger libcamera's locked -> manual state change */ if (aeMode == AeMode::Manual && !settings.hasEntry(ANDROID_SENSOR_EXPOSURE_TIME)) { const auto &info = camera_->controls().find(&controls::ExposureTime); @@ -915,6 +968,17 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ExposureTime, lastExposureTime_); } + /* Trigger libcamera's locked -> manual state change */ + if (aeMode == AeMode::Manual && !settings.hasEntry(ANDROID_SENSOR_SENSITIVITY)) { + const auto &aInfo = camera_->controls().find(&controls::AnalogueGain); + if (aInfo != camera_->controls().end()) + controls.set(controls::AnalogueGain, lastAnalogueGain_); + + const auto &dInfo = camera_->controls().find(&controls::DigitalGain); + if (dInfo != camera_->controls().end()) + controls.set(controls::DigitalGain, lastDigitalGain_); + } + return 0; } @@ -1548,6 +1612,16 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, exposure); } + if (metadata.contains(controls::AnalogueGain) && + settings.hasEntry(ANDROID_SENSOR_SENSITIVITY)) { + bool hasDigitalGain = metadata.contains(controls::DigitalGain); + float analogGain = metadata.get(controls::AnalogueGain); + float digitalGain = hasDigitalGain ? metadata.get(controls::DigitalGain) : 1.0f; + + resultMetadata->addEntry(ANDROID_SENSOR_SENSITIVITY, + minISO_ * analogGain * digitalGain); + } + if (metadata.contains(controls::FrameDuration)) { int64_t duration = metadata.get(controls::FrameDuration) * 1000; resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, diff --git a/src/android/camera_device.h b/src/android/camera_device.h index f693cdbc..e119dfd9 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -154,6 +154,7 @@ private: int facing_; int orientation_; + int minISO_; /* Track the last-set android AE controls */ bool aeOn_; diff --git a/src/android/camera_hal_config.cpp b/src/android/camera_hal_config.cpp index aa90dac7..863cdff0 100644 --- a/src/android/camera_hal_config.cpp +++ b/src/android/camera_hal_config.cpp @@ -180,6 +180,13 @@ int CameraHalConfig::Private::parseCameraConfigData(const std::string &cameraId) return -EINVAL; } cameraConfigData.rotation = ret; + } else if (key == "minISO") { + ret = std::stoi(value); + if (ret < 0 || ret > 100) { + LOG(HALConfig, Error) + << "Invalid minimum ISO: " << value; + return -EINVAL; + } } else { LOG(HALConfig, Error) << "Unknown key: " << key; @@ -384,7 +391,8 @@ int CameraHalConfig::parseConfigurationFile() const CameraConfigData &camera = c.second; LOG(HALConfig, Debug) << "'" << cameraId << "' " << "(" << camera.facing << ")[" - << camera.rotation << "]"; + << camera.rotation << "], " + << "minISO: " << camera.minISO; } return 0; diff --git a/src/android/camera_hal_config.h b/src/android/camera_hal_config.h index a79d5d6c..afa02a9b 100644 --- a/src/android/camera_hal_config.h +++ b/src/android/camera_hal_config.h @@ -15,6 +15,7 @@ struct CameraConfigData { int facing = -1; int rotation = -1; + int minISO = -1; }; class CameraHalConfig final : public libcamera::Extensible