From patchwork Tue Dec 21 04:36:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15179 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C071EBE080 for ; Tue, 21 Dec 2021 04:36:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 361DC608F9; Tue, 21 Dec 2021 05:36:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="krON6oLD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C438E60592 for ; Tue, 21 Dec 2021 05:36:20 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C2223FD2; Tue, 21 Dec 2021 05:36:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061380; bh=g52XoS8OdmFvRJ7mJBhylmfhTKgyygElo9/q5i1W9B0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=krON6oLDzmsnrcmvRec9CPTs6g29dAX2F2bF59m/HGq5vTVqsyCA0sJyMiwGHXmWO RSQFkOkE6zwHN31Mm5DT1AenpQN4635TbhuqY1wESX8UWxUs5e2B05BA+oviUtmnpg fVla4cb92gjxWLQ+FvzjLy0umRM/PhoQFCXNNsik= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:03 -0600 Message-Id: <20211221043610.2512334-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/8] controls: Reorganize the AE-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We have multiple goals: - we need a lock of some sort, to instruct the AEGC to not update output results - we need manual modes, to override the values computed by the AEGC - we need to support seamless transitions from auto -> manual, and do so without flickering - we need custom minimum values for the manual controls, that is no magic values for enabling/disabling auto - all of these need to be done with AE sub-controls (exposure time, analogue gain) To achieve these goals, we introduce mode controls for the AE sub-controls: ExposureTimeMode and AnalogueGainMode. These have an auto state, and a disabled state. The disabled state has an internal one-way state change from locked to manual, triggered by the presence of the value-controls (ExposureTime and AnalogueGain). We then remove the AeEnable control, as it is a redundant control in the face of these two mode controls. We also remove AeLocked, as it is insufficient for reporting the AE state, and we promote AeState to non-draft to fill its role. Notably, the locked state is removed, since this information can be obtained from the aforementioned mode controls. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- Changes in v3: - improve wording of the control descriptions - make more succinct and clear - add description of how to do a flickerless transition Changes in v2: - No changes, just resubmitting at the head of this series so that it's together and so that /people will actually see it/ Initial version: Still RFC as I haven't updated the users of the control yet, and I want to check that these are the controls and docs that we want. We've decided that the "master AE control" will be implemented by a helper... but looking at uvcvideo and the V4L2 controls I'm wondering if such helper should come earlier than later? --- src/libcamera/control_ids.yaml | 239 ++++++++++++++++++++++++--------- 1 file changed, 177 insertions(+), 62 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index 9d4638ae..84679317 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -7,23 +7,63 @@ # Unless otherwise stated, all controls are bi-directional, i.e. they can be # set through Request::controls() and returned out through Request::metadata(). controls: - - AeEnable: - type: bool + - AeState: + type: int32_t description: | - Enable or disable the AE. + Control to report the current AE algorithm state. The camera device can do + several state transitions between two results, if it is allowed by the + state transition table. For example, AeStateInactive may never + actually be seen in a result. - \sa ExposureTime AnalogueGain + The state in the result is the state for this request. If AE state + becomes AeStateConverged, then the image data associated with the + result should be good to use. - - AeLocked: - type: bool - description: | - Report the lock status of a running AE algorithm. + The state is still reported even if ExposureTimeMode or + AnalogueGainMode is set to Disabled. - If the AE algorithm is locked the value shall be set to true, if it's - converging it shall be set to false. If the AE algorithm is not - running the control shall not be present in the metadata control list. + \sa AnalogueGain + \sa AnalogueGainMode + \sa ExposureTime + \sa ExposureTimeMode - \sa AeEnable + enum: + - name: AeStateInactive + value: 0 + description: | + The AE algorithm is inactive. + If the camera initiates an AE scan, the state shall go to + AeStateSearching. + + Unlike the analogous control in Android, enabling or disabling any + of the AE-related mode controls (eg. AnalogueGainMode, + ExposureTimeMode) does not require the state to be reset to to + AeStateInactive. + - name: AeStateSearching + value: 1 + description: | + The AE algorithm has not converged yet. + If the camera finishes an AE scan, the state shall go to + AeStateConverged. If the camera finishes an AE scan, but flash is + required, the state shall go to AeStateFlashRequired. + - name: AeStateConverged + value: 2 + description: | + The AE algorithm has converged. + If the camera initiates an AE scan, the state shall go to + AeStateSearching. + - name: AeStateFlashRequired + value: 3 + description: | + The AE algorithm would need a flash for good results. + If the camera initiates an AE scan, the state shall go to + AeStateSearching. + - name: AeStatePrecapture + value: 4 + description: | + The AE algorithm has started a pre-capture metering session. + After the sequence is finished, the state shall go to AeStateConverged if + \sa AePrecaptureTrigger # AeMeteringMode needs further attention: # - Auto-generate max enum value. @@ -93,6 +133,13 @@ controls: how the desired total exposure is divided between the shutter time and the sensor's analogue gain. The exposure modes are platform specific, and not all exposure modes may be supported. + + When one of AnalogueGainMode or ExposureTimeMode is set to Disabled, + the fixed values will override any choices made by AeExposureMode. + + \sa AnalogueGainMode + \sa ExposureTimeMode + enum: - name: ExposureNormal value: 0 @@ -111,13 +158,15 @@ controls: type: float description: | Specify an Exposure Value (EV) parameter. The EV parameter will only be - applied if the AE algorithm is currently enabled. + applied if the AE algorithm is currently enabled, that is, at least one + of AnalogueGainMode and ExposureTimeMode are auto. By convention EV adjusts the exposure as log2. For example EV = [-2, -1, 0.5, 0, 0.5, 1, 2] results in an exposure adjustment of [1/4x, 1/2x, 1/sqrt(2)x, 1x, sqrt(2)x, 2x, 4x]. - \sa AeEnable + \sa AnalogueGainMode + \sa ExposureTimeMode - ExposureTime: type: int32_t @@ -125,17 +174,65 @@ controls: Exposure time (shutter speed) for the frame applied in the sensor device. This value is specified in micro-seconds. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. Its + presence in a request acts as a trigger to switch to the internal + manual mode within ExposureTimeModeDisabled. + + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. + + \sa AnalogueGain + \sa ExposureTimeMode - \sa AnalogueGain AeEnable + - ExposureTimeMode: + type: int32_t + description: | + Controls how the frame exposure time is computed. When set to Auto, the + AE algorithm computes the exposure time of the next frame and + configures the image sensor accordingly. When set to Disabled, the AE + algorithm stops updating the exposure time. - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + As the camera device has a pipeline of in-flight requests, there is a + period of time between submitting a request with ExposureTimeMode set + to Disabled and the exposure time component of the AE actually being + disabled, during which the AE algorithm can still update the exposure + time. If an application is switching from automatic and manual control + and wishes to eliminate any flicker during the switch, the following + procedure is recommended. + + 1. Start with ExposureTimeMode set to Auto + + 2. Set ExposureTimeMode to Disabled + + 3. Wait for the first request to be output that has ExposureTimeMode + set to Disabled + + 4. Copy the value reported in ExposureTime into a new request, and + submit it + + 5. Proceed to run manual exposure time + + \sa ExposureTime + enum: + - name: ExposureTimeModeAuto + value: 0 + description: | + The exposure time will be calculated automatically and set by the + AE algorithm. + - name: ExposureTimeModeDisabled + value: 1 + description: | + The exposure time will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in ExposureTime. + + When transitioning from Auto to Disabled mode the last computed + exposure value is used until a new value is specified through the + ExposureTime control. If an ExposureTime value is specified in the + same request where the ExposureTimeMode is set to Disabled, the + provided ExposureTime is applied immediately. - AnalogueGain: type: float @@ -144,17 +241,65 @@ controls: The value of the control specifies the gain multiplier applied to all colour channels. This value cannot be lower than 1.0. - Setting this value means that it is now fixed and the AE algorithm may - not change it. Setting it back to zero returns it to the control of the - AE algorithm. + This control will only take effect if ExposureTimeMode is Disabled. Its + presence in a request acts as a trigger to switch to the internal + manual mode within ExposureTimeModeDisabled. + + When reported in metadata, this control indicates what exposure time + was used for the current request, regardless of ExposureTimeMode. + ExposureTimeMode will indicate the source of the exposure time value, + whether it came from the AE algorithm or not. + + \sa ExposureTime + \sa AnalogueGainMode + + - AnalogueGainMode: + type: int32_t + description: | + Controls how the analogue gain is computed. When set to Auto, the + AE algorithm computes the analogue gain of the next frame and + configures the image sensor accordingly. When set to Disabled, the AE + algorithm stops updating the analogue gain. + + As the camera device has a pipeline of in-flight requests, there is a + period of time between submitting a request with AnalogueGainMode set + to Disabled and the analogue gain component of the AE actually being + disabled, during which the AE algorithm can still update the analogue + gain. If an application is switching from automatic and manual control + and wishes to eliminate any flicker during the switch, the following + procedure is recommended. + + 1. Start with AnalogueGainMode set to Auto - \sa ExposureTime AeEnable + 2. Set AnalogueGainMode to Disabled - \todo Document the interactions between AeEnable and setting a fixed - value for this control. Consider interactions with other AE features, - such as aperture and aperture/shutter priority mode, and decide if - control of which features should be automatically adjusted shouldn't - better be handled through a separate AE mode control. + 3. Wait for the first request to be output that has AnalogueGainMode + set to Disabled + + 4. Copy the value reported in AnalogueGain into a new request, and + submit it + + 5. Proceed to run manual analogue gain + + \sa AnalogueGain + enum: + - name: AnalogueGainModeAuto + value: 0 + description: | + The analogue gain will be calculated automatically and set by the + AE algorithm. + - name: AnalogueGainModeDisabled + value: 1 + description: | + The analogue gain will not be updated by the AE algorithm. It will + come from the last calculated value when the mode was Auto, or from + the value specified in AnalogueGain. + + When transitioning from Auto to Disabled mode the last computed + gain value is used until a new value is specified through the + AnalogueGain control. If an AnalogueGain value is specified in the + same request where the AnalogueGainMode is set to Disabled, the + provided AnalogueGain is applied immediately. - Brightness: type: float @@ -477,36 +622,6 @@ controls: High quality aberration correction which might reduce the frame rate. - - AeState: - type: int32_t - draft: true - description: | - Control to report the current AE algorithm state. Currently identical to - ANDROID_CONTROL_AE_STATE. - - Current state of the AE algorithm. - enum: - - name: AeStateInactive - value: 0 - description: The AE algorithm is inactive. - - name: AeStateSearching - value: 1 - description: The AE algorithm has not converged yet. - - name: AeStateConverged - value: 2 - description: The AE algorithm has converged. - - name: AeStateLocked - value: 3 - description: The AE algorithm is locked. - - name: AeStateFlashRequired - value: 4 - description: The AE algorithm would need a flash for good results - - name: AeStatePrecapture - value: 5 - description: | - The AE algorithm has started a pre-capture metering session. - \sa AePrecaptureTrigger - - AfState: type: int32_t draft: true From patchwork Tue Dec 21 04:36:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15180 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DF0FBBE080 for ; Tue, 21 Dec 2021 04:36:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 588A460905; Tue, 21 Dec 2021 05:36:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="U+E6phbG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB7756021D for ; Tue, 21 Dec 2021 05:36:21 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 06489881; Tue, 21 Dec 2021 05:36:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061381; bh=fbY+QoPion+Cw2pf5Jk7DtTS9syhd/QPAWaXpuqpB/c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U+E6phbGRJg/FRulfG+kr9joN3TPbxToGeySvh7/zJpKkDx4u0up6M02AknNoBRfc bD4EtwPZGOGFr5sIzsBmrAxtsIXEmNqCBP285uBWKnd7+QDW7EFEmo/e+txcfvzd5a cv/65A8SIMf/10eH2XbIeLG3gA7LvMxTYuNlreSs= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:04 -0600 Message-Id: <20211221043610.2512334-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/8] libcamera: pipeline: uvcvideo: Support the new AE controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the new AE controls in the uvcvideo pipeline handler. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- Changes in v3: - fix the construction of the ControlInfo of enum values - fix auto gain in processControl - improve auto exposure in processControl Changes in v2: - fix the rebase error where some uvc stuff was in rasberrypi --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 43 +++++++++++++++++--- 1 file changed, 38 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 40654a0b..ee17add8 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -268,8 +268,10 @@ int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id, cid = V4L2_CID_CONTRAST; else if (id == controls::Saturation) cid = V4L2_CID_SATURATION; - else if (id == controls::AeEnable) + else if (id == controls::ExposureTimeMode) cid = V4L2_CID_EXPOSURE_AUTO; + else if (id == controls::AnalogueGainMode) + cid = V4L2_CID_AUTOGAIN; else if (id == controls::ExposureTime) cid = V4L2_CID_EXPOSURE_ABSOLUTE; else if (id == controls::AnalogueGain) @@ -302,13 +304,44 @@ int PipelineHandlerUVC::processControl(ControlList *controls, unsigned int id, } case V4L2_CID_EXPOSURE_AUTO: { - int32_t ivalue = value.get() - ? V4L2_EXPOSURE_APERTURE_PRIORITY + bool autoGainSet = controls->contains(V4L2_CID_AUTOGAIN); + + int32_t ivalue; + if (autoGainSet) { + bool autoGain = controls->get(V4L2_CID_AUTOGAIN).get(); + ivalue = value.get() == controls::ExposureTimeModeAuto + ? (autoGain + /* Both exposure time and gain are auto */ + ? V4L2_EXPOSURE_AUTO + /* Exposure time is auto but gain is manual */ + : V4L2_EXPOSURE_APERTURE_PRIORITY) + : (autoGain + /* Exposure time is manual but gain is auto */ + ? V4L2_EXPOSURE_SHUTTER_PRIORITY + /* Both exposure time and gain are manual */ + : V4L2_EXPOSURE_MANUAL); + } else { + /* + * auto gain is not set, so simply set the auto exposure + * mode from what's set in the libcamera control. + * + * \todo Figure out how to properly handle priority + * when auto exposure is set before auto gain + */ + ivalue = value.get() == controls::ExposureTimeModeAuto + ? V4L2_EXPOSURE_AUTO : V4L2_EXPOSURE_MANUAL; + } + controls->set(V4L2_CID_EXPOSURE_AUTO, ivalue); + break; } + case V4L2_CID_AUTOGAIN: + controls->set(V4L2_CID_AUTOGAIN, value.get()); + break; + case V4L2_CID_EXPOSURE_ABSOLUTE: controls->set(cid, value.get() / 100); break; @@ -559,7 +592,7 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, id = &controls::Saturation; break; case V4L2_CID_EXPOSURE_AUTO: - id = &controls::AeEnable; + id = &controls::ExposureTimeMode; break; case V4L2_CID_EXPOSURE_ABSOLUTE: id = &controls::ExposureTime; @@ -610,7 +643,7 @@ void UVCCameraData::addControl(uint32_t cid, const ControlInfo &v4l2Info, break; case V4L2_CID_EXPOSURE_AUTO: - info = ControlInfo{ false, true, true }; + info = ControlInfo(controls::ExposureTimeModeValues); break; case V4L2_CID_EXPOSURE_ABSOLUTE: From patchwork Tue Dec 21 04:36:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15181 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E2C18C3258 for ; Tue, 21 Dec 2021 04:36:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B71B608FD; Tue, 21 Dec 2021 05:36:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pjmfHhYy"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CCF94608E8 for ; Tue, 21 Dec 2021 05:36:22 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0B8D2881; Tue, 21 Dec 2021 05:36:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061382; bh=xeuyaNCaNS1VCUnxH5qrJU+44XXUrBA/zwpl1s4vbto=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pjmfHhYyEGo4t7xMK3N6kS1ZluiEtQNreteTu9HssvSE2jZITFrWNUSbDieJGadYn oG4EXVc02QsYZxZuDDSeFLRfOBb8RX56ZDjI3GbMdRjbN7YRPMAP8kbcQCqJWReeSD 07cfBs6oO54rT/Nl5GKDWeLpZMAWIuOG1L6mwj9k= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:05 -0600 Message-Id: <20211221043610.2512334-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/8] libcamera: pipeline: raspberrypi: Support the new AE controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the new AE controls in the raspberrypi pipeline handler, and in the IPA. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- Changes in v3: - s/Exposure/Shutter/ - move the new Pause/Resume functions to AgcAlgorithm - update callers as appropriate - make the original Pause/Resume functions call the new ones Changes in v2: - fix the rebase error where some uvc stuff was in rasberrypi - i haven't yet taken in the comments to move the new Pause/Resume functions Initial versoin: This is very hacky. I wasn't sure what the best way was to plumb it into the raspberrypi IPA as it was a bit hairy... --- include/libcamera/ipa/raspberrypi.h | 3 +- .../raspberrypi/controller/agc_algorithm.hpp | 4 ++ src/ipa/raspberrypi/controller/rpi/agc.cpp | 22 +++++++++- src/ipa/raspberrypi/controller/rpi/agc.hpp | 5 +++ src/ipa/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++---- 5 files changed, 66 insertions(+), 10 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 7f705e49..672b85a5 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -28,8 +28,9 @@ namespace RPi { * unsupported control is encountered. */ static const ControlInfoMap Controls({ - { &controls::AeEnable, ControlInfo(false, true) }, + { &controls::ExposureTimeMode, ControlInfo(controls::ExposureTimeModeValues) }, { &controls::ExposureTime, ControlInfo(0, 999999) }, + { &controls::AnalogueGainMode, ControlInfo(controls::AnalogueGainModeValues) }, { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, { &controls::AeMeteringMode, ControlInfo(controls::AeMeteringModeValues) }, { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, diff --git a/src/ipa/raspberrypi/controller/agc_algorithm.hpp b/src/ipa/raspberrypi/controller/agc_algorithm.hpp index 61595ea2..538cfa88 100644 --- a/src/ipa/raspberrypi/controller/agc_algorithm.hpp +++ b/src/ipa/raspberrypi/controller/agc_algorithm.hpp @@ -27,6 +27,10 @@ public: virtual void SetExposureMode(std::string const &exposure_mode_name) = 0; virtual void SetConstraintMode(std::string const &contraint_mode_name) = 0; + virtual void PauseShutter(); + virtual void PauseGain(); + virtual void ResumeShutter(); + virtual void ResumeGain(); }; } // namespace RPiController diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index f6a9cb0a..14d6e024 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -210,14 +210,34 @@ bool Agc::IsPaused() const } void Agc::Pause() +{ + PauseShutter(); + PauseGain(); +} + +void Agc::Resume() +{ + ResumeShutter(); + ResumeGain(); +} + +void Agc::PauseShutter() { fixed_shutter_ = status_.shutter_time; +} + +void Agc::PauseGain() +{ fixed_analogue_gain_ = status_.analogue_gain; } -void Agc::Resume() +void Agc::ResumeShutter() { fixed_shutter_ = 0s; +} + +void Agc::ResumeGain() +{ fixed_analogue_gain_ = 0; } diff --git a/src/ipa/raspberrypi/controller/rpi/agc.hpp b/src/ipa/raspberrypi/controller/rpi/agc.hpp index c100d312..af4c72ff 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.hpp @@ -90,6 +90,11 @@ public: void Prepare(Metadata *image_metadata) override; void Process(StatisticsPtr &stats, Metadata *image_metadata) override; + void PauseShutter() override; + void PauseGain() override; + void ResumeShutter() override; + void ResumeGain() override; + private: void updateLockStatus(DeviceStatus const &device_status); AgcConfig config_; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 0ed41385..9bea16e7 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -53,6 +53,8 @@ #include "sharpen_algorithm.hpp" #include "sharpen_status.h" +#include "controller/rpi/agc.hpp" + namespace libcamera { using namespace std::literals::chrono_literals; @@ -483,7 +485,10 @@ void IPARPi::reportMetadata() AgcStatus *agcStatus = rpiMetadata_.GetLocked("agc.status"); if (agcStatus) { - libcameraMetadata_.set(controls::AeLocked, agcStatus->locked); + libcameraMetadata_.set(controls::AeState, + agcStatus->locked ? + controls::AeStateConverged : + controls::AeStateSearching); libcameraMetadata_.set(controls::DigitalGain, agcStatus->digital_gain); } @@ -628,20 +633,22 @@ void IPARPi::queueRequest(const ControlList &controls) << " = " << ctrl.second.toString(); switch (ctrl.first) { - case controls::AE_ENABLE: { - RPiController::Algorithm *agc = controller_.GetAlgorithm("agc"); + case controls::EXPOSURE_TIME_MODE: { + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.GetAlgorithm("agc")); if (!agc) { LOG(IPARPI, Warning) - << "Could not set AE_ENABLE - no AGC algorithm"; + << "Could not set EXPOSURE_TIME_MODE - no AGC algorithm"; break; } - if (ctrl.second.get() == false) - agc->Pause(); + if (ctrl.second.get() == controls::ExposureTimeModeDisabled) + agc->PauseShutter(); else - agc->Resume(); + agc->ResumeShutter(); - libcameraMetadata_.set(controls::AeEnable, ctrl.second.get()); + libcameraMetadata_.set(controls::ExposureTimeMode, + ctrl.second.get()); break; } @@ -661,6 +668,25 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::ANALOGUE_GAIN_MODE: { + RPiController::AgcAlgorithm *agc = dynamic_cast( + controller_.GetAlgorithm("agc")); + if (!agc) { + LOG(IPARPI, Warning) + << "Could not set ANALOGUE_GAIN_MODE - no AGC algorithm"; + break; + } + + if (ctrl.second.get() == controls::AnalogueGainModeDisabled) + agc->PauseGain(); + else + agc->ResumeGain(); + + libcameraMetadata_.set(controls::AnalogueGainMode, + ctrl.second.get()); + break; + } + case controls::ANALOGUE_GAIN: { RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.GetAlgorithm("agc")); From patchwork Tue Dec 21 04:36:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15182 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F445BE080 for ; Tue, 21 Dec 2021 04:36:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1937F608EB; Tue, 21 Dec 2021 05:36:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fUG4TCW7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0F781608EB for ; Tue, 21 Dec 2021 05:36:24 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 11762881; Tue, 21 Dec 2021 05:36:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061383; bh=QvLMwWk6M6u1YkWXtbI/cPfWPjM+DLcphCnDGHGh1Jo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fUG4TCW7q8rGc+F4c4KO1kHLO3e9pwl1yjhA6pKXjknfoNR2HyC63x3ewtFBEcETA juwbMp1lzG0zYtLqXzZwEa4BDzroPne/VXT555sjTgMqbm1nWSaG9L4kAmtfZ0QMoQ ijMLe8DE5MrMsVpDfenA9s9KPzKuGLX/8eMi8ALo= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:06 -0600 Message-Id: <20211221043610.2512334-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/8] test: ipa_data_serializer: Remove controls::AeEnable X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove controls::AeEnable from the IPA data serializer test, as it is no longer an available control. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- No change in v2 --- test/serialization/ipa_data_serializer_test.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/test/serialization/ipa_data_serializer_test.cpp b/test/serialization/ipa_data_serializer_test.cpp index d2050a86..454e93ad 100644 --- a/test/serialization/ipa_data_serializer_test.cpp +++ b/test/serialization/ipa_data_serializer_test.cpp @@ -33,7 +33,6 @@ using namespace std; using namespace libcamera; static const ControlInfoMap Controls = ControlInfoMap({ - { &controls::AeEnable, ControlInfo(false, true) }, { &controls::ExposureTime, ControlInfo(0, 999999) }, { &controls::AnalogueGain, ControlInfo(1.0f, 32.0f) }, { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, From patchwork Tue Dec 21 04:36:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15183 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4131C3258 for ; Tue, 21 Dec 2021 04:36:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 80B916090D; Tue, 21 Dec 2021 05:36:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b40D8j0m"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A1F0608EB for ; Tue, 21 Dec 2021 05:36:25 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 493EB881; Tue, 21 Dec 2021 05:36:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061385; bh=aETxTnlwCxl/PzdYj8t+u7nf2NpC+nUu0Tskm2TU8VA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b40D8j0mQ5xkts4K1cJhLXCiKMrrxZyGvNW+7LdibsW3kDyixpFm+i+BWPKXdP/DF /ASY2iGLnD2zvyWFaZc8pqwuHnCBPnrsfsMi4qd+HgxWWeJOvDHgSblhRsQ0XGLR4D 1AWR/lQk3XOmdkhph0udEepkRk5Jl5Hk5bP4Ihp0= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:07 -0600 Message-Id: <20211221043610.2512334-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/8] libcamera: pipeline: rkisp1: Support the new AE controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the new AE controls in the rkisp1 pipeline handler, and in the IPA. Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Signed-off-by: Paul Elder --- No change in v2 Initial version: It seems that the rkisp1 pipeline handler doesn't properly expose its control capabilities? And even though it exposed AeEnable with both true and false it didn't expose or handle AnalogueGain nor ExposureTime either? So I just simply replaced AeEnable with ExposureTimeMode for now. --- src/ipa/rkisp1/rkisp1.cpp | 9 ++++----- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++-- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 2d79f15f..245e9940 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -60,7 +60,7 @@ private: const rkisp1_stat_buffer *stats); void setControls(unsigned int frame); - void metadataReady(unsigned int frame, unsigned int aeState); + void metadataReady(unsigned int frame, int aeState); std::map buffers_; std::map mappedBuffers_; @@ -289,7 +289,7 @@ void IPARkISP1::queueRequest(unsigned int frame, rkisp1_params_cfg *params, void IPARkISP1::updateStatistics(unsigned int frame, const rkisp1_stat_buffer *stats) { - unsigned int aeState = 0; + int aeState = controls::AeStateInactive; for (auto const &algo : algorithms_) algo->process(context_, stats); @@ -315,12 +315,11 @@ void IPARkISP1::setControls(unsigned int frame) queueFrameAction.emit(frame, op); } -void IPARkISP1::metadataReady(unsigned int frame, unsigned int aeState) +void IPARkISP1::metadataReady(unsigned int frame, int aeState) { ControlList ctrls(controls::controls); - if (aeState) - ctrls.set(controls::AeLocked, aeState == 2); + ctrls.set(controls::AeState, aeState); RkISP1Action op; op.op = ActionMetadata; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 8cca8a15..659ece06 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -935,8 +935,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) ControlInfoMap::Map ctrls; ctrls.emplace(std::piecewise_construct, - std::forward_as_tuple(&controls::AeEnable), - std::forward_as_tuple(false, true)); + std::forward_as_tuple(&controls::ExposureTimeMode), + std::forward_as_tuple(ControlInfo(controls::AeExposureModeValues))); data->controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); From patchwork Tue Dec 21 04:36:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15184 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BC0D0C3259 for ; Tue, 21 Dec 2021 04:36:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 326BF60900; Tue, 21 Dec 2021 05:36:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WnU0IEU8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A023608F8 for ; Tue, 21 Dec 2021 05:36:26 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 55BD2881; Tue, 21 Dec 2021 05:36:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061386; bh=MyRNP/dDBWmJxj5zE8GNAUEn8v6pO9tLjUCL9N+TAy8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WnU0IEU8eGaJaedGE9Dx+soRM+N9swY/5Mq4bdQ28lzX3Rzm5UueAEIrRLtgp2wQV lqex/PTy2/qj9Q6M6LKZwzMxhM/RRwwlJORJwDS45NSQDvIiH+UIDgr51OX8TBhBAz iIFUlHQQmsLLvkSWGKJPEp1wFg+g/Q50Z5rmuEmc= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:08 -0600 Message-Id: <20211221043610.2512334-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/8] android: Plumb all AE-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the AE-related controls reorganized in libcamera, with separate value and lever controls for exposure and gain, plumb these through the HAL layer (in order of appearence in the patch): - static metadata: available AE modes, AE lock available - manual template: add AE off (the other templates already have AE on) - preview template: set exposure time to the minimum in the exposure time range - request metadata: HAL -> libcamera controls conversion - result metadata: libcamera -> HAL controls conversion - result metadata: AE state We add class variables to CameraDevice to save the last set android AE controls, as that is how android controls function (no need to resubmit controls if they are unchanged). We also save libcamera's AE state in the request descriptor, as otherwise there is insufficient information from libcamera's result metadata alone to tell if the state is locked or manual (as they are an internal state in libcamera). Bug: https://bugs.libcamera.org/show_bug.cgi?id=42 Bug: https://bugs.libcamera.org/show_bug.cgi?id=43 Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- Changes in v3: - rename AeMode to AutoMode as we'll also use it for awb later - set the exposure time in the preview template - a bunch of other big changes (mentioned by Jacopo, and discussed a bit) were not done, as more discussion is needed New in v2 --- src/android/camera_capabilities.cpp | 71 +++++++++++++++++++--- src/android/camera_device.cpp | 93 +++++++++++++++++++++++++++-- src/android/camera_device.h | 13 ++++ src/android/camera_request.h | 9 +++ 4 files changed, 174 insertions(+), 12 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index a2e42a5c..318ab666 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -922,12 +922,62 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_ANTIBANDING_MODES, aeAvailableAntiBandingModes); - std::vector aeAvailableModes = { - ANDROID_CONTROL_AE_MODE_ON, - }; + std::vector aeAvailableModes; + /* This will cause problems if one supports only on and the other only off */ + bool aeAddedOn = false; + bool aeAddedOff = false; + + const auto &analogGainModeInfo = controlsInfo.find(&controls::AnalogueGainMode); + if (analogGainModeInfo != controlsInfo.end()) { + for (const auto &value : analogGainModeInfo->second.values()) { + switch (value.get()) { + case controls::AnalogueGainModeAuto: + if (!aeAddedOn) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_ON); + aeAddedOn = true; + break; + case controls::AnalogueGainModeDisabled: + if (!aeAddedOff) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_OFF); + aeAddedOff = true; + break; + default: + break; + } + } + } + + const auto &exposureTimeModeInfo = controlsInfo.find(&controls::ExposureTimeMode); + if (exposureTimeModeInfo != controlsInfo.end()) { + for (const auto &value : exposureTimeModeInfo->second.values()) { + switch (value.get()) { + case controls::ExposureTimeModeAuto: + if (!aeAddedOn) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_ON); + aeAddedOn = true; + break; + case controls::ExposureTimeModeDisabled: + if (!aeAddedOff) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_OFF); + aeAddedOff = true; + break; + default: + break; + } + } + } + + if (aeAvailableModes.empty()) + aeAvailableModes.push_back(ANDROID_CONTROL_AE_MODE_ON); staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES, aeAvailableModes); + /* In libcamera, turning off AE is equivalient to locking. */ + uint8_t aeLockAvailable = aeAddedOff ? ANDROID_CONTROL_AE_LOCK_AVAILABLE_TRUE + : ANDROID_CONTROL_AE_LOCK_AVAILABLE_FALSE; + staticMetadata_->addEntry(ANDROID_CONTROL_AE_LOCK_AVAILABLE, + aeLockAvailable); + std::vector aeCompensationRange = { 0, 0, }; @@ -988,10 +1038,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_SCENE_MODE_OVERRIDES, sceneModesOverride); - uint8_t aeLockAvailable = ANDROID_CONTROL_AE_LOCK_AVAILABLE_FALSE; - staticMetadata_->addEntry(ANDROID_CONTROL_AE_LOCK_AVAILABLE, - aeLockAvailable); - uint8_t awbLockAvailable = ANDROID_CONTROL_AWB_LOCK_AVAILABLE_FALSE; staticMetadata_->addEntry(ANDROID_CONTROL_AWB_LOCK_AVAILABLE, awbLockAvailable); @@ -1468,6 +1514,12 @@ std::unique_ptr CameraCapabilities::requestTemplateManual() cons manualTemplate->appendEntry(ANDROID_CONTROL_MODE, mode); } + if (staticMetadata_->entryContains(ANDROID_CONTROL_AE_AVAILABLE_MODES, + ANDROID_CONTROL_AE_MODE_OFF)) { + uint8_t aeMode = ANDROID_CONTROL_AE_MODE_OFF; + manualTemplate->appendEntry(ANDROID_CONTROL_AE_MODE, aeMode); + } + return manualTemplate; } @@ -1542,6 +1594,11 @@ std::unique_ptr CameraCapabilities::requestTemplatePreview() con uint8_t awbLock = ANDROID_CONTROL_AWB_LOCK_OFF; requestTemplate->addEntry(ANDROID_CONTROL_AWB_LOCK, awbLock); + if (availableRequestKeys_.count(ANDROID_SENSOR_EXPOSURE_TIME)) { + staticMetadata_->getEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE, &entry); + requestTemplate->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, entry.data.i64[0]); + } + uint8_t flashMode = ANDROID_FLASH_MODE_OFF; requestTemplate->addEntry(ANDROID_FLASH_MODE, flashMode); diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 83825736..3ade44c4 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -246,7 +246,9 @@ bool validateCropRotate(const camera3_stream_configuration_t &streamList) CameraDevice::CameraDevice(unsigned int id, std::shared_ptr camera) : id_(id), state_(State::Stopped), camera_(std::move(camera)), - facing_(CAMERA_FACING_FRONT), orientation_(0) + facing_(CAMERA_FACING_FRONT), orientation_(0), + aeOn_(true), aeLocked_(false), lastExposureTime_(0), + lastAnalogueGain_(1.0f), lastDigitalGain_(1.0f) { camera_->requestCompleted.connect(this, &CameraDevice::requestComplete); @@ -812,6 +814,59 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::draft::TestPatternMode, testPatternMode); } + /* + * \todo Can we trust that we won't receive values that we didn't + * report supporting? + */ + if (settings.getEntry(ANDROID_CONTROL_AE_LOCK, &entry)) + aeLocked_ = *entry.data.u8 != ANDROID_CONTROL_AE_LOCK_OFF; + + if (settings.getEntry(ANDROID_CONTROL_AE_MODE, &entry)) + aeOn_ = *entry.data.u8 != ANDROID_CONTROL_AE_MODE_OFF; + + Camera3RequestDescriptor::AutoMode aeMode; + if (aeLocked_ && aeOn_) + aeMode = Camera3RequestDescriptor::AutoMode::Lock; + else if (aeLocked_ && !aeOn_) + aeMode = Camera3RequestDescriptor::AutoMode::Manual; + else if (!aeLocked_ && aeOn_) + aeMode = Camera3RequestDescriptor::AutoMode::Auto; + else /* !aeLocked_ && !aeOn_ */ + aeMode = Camera3RequestDescriptor::AutoMode::Manual; + + /* Save this so that we can recover it in the result */ + descriptor->aeMode_ = aeMode; + + const auto &eInfo = camera_->controls().find(&controls::ExposureTimeMode); + if (eInfo != camera_->controls().end()) { + controls.set(controls::ExposureTimeMode, + aeMode == Camera3RequestDescriptor::AutoMode::Auto ? + controls::ExposureTimeModeAuto : + controls::ExposureTimeModeDisabled); + } + + const auto &gInfo = camera_->controls().find(&controls::AnalogueGainMode); + if (gInfo != camera_->controls().end()) { + controls.set(controls::AnalogueGainMode, + aeMode == Camera3RequestDescriptor::AutoMode::Auto ? + controls::AnalogueGainModeAuto : + controls::AnalogueGainModeDisabled); + } + + if (settings.getEntry(ANDROID_SENSOR_EXPOSURE_TIME, &entry)) { + const auto &info = camera_->controls().find(&controls::ExposureTime); + if (info != camera_->controls().end()) { + lastExposureTime_ = (*entry.data.i64) / 1000; + } + } + + /* Trigger libcamera's locked -> manual state change */ + if (aeMode == Camera3RequestDescriptor::AutoMode::Manual) { + const auto &info = camera_->controls().find(&controls::ExposureTime); + if (info != camera_->controls().end()) + controls.set(controls::ExposureTime, lastExposureTime_); + } + return 0; } @@ -1336,11 +1391,16 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_CONTROL_AE_EXPOSURE_COMPENSATION, value32); - value = ANDROID_CONTROL_AE_LOCK_OFF; - resultMetadata->addEntry(ANDROID_CONTROL_AE_LOCK, value); + uint8_t aeLock = (descriptor.aeMode_ == Camera3RequestDescriptor::AutoMode::Lock) + ? ANDROID_CONTROL_AE_LOCK_ON + : ANDROID_CONTROL_AE_LOCK_OFF; + resultMetadata->addEntry(ANDROID_CONTROL_AE_LOCK, aeLock); - value = ANDROID_CONTROL_AE_MODE_ON; - resultMetadata->addEntry(ANDROID_CONTROL_AE_MODE, value); + /* Locked means auto + locked in android */ + uint8_t aeMode = (descriptor.aeMode_ != Camera3RequestDescriptor::AutoMode::Manual) + ? ANDROID_CONTROL_AE_MODE_ON + : ANDROID_CONTROL_AE_MODE_OFF; + resultMetadata->addEntry(ANDROID_CONTROL_AE_MODE, aeMode); if (settings.getEntry(ANDROID_CONTROL_AE_TARGET_FPS_RANGE, &entry)) /* @@ -1357,6 +1417,29 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_CONTROL_AE_PRECAPTURE_TRIGGER, value); value = ANDROID_CONTROL_AE_STATE_CONVERGED; + if (metadata.contains(controls::AeState)) { + switch (metadata.get(controls::AeState)) { + case controls::AeStateInactive: + value = ANDROID_CONTROL_AE_STATE_INACTIVE; + break; + case controls::AeStateSearching: + value = ANDROID_CONTROL_AE_STATE_SEARCHING; + break; + case controls::AeStateConverged: + value = ANDROID_CONTROL_AE_STATE_CONVERGED; + break; + case controls::AeStateFlashRequired: + value = ANDROID_CONTROL_AE_STATE_FLASH_REQUIRED; + break; + case controls::AeStatePrecapture: + value = ANDROID_CONTROL_AE_STATE_PRECAPTURE; + break; + default: + LOG(HAL, Error) << "Invalid AeState, setting converged"; + } + } + if (descriptor.aeMode_ == Camera3RequestDescriptor::AutoMode::Lock) + value = ANDROID_CONTROL_AE_STATE_LOCKED; resultMetadata->addEntry(ANDROID_CONTROL_AE_STATE, value); value = ANDROID_CONTROL_AF_MODE_OFF; diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 64050416..a65f1670 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -126,5 +126,18 @@ private: int facing_; int orientation_; + /* + * Normally resubmission of controls would be handled on libcamera's + * side, but these controls are not one-to-one between libcamera and + * android, so we need to save them here + */ + + /* Track the last-set android AE controls */ + bool aeOn_; + bool aeLocked_; + int32_t lastExposureTime_; + float lastAnalogueGain_; + float lastDigitalGain_; + CameraMetadata lastSettings_; }; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 37b6ae32..b2809179 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -28,6 +28,12 @@ class CameraStream; class Camera3RequestDescriptor { public: + enum class AutoMode { + Auto, + Lock, + Manual, + }; + enum class Status { Success, Error, @@ -78,6 +84,9 @@ public: bool complete_ = false; Status status_ = Status::Success; + /* The libcamera internal AE state for this request */ + AutoMode aeMode_ = AutoMode::Auto; + private: LIBCAMERA_DISABLE_COPY(Camera3RequestDescriptor) }; From patchwork Tue Dec 21 04:36:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15185 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5087DBE080 for ; Tue, 21 Dec 2021 04:36:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EEB8060903; Tue, 21 Dec 2021 05:36:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aUEv6oX/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 147A2608E8 for ; Tue, 21 Dec 2021 05:36:28 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6EBFC881; Tue, 21 Dec 2021 05:36:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061387; bh=NCGU+UFk0uU5BG8rtRhiVTrbDKwMgSL6EDVWg26Nt2M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aUEv6oX/lA4H36mD17bjW1qV3LVlvxBA6fzQuwg5FGtYYJU4ssQo9ULP53tdCdOqT mtRbMDExt4WQEuZwBlOPnZrhGop1pA6T6uhW4sT/omqCA5fuqvv5thPqAW/z6G0a+u XjCiGCrNZhMVk9ato9SaV8RFY4+JZfzCCHVnzCww= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:09 -0600 Message-Id: <20211221043610.2512334-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 7/8] android: Check exposure time range for manual sensor capability X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the manual sensor capability validator, add a check for the presence of the exposure time range key, and for the maximum exposure time. The minimum exposure time is a requirement for adding the key in the first place; add a check for this as well. If either requirement is not met, the manual sensor capability validation will fail, therefore disabling the FULL hardware level. The exposure time range key is optional in non-FULL hardware levels. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- Changes in v4: - s/i32/i64 Changes in v3: - squash "android: capabilities: Add exposure time keys only if available" - fix the minumum exposure time check - only make the exposure time range key available if this check passes. additionally, if the max exposure time passes its check, tick the box for manual sensor and FULL - update commit message accordingly Changes in v2: - fix comparator order (cosmetic) - change comparators and comments to "equal or", as that is what is specificied in the hal docs - add check for minimum exposure time when initializing static metadata --- src/android/camera_capabilities.cpp | 33 +++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 318ab666..1bea15ca 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -217,6 +217,8 @@ std::vector setMetadata(CameraMetadata *metadata, uint32_t tag, bool CameraCapabilities::validateManualSensorCapability() { + camera_metadata_ro_entry_t entry; + const char *noMode = "Manual sensor capability unavailable: "; if (!staticMetadata_->entryContains(ANDROID_CONTROL_AE_AVAILABLE_MODES, @@ -231,6 +233,19 @@ bool CameraCapabilities::validateManualSensorCapability() return false; } + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE)) { + LOG(HAL, Info) << noMode << "missing exposure time range"; + return false; + } + + staticMetadata_->getEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE, &entry); + if (entry.data.i64[1] <= 100000000) { + LOG(HAL, Info) + << noMode + << "exposure time range maximum must be larger than 100ms"; + return false; + } + /* * \todo Return true here after we satisfy all the requirements: * https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#REQUEST_AVAILABLE_CAPABILITIES_MANUAL_SENSOR @@ -808,7 +823,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_SENSOR_AVAILABLE_TEST_PATTERN_MODES, ANDROID_SENSOR_INFO_ACTIVE_ARRAY_SIZE, ANDROID_SENSOR_INFO_COLOR_FILTER_ARRANGEMENT, - ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE, ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, ANDROID_SENSOR_INFO_PHYSICAL_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, @@ -886,7 +900,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_NOISE_REDUCTION_MODE, ANDROID_REQUEST_PIPELINE_DEPTH, ANDROID_SCALER_CROP_REGION, - ANDROID_SENSOR_EXPOSURE_TIME, ANDROID_SENSOR_FRAME_DURATION, ANDROID_SENSOR_ROLLING_SHUTTER_SKEW, ANDROID_SENSOR_TEST_PATTERN_MODE, @@ -1148,8 +1161,20 @@ int CameraCapabilities::initializeStaticMetadata() exposureInfo->second.min().get() * 1000LL, exposureInfo->second.max().get() * 1000LL, }; - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE, - exposureTimeRange, 2); + + if (exposureTimeRange[0] < 100000) { + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE, + exposureTimeRange, 2); + + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE); + availableRequestKeys_.insert(ANDROID_SENSOR_EXPOSURE_TIME); + availableResultKeys_.insert(ANDROID_SENSOR_EXPOSURE_TIME); + } else { + LOG(HAL, Info) + << "Minimum exposure time " + << exposureTimeRange[0] + << "ns is too big (should be smaller than 100us)"; + } } staticMetadata_->addEntry(ANDROID_SENSOR_ORIENTATION, orientation_); From patchwork Tue Dec 21 04:36:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15186 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C9DBC3258 for ; Tue, 21 Dec 2021 04:36:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B6472608E9; Tue, 21 Dec 2021 05:36:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XxZ9pcks"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 446486090B for ; Tue, 21 Dec 2021 05:36:29 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 47B35881; Tue, 21 Dec 2021 05:36:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061389; bh=RzfdXJCi0ta34mXKRkskLbybgflNzHpIYdSBBBO21m4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XxZ9pcksw451/He3ed+e3IESNyDjlJCGzCJNbrim/JBknTF1QkEv6zRlVoTPDBxGE GFPI5T1bCYhdgoHIEdOQhCrkBis0i5yV6xf6cab9xfQEv4YO7pPFuh7mSWN1r5iDdu 4XM0T6VdmP8l/i3Y3avftgoACIldzvfpRpz5RH20= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:10 -0600 Message-Id: <20211221043610.2512334-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/8] android: Plumb all sensitivity-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Plumb through the HAL the following three controls: - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) - ANDROID_SENSOR_SENSITIVITY (request, result) Also add a minISO to the HAL config, and add appropriate capability detection. The sensitivity range comes from: - min: hardcode to 100, or take minISO from the HAL config - max: set to minISO * max(AnalogueGain) * max(DigitalGain) The max analog sensitivity comes from: - hardcode to minISO * max(AnalogueGain) The request sensitivity is mapped to requested ISO / minISO, and split between analog and digital gain. The result sensitivity is mapped from minISO * analog gain * digital gain. Also set the sensor sensitivity in the preview template to the minimum ISO. Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- Changes in v3: - fix setting metadata entries - temp holder variable is not optional - fix gain calculation (used to be integer division, fix to float division) - fix splitting the gain into analog and digital - set sensor sensitivity in preview template No changes in v2 - I'm moving this into the AE-related series, because this is AE-related --- src/android/camera_capabilities.cpp | 65 +++++++++++++++++++++++++---- src/android/camera_capabilities.h | 3 +- src/android/camera_device.cpp | 64 +++++++++++++++++++++++++++- src/android/camera_device.h | 1 + src/android/camera_hal_config.cpp | 10 ++++- src/android/camera_hal_config.h | 1 + 6 files changed, 134 insertions(+), 10 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 1bea15ca..b9a1f6e5 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -233,6 +233,15 @@ bool CameraCapabilities::validateManualSensorCapability() return false; } + /* + * Checking the sensitivity range is sufficient, as it also covers the + * max analog sensitivity and the sensor sensitivity request/result key + */ + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE)) { + LOG(HAL, Info) << noMode << "missing sensitivity range"; + return false; + } + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE)) { LOG(HAL, Info) << noMode << "missing exposure time range"; return false; @@ -402,11 +411,12 @@ void CameraCapabilities::computeHwLevel( } int CameraCapabilities::initialize(std::shared_ptr camera, - int orientation, int facing) + int orientation, int facing, int minISO) { camera_ = camera; orientation_ = orientation; facing_ = facing; + minISO_ = minISO; rawStreamAvailable_ = false; maxFrameDuration_ = 0; @@ -826,7 +836,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, ANDROID_SENSOR_INFO_PHYSICAL_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, ANDROID_SENSOR_ORIENTATION, ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, @@ -1142,11 +1151,50 @@ int CameraCapabilities::initializeStaticMetadata() data); } - int32_t sensitivityRange[] = { - 32, 2400, - }; - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, - sensitivityRange); + /* + * The following three controls are only necessary for FULL + * (specifically the manual sensor capability): + * - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) + * - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) + * - ANDROID_SENSOR_SENSITIVITY (request, result) + * + * The first and second depend on analog gain, and the third depend on + * both, so if analog gain is not available then we can cross out all + * of them. We'll default to 100 minimum sensitivity if it isn't + * specified in the HAL configuration file. + * + * Digital gain is optional; if it's unavailable then the max analog + * sensitivity will be equal to the max sensitivity range. + * + * The minimum sensitivity must be at most 100, while the maximum must + * be at least 800. + */ + const auto &analogGainInfo = controlsInfo.find(&controls::AnalogueGain); + if (analogGainInfo != controlsInfo.end()) { + float maxAnalogGain = analogGainInfo->second.max().get(); + + const auto &digitalGainInfo = controlsInfo.find(&controls::DigitalGain); + float maxDigitalGain = digitalGainInfo == controlsInfo.end() ? + 1.0f : + digitalGainInfo->second.max().get(); + + int32_t maxISO = minISO_ * maxAnalogGain * maxDigitalGain; + int32_t sensitivityRange[] = { + minISO_, + maxISO < 800 ? 800 : maxISO, + }; + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, + sensitivityRange); + + int32_t maxAnalogISO = minISO_ * maxAnalogGain; + staticMetadata_->addEntry(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY, + maxAnalogISO); + + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE); + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY); + availableRequestKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + availableResultKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + } /* Report the color filter arrangement if the camera reports it. */ if (properties.contains(properties::draft::ColorFilterArrangement)) { @@ -1624,6 +1672,9 @@ std::unique_ptr CameraCapabilities::requestTemplatePreview() con requestTemplate->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, entry.data.i64[0]); } + if (availableRequestKeys_.count(ANDROID_SENSOR_SENSITIVITY)) + requestTemplate->addEntry(ANDROID_SENSOR_SENSITIVITY, minISO_); + uint8_t flashMode = ANDROID_FLASH_MODE_OFF; requestTemplate->addEntry(ANDROID_FLASH_MODE, flashMode); diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index 6f66f221..7f554c63 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -26,7 +26,7 @@ public: CameraCapabilities() = default; int initialize(std::shared_ptr camera, - int orientation, int facing); + int orientation, int facing, int minISO); CameraMetadata *staticMetadata() const { return staticMetadata_.get(); } libcamera::PixelFormat toPixelFormat(int format) const; @@ -70,6 +70,7 @@ private: int facing_; int orientation_; + int minISO_; bool rawStreamAvailable_; int64_t maxFrameDuration_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 3ade44c4..1a508923 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -376,7 +376,15 @@ int CameraDevice::initialize(const CameraConfigData *cameraConfigData) orientation_ = 0; } - return capabilities_.initialize(camera_, orientation_, facing_); + if (cameraConfigData && cameraConfigData->minISO != -1) { + minISO_ = cameraConfigData->minISO; + } else { + LOG(HAL, Error) + << "Minimum ISO not in configuration file. Using 100."; + minISO_ = 100; + } + + return capabilities_.initialize(camera_, orientation_, facing_, minISO_); } /* @@ -860,6 +868,40 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) } } + if (settings.getEntry(ANDROID_SENSOR_SENSITIVITY, &entry)) { + const auto &info = camera_->controls().find(&controls::AnalogueGain); + if (info != camera_->controls().end()) { + const auto &dInfo = + camera_->controls().find(&controls::DigitalGain); + bool digitalGainAvailable = + dInfo != camera_->controls().end(); + + float maxAnalogGain = info->second.max().get(); + float maxDigitalGain = digitalGainAvailable ? + dInfo->second.max().get() : 1.0f; + + /* target ISO / min ISO = gain */ + float gain = static_cast(*entry.data.i32) / + static_cast(minISO_); + + /* + * Max out analog gain before applying digital gain, if + * digital gain is available. + */ + if (gain <= maxAnalogGain) { + lastAnalogueGain_ = gain; + } else if (gain <= maxAnalogGain * maxDigitalGain) { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) + lastDigitalGain_ = gain / maxAnalogGain; + } else { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) + lastDigitalGain_ = maxDigitalGain; + } + } + } + /* Trigger libcamera's locked -> manual state change */ if (aeMode == Camera3RequestDescriptor::AutoMode::Manual) { const auto &info = camera_->controls().find(&controls::ExposureTime); @@ -867,6 +909,17 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ExposureTime, lastExposureTime_); } + /* Trigger libcamera's locked -> manual state change */ + if (aeMode == Camera3RequestDescriptor::AutoMode::Manual) { + const auto &aInfo = camera_->controls().find(&controls::AnalogueGain); + if (aInfo != camera_->controls().end()) + controls.set(controls::AnalogueGain, lastAnalogueGain_); + + const auto &dInfo = camera_->controls().find(&controls::DigitalGain); + if (dInfo != camera_->controls().end()) + controls.set(controls::DigitalGain, lastDigitalGain_); + } + return 0; } @@ -1534,6 +1587,15 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, exposure); } + if (metadata.contains(controls::AnalogueGain)) { + bool hasDigitalGain = metadata.contains(controls::DigitalGain); + float analogGain = metadata.get(controls::AnalogueGain); + float digitalGain = hasDigitalGain ? metadata.get(controls::DigitalGain) : 1.0f; + + int32_t iso = minISO_ * analogGain * digitalGain; + resultMetadata->addEntry(ANDROID_SENSOR_SENSITIVITY, iso); + } + if (metadata.contains(controls::FrameDuration)) { int64_t duration = metadata.get(controls::FrameDuration) * 1000; resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, diff --git a/src/android/camera_device.h b/src/android/camera_device.h index a65f1670..01c269d3 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -125,6 +125,7 @@ private: int facing_; int orientation_; + int minISO_; /* * Normally resubmission of controls would be handled on libcamera's diff --git a/src/android/camera_hal_config.cpp b/src/android/camera_hal_config.cpp index aa90dac7..863cdff0 100644 --- a/src/android/camera_hal_config.cpp +++ b/src/android/camera_hal_config.cpp @@ -180,6 +180,13 @@ int CameraHalConfig::Private::parseCameraConfigData(const std::string &cameraId) return -EINVAL; } cameraConfigData.rotation = ret; + } else if (key == "minISO") { + ret = std::stoi(value); + if (ret < 0 || ret > 100) { + LOG(HALConfig, Error) + << "Invalid minimum ISO: " << value; + return -EINVAL; + } } else { LOG(HALConfig, Error) << "Unknown key: " << key; @@ -384,7 +391,8 @@ int CameraHalConfig::parseConfigurationFile() const CameraConfigData &camera = c.second; LOG(HALConfig, Debug) << "'" << cameraId << "' " << "(" << camera.facing << ")[" - << camera.rotation << "]"; + << camera.rotation << "], " + << "minISO: " << camera.minISO; } return 0; diff --git a/src/android/camera_hal_config.h b/src/android/camera_hal_config.h index 9df554f9..d9ea45a4 100644 --- a/src/android/camera_hal_config.h +++ b/src/android/camera_hal_config.h @@ -15,6 +15,7 @@ struct CameraConfigData { int facing = -1; int rotation = -1; + int minISO = -1; }; class CameraHalConfig final : public libcamera::Extensible