From patchwork Tue Oct 28 09:31:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24839 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCB92BE080 for ; Tue, 28 Oct 2025 09:32:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 96A92607AC; Tue, 28 Oct 2025 10:32:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nBWJxX4T"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 940DD6078C for ; Tue, 28 Oct 2025 10:32:07 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 023BB1AED; Tue, 28 Oct 2025 10:30:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643819; bh=f1o89u6CDxBXPdsmF5cTswP56ZrdhO4N8GFN23yudhs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nBWJxX4Tc/xWe8mbaaphnE+P1I3KULZMgYla7m+K3sMl0UL1YYslcTJ9l9lxJFY0h AFRFLeJ7qBYx8Ftzd6DTFM5+rzu6jh//nEG5huuQADGElfO0x/ibbLIwZsIE+NR9cj 3xDce1UqQ/X67sJAq+mhwiEU+WH+BTY264JqTTi4= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:47 +0100 Subject: [PATCH v2 01/10] ipa: camera_sensor_helper: Introduce exposureMargin() MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-1-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=10952; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=f1o89u6CDxBXPdsmF5cTswP56ZrdhO4N8GFN23yudhs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2UmaJ3xkBUY+rbq9+3+XPWhBT9vYPyEaH36 ffqn+GoceqJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlAAKCRByNAaPFqFW PJzREADD0byAlRv4NwvJ7Yxx1UKyaFilxaAdffovMWqO7lM0MUMuJNyokDMN1zxKrVGBu9V9HCt +sSn0UWQNqXf2RdXo1TcHESjWShMgYh2tnojafpkb6AFdKqxh9zyxn8ovshi9eVzakGw/QNvxji gr0rymhEZUH8ZNQQ+jZbVhQQh8VW+dRG2UgSF7OdpZZYtn9QyE5mnBJe2Fu/44obDDAr1d0mAel CXyOoAPYSHNJW7hVk6NEeI0aK/Orte6t2I1SrorYCkfPCiqhuY1xRzoGrxArW+MI46i5VO2PLpC Uulgksuw6cwitUVBD9hPkg6MtYQfq73nwHjTZf0prZgstwgzmuYHf4wcQFsQRWUmvIT6MCRYhdH 6W0ZGrsprcgvOxK3RgWNmmd82m8P8Z47YGipL6rMkShC2LwBCWu6Wcp/5TsikwdTlwK4ImxJ7Jj UtLYr+iQy6Mxno/U8HMcxA1h2PH6dTbuDJZYN5owB9CH+BnNmoGyV/P8VSPSuJreMbXBqVl0bef C3odpikrGHHhRGSbZQKV1aPcXxbAQH37VktNeppgyNsMCHdGKvXf1L12yuG9iewuWe6XwYvK6j7 PNyKkSZEc1PtDQGPPTYBgMKfK76cbw9jeU3loiMcYbbD5khyWsFAllMHqnEMHE/UfCU0Jynt00S hqVDgA41rJcb86w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There is a margin between the maximum achievable exposure time and the frame length. The value is sensor-dependent and should be used to adjust the maximum achievable exposure time whenever the frame length changes. Introduce in the CameraSensorHelper an exposureMargin() function that returns a std::optional<> with the sensor-specific margin. Use a std::optional<> to easily identify CameraSensorHelper implementations that do not populate exposureMargin_ and warn about possibly sub-optimal results in exposure time calculations. This feature is already implemented in the RPi CamHelper class hierarchy with the name of frameIntegrationDiff. Populate the CameraSensorHelper instances with an exposureMargin_. The value of the exposure margin comes from the mainline driver version of each sensor, and it has been compared with the frameIntegrationDiff_ value in the rpi camera helpers. The only outliner is the Vd56g3 sensor, whose exposure margin from the driver differs from the RPi camera helper. Signed-off-by: Jacopo Mondi --- src/ipa/libipa/camera_sensor_helper.cpp | 72 +++++++++++++++++++++++++++++++++ src/ipa/libipa/camera_sensor_helper.h | 2 + 2 files changed, 74 insertions(+) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index ef3bd0d621e6bead2b3f8a7fe6988f642e4791f7..6d14405657903704d6ffee72eb336ec8b78db1cd 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -154,6 +154,18 @@ double CameraSensorHelper::quantizeGain(double _gain, double *quantizationGain) return g; } +/** + * \fn CameraSensorHelper::exposureMargin() + * \brief Fetch the integration time margin + * + * This function returns the number of lines that represent the minimum + * difference between the frame length and the maximum achievable integration + * time. If it is unknown an empty optional is returned. + * + * \return The minimum difference between the exposure time and the frame + * length in lines, or an empty optional + */ + /** * \struct CameraSensorHelper::AnalogueGainLinear * \brief Analogue gain constants for the linear gain model @@ -229,6 +241,13 @@ double CameraSensorHelper::quantizeGain(double _gain, double *quantizationGain) * sensor specific. Use this variable to store the values at init time. */ +/** + * \var CameraSensorHelper::exposureMargin_ + * \brief The smallest margin between the integration time and the frame lenght + * in lines + * \sa CameraSensorHelper::exposureMargin() + */ + /** * \class CameraSensorHelperFactoryBase * \brief Base class for camera sensor helper factories @@ -385,6 +404,7 @@ public: { /* Power-on default value: 168 at 12bits. */ blackLevel_ = 2688; + exposureMargin_ = 4; } uint32_t gainCode(double gain) const override @@ -474,6 +494,11 @@ REGISTER_CAMERA_SENSOR_HELPER("ar0144", CameraSensorHelperAr0144) class CameraSensorHelperAr0521 : public CameraSensorHelper { public: + CameraSensorHelperAr0521() + { + exposureMargin_ = 4; + } + uint32_t gainCode(double gain) const override { gain = std::clamp(gain, 1.0, 15.5); @@ -504,6 +529,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 100, 0, 0, 1024 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("gc05a2", CameraSensorHelperGc05a2) @@ -516,6 +542,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 100, 0, 0, 1024 }; + exposureMargin_ = 16; } }; REGISTER_CAMERA_SENSOR_HELPER("gc08a3", CameraSensorHelperGc08a3) @@ -526,6 +553,7 @@ public: CameraSensorHelperHm1246() { gain_ = AnalogueGainLinear{ 1, 16, 0, 16 }; + exposureMargin_ = 2; } }; REGISTER_CAMERA_SENSOR_HELPER("hm1246", CameraSensorHelperHm1246) @@ -538,6 +566,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 512, -1, 512 }; + exposureMargin_ = 10; } }; REGISTER_CAMERA_SENSOR_HELPER("imx214", CameraSensorHelperImx214) @@ -550,6 +579,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 256, -1, 256 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("imx219", CameraSensorHelperImx219) @@ -562,6 +592,7 @@ public: /* From datasheet: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 512, -1, 512 }; + exposureMargin_ = 10; } }; REGISTER_CAMERA_SENSOR_HELPER("imx258", CameraSensorHelperImx258) @@ -574,6 +605,7 @@ public: /* From datasheet: 0x32 at 10bits. */ blackLevel_ = 3200; gain_ = AnalogueGainLinear{ 0, 2048, -1, 2048 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("imx283", CameraSensorHelperImx283) @@ -586,6 +618,7 @@ public: /* From datasheet: 0xf0 at 12bits. */ blackLevel_ = 3840; gain_ = AnalogueGainExp{ 1.0, expGainDb(0.3) }; + exposureMargin_ = 2; } }; REGISTER_CAMERA_SENSOR_HELPER("imx290", CameraSensorHelperImx290) @@ -596,6 +629,11 @@ public: CameraSensorHelperImx296() { gain_ = AnalogueGainExp{ 1.0, expGainDb(0.1) }; + /* + * The driver doesn't apply any margin. Use the value + * in RPi's CamHelper. + */ + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("imx296", CameraSensorHelperImx296) @@ -613,6 +651,7 @@ public: /* From datasheet: 0x32 at 10bits. */ blackLevel_ = 3200; gain_ = AnalogueGainExp{ 1.0, expGainDb(0.3) }; + exposureMargin_ = 9; } }; REGISTER_CAMERA_SENSOR_HELPER("imx335", CameraSensorHelperImx335) @@ -623,6 +662,7 @@ public: CameraSensorHelperImx415() { gain_ = AnalogueGainExp{ 1.0, expGainDb(0.3) }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("imx415", CameraSensorHelperImx415) @@ -638,6 +678,7 @@ public: CameraSensorHelperImx477() { gain_ = AnalogueGainLinear{ 0, 1024, -1, 1024 }; + exposureMargin_ = 22; } }; REGISTER_CAMERA_SENSOR_HELPER("imx477", CameraSensorHelperImx477) @@ -652,6 +693,7 @@ public: * This has been validated with some empirical testing only. */ gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov2685", CameraSensorHelperOv2685) @@ -662,6 +704,7 @@ public: CameraSensorHelperOv2740() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov2740", CameraSensorHelperOv2740) @@ -674,6 +717,7 @@ public: /* From datasheet: 0x40 at 12bits. */ blackLevel_ = 1024; gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov4689", CameraSensorHelperOv4689) @@ -686,6 +730,14 @@ public: /* From datasheet: 0x10 at 10bits. */ blackLevel_ = 1024; gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + /* + * Very convoluted in the driver that however applies a margin + * of 4 lines when setting vts. + * + * cap_vts = cap_shutter + 4; + * ret = ov5640_set_vts(sensor, cap_vts); + */ + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5640", CameraSensorHelperOv5640) @@ -696,6 +748,7 @@ public: CameraSensorHelperOv5647() { gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5647", CameraSensorHelperOv5647) @@ -706,6 +759,7 @@ public: CameraSensorHelperOv5670() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5670", CameraSensorHelperOv5670) @@ -718,6 +772,7 @@ public: /* From Linux kernel driver: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5675", CameraSensorHelperOv5675) @@ -728,6 +783,7 @@ public: CameraSensorHelperOv5693() { gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5693", CameraSensorHelperOv5693) @@ -738,6 +794,7 @@ public: CameraSensorHelperOv64a40() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 32; } }; REGISTER_CAMERA_SENSOR_HELPER("ov64a40", CameraSensorHelperOv64a40) @@ -754,6 +811,7 @@ public: * See: https://patchwork.linuxtv.org/project/linux-media/patch/20221106171129.166892-2-nicholas@rothemail.net/#142267 */ gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov8858", CameraSensorHelperOv8858) @@ -764,6 +822,7 @@ public: CameraSensorHelperOv8865() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov8865", CameraSensorHelperOv8865) @@ -774,6 +833,7 @@ public: CameraSensorHelperOv13858() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov13858", CameraSensorHelperOv13858) @@ -786,6 +846,7 @@ public: /* From datasheet: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 32, -1, 32 }; + exposureMargin_ = 64; } }; REGISTER_CAMERA_SENSOR_HELPER("vd55g1", CameraSensorHelperVd55g1) @@ -798,6 +859,17 @@ public: /* From datasheet: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 32, -1, 32 }; + /* + * The mainline driver version has + * #define VD56G3_EXPOSURE_MARGIN 75 + * while the frameIntegrationDiff value in the RPi cam + * helper for this sensor has + * static constexpr int frameIntegrationDiff = 61; + * + * Trust the driver and report 75 which is also larger and + * hence "safer" + */ + exposureMargin_ = 75; } }; REGISTER_CAMERA_SENSOR_HELPER("vd56g3", CameraSensorHelperVd56g3) diff --git a/src/ipa/libipa/camera_sensor_helper.h b/src/ipa/libipa/camera_sensor_helper.h index bd3d0beec77f006b68fecf45eee850c5283fefa5..57dd4c37ed76733710f1673a98e82ae47475baa7 100644 --- a/src/ipa/libipa/camera_sensor_helper.h +++ b/src/ipa/libipa/camera_sensor_helper.h @@ -30,6 +30,7 @@ public: virtual uint32_t gainCode(double gain) const; virtual double gain(uint32_t gainCode) const; double quantizeGain(double gain, double *quantizationGain) const; + std::optional exposureMargin() const { return exposureMargin_; } protected: struct AnalogueGainLinear { @@ -46,6 +47,7 @@ protected: std::optional blackLevel_; std::variant gain_; + std::optional exposureMargin_; private: LIBCAMERA_DISABLE_COPY_AND_MOVE(CameraSensorHelper) From patchwork Tue Oct 28 09:31:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24840 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BDA1BE080 for ; Tue, 28 Oct 2025 09:32:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A07F360794; Tue, 28 Oct 2025 10:32:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="On0EWrKk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3038760791 for ; Tue, 28 Oct 2025 10:32:08 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 812431D29; Tue, 28 Oct 2025 10:30:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643819; bh=LnOy57B0dhajeFihc9nwOXI4ugAWSbZamws1XO1+2V4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=On0EWrKkaVz1YXUTzFMgg4WmSM3s1LxMb+F8PhwJ36gyQAO6hncLpGd2higTY6cY4 8TR0Z6N30ntQuUluEPdduDKdqyWC5TJMvDOAHRMzK+cZ8zhycKNWj9LcVETxF1N0ps mXHr1O9xuwLZZMLnfiwZH5nefKasDpV/16ja6Od8= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:48 +0100 Subject: [PATCH v2 02/10] ipa: mali-c55: Move CameraHelper to context MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-2-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5693; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=PQkhZX8GHLnV60t4CeY04DEIjNCw6nFgT/razeFVaWE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2Vf/+iu1EP0c1d6uDJ4uY8Uh6VjGDQND4hy WtGLtzzddaJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlQAKCRByNAaPFqFW PAM7EAC1giyUMVw3buFTrIrCtaXgYrGKuF1PuJ5I2hxScOeYp44xHyYh+/i5virirL4RhxIisEc IYLr9JDhxPSWHTWyFqmsNEoVRot5AlRm6fA3nXY95z22L4w1U4YrOHrnD6ltxgZ1nwdo0RIgs+I 7IKMrXqT2LFlw6WWqFNRotL2h/ZVOWopqtVZaqOeKAUNPi7uwWc6Pgzka1RANVcnij8JGKMjMia mXoKS2gIuuchUOUQfJ/xKv+SN3xEffWD6DP0lZdL9PkEFI7M52zEwMkqZGlC1eCr/CcBXpSXXT+ ypbEf5HkvubcFDS40cLjt/C7h5/uA/4UhEVlOnlca2VQPeffJ1uvfdHisltxEn5Qjq3bAeW/1a2 QzNEHYczwNreXLnFakUNc/2WWAlMbyAUaJBsQbwXAdMCn5WvVLwFlmPJ1n4YR2RRonF54vfcPqq D6nNQ8sbPOhYO+BE9LQbOeDpdDs0PPfa/fl4EO56byAVcMLUMEDgC+SIoTBhm5AoU1Wwl+dDvmU N6UtivmevEQcCuKiBJ2N18rchMtho5M8/4uMrUPkFR/5ZOHAwCnlXClNJmGg3WeQMNawNmCqnqW nZCDJJzz1Vb594J0Zb6GLfvRVAShCmUiiJy6PdzVGK9zT5QSJDtDe8efWXfnZ/5ey2A8E4obm0G P0o1uaXGYaXhXJQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Move the CameraHelper sensor to the Context so that it can be used by the algorithms directly. While at it, document the undocumented ctrlMap member of IPAContext. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi --- src/ipa/mali-c55/ipa_context.cpp | 6 ++++++ src/ipa/mali-c55/ipa_context.h | 4 ++++ src/ipa/mali-c55/mali-c55.cpp | 27 ++++++++++++--------------- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/src/ipa/mali-c55/ipa_context.cpp b/src/ipa/mali-c55/ipa_context.cpp index 1b203e2b260592cda945855d54f1aceefaaece91..a1e4c39f38704d3a4b5948ab789ffb9773f216c9 100644 --- a/src/ipa/mali-c55/ipa_context.cpp +++ b/src/ipa/mali-c55/ipa_context.cpp @@ -96,6 +96,12 @@ namespace libcamera::ipa::mali_c55 { * * \var IPAContext::frameContexts * \brief Ring buffer of per-frame contexts + * + * \var IPAContext::ctrlMap + * \brief A ControlInfoMap::Map of controls populated by the algorithms + * + * \var IPAContext::camHelper + * \brief The camera sensor helper */ } /* namespace libcamera::ipa::mali_c55 */ diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index 13885eb83b5c4d7c7af6c4f8d5673890834fed58..bfa805c7b93f313dda2497365e83542bbc39e291 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -12,6 +12,7 @@ #include "libcamera/internal/bayer_format.h" +#include #include namespace libcamera { @@ -83,6 +84,9 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; }; } /* namespace ipa::mali_c55 */ diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index 7d45e7310aecdae0e47655e6d2e8830e776d74cd..0751513dc584ca84dd212bf8c1469dd4b40c053d 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -74,9 +74,6 @@ private: ControlInfoMap sensorControls_; - /* Interface to the Camera Helper */ - std::unique_ptr camHelper_; - /* Local parameter storage */ struct IPAContext context_; }; @@ -98,8 +95,8 @@ std::string IPAMaliC55::logPrefix() const int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, ControlInfoMap *ipaControls) { - camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); - if (!camHelper_) { + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { LOG(IPAMaliC55, Error) << "Failed to create camera sensor helper for " << settings.sensorModel; @@ -142,10 +139,10 @@ void IPAMaliC55::setControls() if (activeState.agc.autoEnabled) { exposure = activeState.agc.automatic.exposure; - gain = camHelper_->gainCode(activeState.agc.automatic.sensorGain); + gain = context_.camHelper->gainCode(activeState.agc.automatic.sensorGain); } else { exposure = activeState.agc.manual.exposure; - gain = camHelper_->gainCode(activeState.agc.manual.sensorGain); + gain = context_.camHelper->gainCode(activeState.agc.manual.sensorGain); } ControlList ctrls(sensorControls_); @@ -191,17 +188,17 @@ void IPAMaliC55::updateSessionConfiguration(const IPACameraSensorInfo &info, context_.configuration.agc.minShutterSpeed = minExposure * context_.configuration.sensor.lineDuration; context_.configuration.agc.maxShutterSpeed = maxExposure * context_.configuration.sensor.lineDuration; context_.configuration.agc.defaultExposure = defExposure; - context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); - context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); + context_.configuration.agc.minAnalogueGain = context_.camHelper->gain(minGain); + context_.configuration.agc.maxAnalogueGain = context_.camHelper->gain(maxGain); - if (camHelper_->blackLevel().has_value()) { + if (context_.camHelper->blackLevel().has_value()) { /* * The black level from CameraSensorHelper is a 16-bit value. * The Mali-C55 ISP expects 20-bit settings, so we shift it to * the appropriate width */ context_.configuration.sensor.blackLevel = - camHelper_->blackLevel().value() << 4; + context_.camHelper->blackLevel().value() << 4; } } @@ -252,9 +249,9 @@ void IPAMaliC55::updateControls(const IPACameraSensorInfo &sensorInfo, /* Compute the analogue gain limits. */ const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; - float minGain = camHelper_->gain(v4l2Gain.min().get()); - float maxGain = camHelper_->gain(v4l2Gain.max().get()); - float defGain = camHelper_->gain(v4l2Gain.def().get()); + float minGain = context_.camHelper->gain(v4l2Gain.min().get()); + float maxGain = context_.camHelper->gain(v4l2Gain.max().get()); + float defGain = context_.camHelper->gain(v4l2Gain.def().get()); ctrlMap[&controls::AnalogueGain] = ControlInfo(minGain, maxGain, defGain); /* @@ -362,7 +359,7 @@ void IPAMaliC55::processStats(unsigned int request, unsigned int bufferId, frameContext.agc.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.agc.sensorGain = - camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); ControlList metadata(controls::controls); From patchwork Tue Oct 28 09:31:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24841 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BF99BBE080 for ; Tue, 28 Oct 2025 09:32:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4CE21607A6; Tue, 28 Oct 2025 10:32:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Van7cVtN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9FC37607A2 for ; Tue, 28 Oct 2025 10:32:08 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 13AFE1F0E; Tue, 28 Oct 2025 10:30:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643820; bh=V/LKpOf9yhdXRyBPE0ppKYaI4nGg8p0p2ziSbJ5qtY8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Van7cVtNxSOfKwy9ar479LdhcE+/AvjTudJ/BvCIJqm2vE3z+40iW/E69gCFM4Da7 j5Xx2UuyyRnl5SoDO+gztovfNRQ7/Y0q2wLP177LPAKezU2zA2C89h8xzQv9wzkc8n wW0KJP0EadKGx+U+kdKYpi+SRFn+C4TzNEasHY58= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:49 +0100 Subject: [PATCH v2 03/10] ipa: ipu3: Move CameraHelper to context MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-3-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4027; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=V/LKpOf9yhdXRyBPE0ppKYaI4nGg8p0p2ziSbJ5qtY8=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2VhIPNLjHNbc6qJQ1o6nlXRY4ND7bRdHWn9 Qlplis7HeSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlQAKCRByNAaPFqFW PLjbD/wI4rcrIJpYZFr3WPiiMpNHQt78NrXbhewfcpOlMMzLDnJVYEf3nm4EG7PdFR8u4BkYWJs 8K3E8vu7YovxeIAZ/sUJ/rs/D2Rag4YOehFQ2iVu0TN+sQKWgact8r6kuYNFcbrEZaR/WC7Hp4U eHPmqsgw2dVeTuRowh8h7qp/stv3PTCbCMBAGmIWlDrJXbN0m9soRxB4z7H/n3kWww4S7Vh7cHn eEpy+Zlzr0vCMdC+wuR2BnBUVP7UxpJ+fU/eC3hFSGPk+aPrLYWQGAhTVn2ig98+3nYtXM4IG0r ucaW2XYA3VRTYIfdcvYlHEFmVZ1IhZmplkbgyxL+MPSzXfttoHdQHGD+bT4rNAAUdz8MRgHy7GO h189D/b1dN+DlxymyKp5UvtYvID0bFdOqUjjIszL+co8V1pM23otvJ4vvr5nLHpMgqSp3p/gzDA DWW+4o3DBTST6JRozLpuWfnOYVhYadSz+fkw7nKjO8hxLX3ckMKeYYfwwjqQCyw3EmboF3z+p0j gagI11402RnO+FPku9nO4NscWCq36qMbHjhbUqs+/G+jdvvG70XT6k5WglM24AfOY+EttgMZPZQ Eg0SzCVf6YTyqJxKOfS8FVj7MylZXVZGcZYZOE3cf7ahnz36QP9HnaNn+nZP3fbzcdo1Dcv75FP tqIhFKbAyb9SeAQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move the CameraHelper sensor to the Context so that it can be used by the algorithms directly. Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.cpp | 3 +++ src/ipa/ipu3/ipa_context.h | 3 +++ src/ipa/ipu3/ipu3.cpp | 15 ++++++--------- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 3b22f7917650d9e400d5368c2f890d6b2dc846a0..5d0ebaaec454fca823851228614a3eb229f0132b 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -54,6 +54,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAContext::ctrlMap * \brief A ControlInfoMap::Map of controls populated by the algorithms + * + * \var IPAContext::camHelper + * \brief The camera sensor helper */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 97fcf06cd4ac9ac6d64c4933fcea80ace0e572df..4544d663642fb6fde1cf69a32f1c0fbb2fbd98f6 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -15,6 +15,7 @@ #include #include +#include #include namespace libcamera { @@ -95,6 +96,8 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + std::unique_ptr camHelper; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1cae08bf255f6f53a0ebee02ba3aadda5a650ae3..36d8f0da430a3d4f94e2a3f760850519742ea992 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -179,9 +179,6 @@ private: IPACameraSensorInfo sensorInfo_; - /* Interface to the Camera Helper */ - std::unique_ptr camHelper_; - /* Local parameter storage */ struct IPAContext context_; }; @@ -222,8 +219,8 @@ void IPAIPU3::updateSessionConfiguration(const ControlInfoMap &sensorControls) */ context_.configuration.agc.minExposureTime = minExposure * context_.configuration.sensor.lineDuration; context_.configuration.agc.maxExposureTime = maxExposure * context_.configuration.sensor.lineDuration; - context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); - context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); + context_.configuration.agc.minAnalogueGain = context_.camHelper->gain(minGain); + context_.configuration.agc.maxAnalogueGain = context_.camHelper->gain(maxGain); } /** @@ -301,8 +298,8 @@ int IPAIPU3::init(const IPASettings &settings, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) { - camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); - if (camHelper_ == nullptr) { + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (context_.camHelper == nullptr) { LOG(IPAIPU3, Error) << "Failed to create camera sensor helper for " << settings.sensorModel; @@ -597,7 +594,7 @@ void IPAIPU3::processStats(const uint32_t frame, IPAFrameContext &frameContext = context_.frameContexts.get(frame); frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + frameContext.sensor.gain = context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); ControlList metadata(controls::controls); @@ -643,7 +640,7 @@ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) void IPAIPU3::setControls(unsigned int frame) { int32_t exposure = context_.activeState.agc.exposure; - int32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); + int32_t gain = context_.camHelper->gainCode(context_.activeState.agc.gain); ControlList ctrls(sensorCtrls_); ctrls.set(V4L2_CID_EXPOSURE, exposure); From patchwork Tue Oct 28 09:31:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24842 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6B31ABE080 for ; Tue, 28 Oct 2025 09:32:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 473B76079C; Tue, 28 Oct 2025 10:32:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G3WQFCST"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7326D607A8 for ; Tue, 28 Oct 2025 10:32:09 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8D6424B22; Tue, 28 Oct 2025 10:30:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643820; bh=o/vOnHG6w0sMiJ7WhNcZD8nDa3NU32S4u0gAPh+/RTc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=G3WQFCSThAO5NQwNYSEOT3Tvd5A/BdazT8Obq4q+r5QOxd7jNCPPZC+hS66Mc603A amS6pKSFgI2jV/5h15+YBdWfkbBbLI0Gs1YAyI32wprCSFhb0oTNGtKnGvh8iJ+wov a73Z3n5IKKoSksU0OQVhUvTwhRGiMz+rYSGOb3h8= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:50 +0100 Subject: [PATCH v2 04/10] ipa: libipa: agc: Make Agc::configure() set limits MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-4-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=11498; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=o/vOnHG6w0sMiJ7WhNcZD8nDa3NU32S4u0gAPh+/RTc=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2VaKV6zhP/RlZUglT+ZkgjLvuriV2xccTyL DT5LawF1Q2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlQAKCRByNAaPFqFW PB+oD/9r6elOz3j1DMbnz3TSKZ7ezVcyN7mvPsooaW431qgSyzicu18rCC6ldo0AKfvMa45eRNF FuWoDZlEV9kswZ5HxsywDFiWiGWuI4DwIZ0amKp3hW+gEo/jBkTMrPhKrmDcKqvh+Dv3pQ5KIES Ue1h/U+o5oXaXMXYCFlsSEW8yoPSJ8npP3DgXfKO6X2urjt+GYROy5aNQteYER0EXStAKJqjdCq Ni2PSVF1YwJYyfJQrNc6q7l3/ANdIO/ygpVAc/B1Gpa30A6pngvJ5JOPYbJWYDHGUpXEoY1Ag8S MAuEo15sAixeInvz8mghbqq2WEeugKii9it8VGOREFG8mhiyNpGhACBFCUmpbu+6/dtrhK0gIkx TVk43Lwy6QnKWigm2hl5hfVpD0g8Zujv5kt97eR955iG7eqdMitQZnVgk3bibNBY7ZvhUYvHIfX d9qEjMSX1Oqxvs9oG4p1jv/B+IWo4C2Lgq+Ba4EEkY+VIcWCmRr0mNSfghAVaRu9mjYu2eKP/pP ljm9N94eDqlubKiOzmc0iltHRMiT5TYrzDmfFk5QfX0EZ1AWtFhbWqVs7muqL/1Q0hG7oJezRo/ rGFxEgSmjE4QJ5cRPzw+GeDpttX8vc6xswuBVhT+oRLAD4T+8CoVibAkMLi+B1UDo5/LSdDYHSb DnUJegcPydFVC6A== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AgcMeanLuminance algorithm interface has a 'configure()' and 'setLimits()' API. configure() doesn't actually do much if not initialzing a few variables, and the Mali and IPU3 IPAs do not even call it. Configuring the AGC requires calling setLimits() at configure() time and at run time. In order to prepare to differentiate between configure-time settings of the Agc and run-time handling of dynamic parameters such as the frame duration limits, make the configure() operation initialize the Agc limits. Update all IPAs Agc implementation deriving from AgcMeanLuminance. Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/agc.cpp | 14 +++++++--- src/ipa/libipa/agc_mean_luminance.cpp | 48 ++++++++++++++++++++++++++++++--- src/ipa/libipa/agc_mean_luminance.h | 11 +++++++- src/ipa/libipa/exposure_mode_helper.cpp | 20 ++++++++++---- src/ipa/libipa/exposure_mode_helper.h | 4 ++- src/ipa/mali-c55/algorithms/agc.cpp | 16 ++++++----- src/ipa/rkisp1/algorithms/agc.cpp | 15 ++++++----- 7 files changed, 101 insertions(+), 27 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index b0d89541da8550bd50472779da7fa8e33b96e2f0..e303d724ab165bac03ac6bb9d3891a76049cff47 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -115,9 +115,17 @@ int Agc::configure(IPAContext &context, context.activeState.agc.constraintMode = constraintModes().begin()->first; context.activeState.agc.exposureMode = exposureModeHelpers().begin()->first; - /* \todo Run this again when FrameDurationLimits is passed in */ - setLimits(minExposureTime_, maxExposureTime_, minAnalogueGain_, - maxAnalogueGain_, {}); + AgcMeanLuminance::AgcSensorConfiguration sensorConfig; + sensorConfig.lineDuration = context.configuration.sensor.lineDuration; + sensorConfig.minExposureTime = minExposureTime_; + sensorConfig.maxExposureTime = maxExposureTime_; + sensorConfig.minAnalogueGain = minAnalogueGain_; + sensorConfig.maxAnalogueGain = maxAnalogueGain_; + + AgcMeanLuminance::configure(sensorConfig, context.camHelper.get()); + + /* \todo Update AGC limits when FrameDurationLimits is passed in */ + resetFrameCount(); return 0; diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 64f36bd75dd213671b5a2e6810245096ed001f21..9886864fc42514040eadb4c0b424796df1f1b6e1 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -95,6 +95,42 @@ static constexpr double kMaxRelativeLuminanceTarget = 0.95; * \brief The luminance target for the constraint */ +/** + * \struct AgcMeanLuminance::AgcSensorConfiguration + * \brief The sensor configuration parameters + * + * This structure collects the sensor configuration parameters which need + * to be provided to the AGC algorithm at configure() time. + * + * Each time configure() is called the sensor configuration need to be updated + * with new parameters. + */ + +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::lineDuration + * \brief The line duration in microseconds + */ + +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::minExposureTime + * \brief The sensor minimum exposure time in microseconds + */ + +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::maxExposureTime + * \brief The sensor maximum exposure time in microseconds + */ + +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::minAnalogueGain + * \brief The sensor minimum analogue gain absolute value + */ + +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::maxAnalogueGain + * \brief The sensor maximum analogue gain absolute value + */ + /** * \class AgcMeanLuminance * \brief A mean-based auto-exposure algorithm @@ -314,17 +350,21 @@ int AgcMeanLuminance::parseExposureModes(const YamlObject &tuningData) /** * \brief Configure the exposure mode helpers - * \param[in] lineDuration The sensor line length + * \param[in] config The sensor configuration * \param[in] sensorHelper The sensor helper * - * This function configures the exposure mode helpers so they can correctly + * This function configures the exposure mode helpers by providing them the + * sensor configuration parameters and the sensor helper, so they can correctly * take quantization effects into account. */ -void AgcMeanLuminance::configure(utils::Duration lineDuration, +void AgcMeanLuminance::configure(const AgcSensorConfiguration &config, const CameraSensorHelper *sensorHelper) { for (auto &[id, helper] : exposureModeHelpers_) - helper->configure(lineDuration, sensorHelper); + helper->configure(config.lineDuration, + config.minExposureTime, config.maxExposureTime, + config.minAnalogueGain, config.maxAnalogueGain, + sensorHelper); } /** diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index d7ec548e3e585e28fd03b956c8de7eaac5dc6e99..5ab75a3ac6c550a655f4bb7c8c30419ad8249183 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -42,7 +42,16 @@ public: double yTarget; }; - void configure(utils::Duration lineDuration, const CameraSensorHelper *sensorHelper); + struct AgcSensorConfiguration { + utils::Duration lineDuration; + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + }; + + void configure(const AgcSensorConfiguration &config, + const CameraSensorHelper *sensorHelper); int parseTuningData(const YamlObject &tuningData); void setExposureCompensation(double gain) diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index 29e316d9d09181cd65cb08f26babb1a680bee62a..557e14230ba4e0f120653bc09ea23a29a54cb4c8 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -82,22 +82,32 @@ ExposureModeHelper::ExposureModeHelper(const Span> stages); ~ExposureModeHelper() = default; - void configure(utils::Duration lineLength, const CameraSensorHelper *sensorHelper); + void configure(utils::Duration lineLength, utils::Duration minExposureTime, + utils::Duration maxExposureTime, double minGain, double maxGain, + const CameraSensorHelper *sensorHelper); void setLimits(utils::Duration minExposureTime, utils::Duration maxExposureTime, double minGain, double maxGain); diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index f60fddac3f04fd6f09dc782e929ff1593758c29b..ccf72cc9e8c33b1a516322c2445efa1684b7f751 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -169,12 +169,16 @@ int Agc::configure(IPAContext &context, context.activeState.agc.constraintMode = constraintModes().begin()->first; context.activeState.agc.exposureMode = exposureModeHelpers().begin()->first; - /* \todo Run this again when FrameDurationLimits is passed in */ - setLimits(context.configuration.agc.minShutterSpeed, - context.configuration.agc.maxShutterSpeed, - context.configuration.agc.minAnalogueGain, - context.configuration.agc.maxAnalogueGain, - {}); + AgcMeanLuminance::AgcSensorConfiguration sensorConfig; + sensorConfig.lineDuration = context.configuration.sensor.lineDuration; + sensorConfig.minExposureTime = context.configuration.agc.minShutterSpeed; + sensorConfig.maxExposureTime = context.configuration.agc.maxShutterSpeed; + sensorConfig.minAnalogueGain = context.configuration.agc.minAnalogueGain; + sensorConfig.maxAnalogueGain = context.configuration.agc.maxAnalogueGain; + + AgcMeanLuminance::configure(sensorConfig, context.camHelper.get()); + + /* \todo Update AGC limits when FrameDurationLimits is passed in */ resetFrameCount(); diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index f5a3c917cb6909f6ef918e5ee8e46cf97ba55010..8c07e6a6eac091d949ad71ee967c893897313cc8 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -200,13 +200,14 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.configuration.agc.measureWindow.h_size = configInfo.outputSize.width; context.configuration.agc.measureWindow.v_size = configInfo.outputSize.height; - AgcMeanLuminance::configure(context.configuration.sensor.lineDuration, - context.camHelper.get()); - - setLimits(context.configuration.sensor.minExposureTime, - context.configuration.sensor.maxExposureTime, - context.configuration.sensor.minAnalogueGain, - context.configuration.sensor.maxAnalogueGain, {}); + AgcMeanLuminance::AgcSensorConfiguration sensorConfig; + sensorConfig.lineDuration = context.configuration.sensor.lineDuration; + sensorConfig.minExposureTime = context.configuration.sensor.minExposureTime; + sensorConfig.maxExposureTime = context.configuration.sensor.maxExposureTime; + sensorConfig.minAnalogueGain = context.configuration.sensor.minAnalogueGain; + sensorConfig.maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; + + AgcMeanLuminance::configure(sensorConfig, context.camHelper.get()); context.activeState.agc.automatic.yTarget = effectiveYTarget(); From patchwork Tue Oct 28 09:31:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24843 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 01753C32CE for ; Tue, 28 Oct 2025 09:32:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 39AD5607D9; Tue, 28 Oct 2025 10:32:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pBgpnwuX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D91A5607A9 for ; Tue, 28 Oct 2025 10:32:09 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2200E4B33; Tue, 28 Oct 2025 10:30:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643821; bh=C4dEBZq342ZteknpVimceWCkvAC91oENsQRg2a/BCj0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pBgpnwuXZi6tJWdWByxWw4X8IqW3Iitv1l8cWT/zKrdY5at7FzKM9sqkWkW4Bg8U/ v4CFOpqvnLOhhLxsq6rBMAs+ORuocr+wARPimV1lg8/8324fpGjQIWs9ln3bmCllbh vaI5C0O8k8+fvit8dE4+Zlc35U/r7srhxiyWM8ic= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:51 +0100 Subject: [PATCH v2 05/10] ipa: libipa: agc: Configure with frame duration MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-5-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=11307; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=C4dEBZq342ZteknpVimceWCkvAC91oENsQRg2a/BCj0=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2Vb355W09mLmiDokHk6mO40/5mUoAv+Y5RI Qu2Pu5ARO+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlQAKCRByNAaPFqFW PL37D/90ttB9jx3emcFfKr0SP0liI+yqkHieig48eFbVOwMccP9KdBCDQvW4/Pc/5F3xWYIAXzJ A7g4hRKL2mS+CYHzg1j5fuL+X8Yhj9qlTY8TJ1Q7cxW3ftQDzVXQZz4rMYflBxvmtZ2tjRWAFrp 12wH4joNjd5y2RBsuWuw9Rdj9nk2JPAyPeHQ+PdDLFyMdFiwqBmv5CGSFBjMXJjAvttssL+cX/j r5wJUXFCW95fw+1WM8CGm8ynJHrwIYNieP3QaJFhDtb6LjBtGynix6qWsiRCBWuWWpxwfjdyB0K oNmGxtfcX9kS2FiGceldHCwa75ecQhexLRLfIKXR0oqc8zyyPVKsOoktFNwksexsM043nqVyZ5Q ifdblaArVEcsLcVXKZr7dloB6+nkWm7hU5/emborpOJgP8KfOsCtAc0RJmRH7Iz+ur9jaFotJoB LkGnRl60FaFaFjeaxognxQrvxgGkV41LKGEDvhap57R/mFffgWkZRKZk36bu2E4GodrlUUDfgVB QQKBQED57e4LbkWaKPbWh09jtSjov26N08AyqmGnXZ0QmO5GjY0aF2NkLiMM/+pWFS35rdNT9pz QfSGimzPLP9xwAJxCvHzPcNr/d0lX1v/5awGxrkkoOG32MDrQZG8pK7bOs30h1wIB8WrFhnaJjz uNc/ovMd23t3g3g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass the maximum allowed frame duration to the AGC configure() operation. This prepares for limiting the exposure time to the frame duration. In order to be able to provide to the AGC the frame duration in the IPU3 and Mali-C55 IPAs it is necessary to store the FrameDurationLimits control in the context. Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/agc.cpp | 4 ++++ src/ipa/ipu3/ipu3.cpp | 7 +++---- src/ipa/libipa/agc_mean_luminance.cpp | 6 ++++++ src/ipa/libipa/agc_mean_luminance.h | 1 + src/ipa/libipa/exposure_mode_helper.cpp | 9 +++++++++ src/ipa/libipa/exposure_mode_helper.h | 5 ++++- src/ipa/mali-c55/algorithms/agc.cpp | 4 ++++ src/ipa/mali-c55/ipa_context.h | 2 ++ src/ipa/mali-c55/mali-c55.cpp | 7 +++---- src/ipa/rkisp1/algorithms/agc.cpp | 1 + 10 files changed, 37 insertions(+), 9 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index e303d724ab165bac03ac6bb9d3891a76049cff47..3a9cde22323719d73ac125179dbfcd5158b352ea 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -115,9 +115,13 @@ int Agc::configure(IPAContext &context, context.activeState.agc.constraintMode = constraintModes().begin()->first; context.activeState.agc.exposureMode = exposureModeHelpers().begin()->first; + ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; + AgcMeanLuminance::AgcSensorConfiguration sensorConfig; sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = minExposureTime_; + sensorConfig.maxFrameDuration = + std::chrono::microseconds(frameDurationLimits.max().get()); sensorConfig.maxExposureTime = maxExposureTime_; sensorConfig.minAnalogueGain = minAnalogueGain_; sensorConfig.maxAnalogueGain = maxAnalogueGain_; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 36d8f0da430a3d4f94e2a3f760850519742ea992..5e7a3d2e1bb8d33d88c5f721d4c25e041cfb2a6e 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -278,11 +278,10 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); } - controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], - frameDurations[1], - frameDurations[2]); + context_.ctrlMap[&controls::FrameDurationLimits] = + ControlInfo(frameDurations[0], frameDurations[1], frameDurations[2]); - controls.merge(context_.ctrlMap); + controls.insert(context_.ctrlMap.begin(), context_.ctrlMap.end()); *ipaControls = ControlInfoMap(std::move(controls), controls::controls); } diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 9886864fc42514040eadb4c0b424796df1f1b6e1..195de2f2e93ce229b8047909b949e1d06ef8d5bf 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -121,6 +121,11 @@ static constexpr double kMaxRelativeLuminanceTarget = 0.95; * \brief The sensor maximum exposure time in microseconds */ +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::maxFrameDuration + * \brief The sensor maximum frame duration in microseconds + */ + /** * \var AgcMeanLuminance::AgcSensorConfiguration::minAnalogueGain * \brief The sensor minimum analogue gain absolute value @@ -363,6 +368,7 @@ void AgcMeanLuminance::configure(const AgcSensorConfiguration &config, for (auto &[id, helper] : exposureModeHelpers_) helper->configure(config.lineDuration, config.minExposureTime, config.maxExposureTime, + config.maxFrameDuration, config.minAnalogueGain, config.maxAnalogueGain, sensorHelper); } diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index 5ab75a3ac6c550a655f4bb7c8c30419ad8249183..c3d52c0b1cba23af14135dace8b3c13da4ae3e4b 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -46,6 +46,7 @@ public: utils::Duration lineDuration; utils::Duration minExposureTime; utils::Duration maxExposureTime; + utils::Duration maxFrameDuration; double minAnalogueGain; double maxAnalogueGain; }; diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index 557e14230ba4e0f120653bc09ea23a29a54cb4c8..edb8f04b245f01119d0d0b0917d10f98c7172dc0 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -84,6 +84,7 @@ ExposureModeHelper::ExposureModeHelper(const Spanfirst; context.activeState.agc.exposureMode = exposureModeHelpers().begin()->first; + ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; + AgcMeanLuminance::AgcSensorConfiguration sensorConfig; sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = context.configuration.agc.minShutterSpeed; sensorConfig.maxExposureTime = context.configuration.agc.maxShutterSpeed; + sensorConfig.maxFrameDuration = + std::chrono::microseconds(frameDurationLimits.max().get()); sensorConfig.minAnalogueGain = context.configuration.agc.minAnalogueGain; sensorConfig.maxAnalogueGain = context.configuration.agc.maxAnalogueGain; diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index bfa805c7b93f313dda2497365e83542bbc39e291..ecb2f79c0dca0e41166b88f2608c22aa72adcf8a 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -10,6 +10,8 @@ #include #include +#include + #include "libcamera/internal/bayer_format.h" #include diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index 0751513dc584ca84dd212bf8c1469dd4b40c053d..12cad7374520db8c6fa5ca233a0ef33dc7b2f287 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -231,9 +231,8 @@ void IPAMaliC55::updateControls(const IPACameraSensorInfo &sensorInfo, frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); } - ctrlMap[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], - frameDurations[1], - frameDurations[2]); + context_.ctrlMap[&controls::FrameDurationLimits] = + ControlInfo(frameDurations[0], frameDurations[1], frameDurations[2]); /* * Compute exposure time limits from the V4L2_CID_EXPOSURE control @@ -258,7 +257,7 @@ void IPAMaliC55::updateControls(const IPACameraSensorInfo &sensorInfo, * Merge in any controls that we support either statically or from the * algorithms. */ - ctrlMap.merge(context_.ctrlMap); + ctrlMap.insert(context_.ctrlMap.begin(), context_.ctrlMap.end()); *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 8c07e6a6eac091d949ad71ee967c893897313cc8..121e75449a6985f951c50fdc85b0298026344297 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -204,6 +204,7 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = context.configuration.sensor.minExposureTime; sensorConfig.maxExposureTime = context.configuration.sensor.maxExposureTime; + sensorConfig.maxFrameDuration = context.activeState.agc.maxFrameDuration; sensorConfig.minAnalogueGain = context.configuration.sensor.minAnalogueGain; sensorConfig.maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; From patchwork Tue Oct 28 09:31:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24844 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6026CBE080 for ; Tue, 28 Oct 2025 09:32:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 905A1607E9; Tue, 28 Oct 2025 10:32:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HQmdQ2lt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 436A8607AA for ; Tue, 28 Oct 2025 10:32:10 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A0F004B34; Tue, 28 Oct 2025 10:30:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643822; bh=hotdbwOYqkfu27YSExC1g/AWsdly5yABlmUf2Zw8aRE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=HQmdQ2lt2Mqoazis40z54zsqLuIc0oY32rVDHHKCi5MCFdg5VBtiMIs/zn4X0JqAM cUe66vk+PrNX8zrdP+Ke40qC0hnOD6bCGhuQTCB+TH4UCk4SezEtxUvTCoZwCaMBal PHV22/gTIcAwD7K3m0Sc0bwsl3CLPQojHwud/aLk= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:52 +0100 Subject: [PATCH v2 06/10] ipa: libipa: agc: Initialize exposure with frame duration MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-6-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1945; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=hotdbwOYqkfu27YSExC1g/AWsdly5yABlmUf2Zw8aRE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2V0Bi8e8LraYeVr4SpKdZcuxzsXkAbhqqi6 J/YBDShUx2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlQAKCRByNAaPFqFW PL4ZD/wJgVts1J6cz9EqxXKhzsbQXLwnL309WPbHfPPsExgKsk+Firh3WeiWPabWKF+Fijw0EwL PgAWjdyAP0baQ6VRBKavWfP0SwIHtA21ffQZHveZdi22bcOXDSqzh4rXhqvSQWSplaBJ3Z+0N40 WwmIYezyo++IuTdORgcv9Glt86bas0ybvj0AnMBAJcSbLTlbdcVReMwWFkpsvDyoi+Ukryn3kBt 4cc5uSg78LWm+/5Zsj2jdyOJrMySqvp7fioNolyboG+VpKa8UvYD5tU9K+tWP18BstlCFgpp969 4k0RGdCecpqknWKiTsv0UgEiI6ths0GP8t72XDq8zuPR20Fy6tYvIha0OMGvGvzugW9vBSAZvNk /0EXhHnHhhwIiIXYYdYO5hp7kJbcUK5jsZiNgd3tbNQ1ejyyVjmmgnVmi/eU8C5nKuBAONbDD9P RkjFBcSiMF3PETA3VOlOFE8B/AkThCXAPDee9NoYxzU6I6u/GQULgjt6p/Ow2PlLHCvK3OuKoZj XToJagW9QFi1zMkXkmWycOfQVHSl6gMjq+awTn12T2Svr1Tcq2RDxW5ghOwTtB3voEDp6mGoj2c T1bz1ZehLppnPWBnCoyofz838r51+G785NiGKMY+Pmg94b9jeNpdgfgxGJA1Thp3PZq6dBK1Ug4 Q5Tb539YNI0/a2w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The maximum exposure time the AGC algorithm can achieve is, by definition, limited by the maximum programmed frame duration. When initializing the AGC algorithm, use the frame duration and the sensor's exposure margin to calculate the maximum exposure, unless the IPA has asked for a fixed exposure by setting min == max. Signed-off-by: Jacopo Mondi --- src/ipa/libipa/exposure_mode_helper.cpp | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index edb8f04b245f01119d0d0b0917d10f98c7172dc0..8f40290959e24294ead008c7228e2f67ffc5adf8 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -106,12 +106,33 @@ void ExposureModeHelper::configure(utils::Duration lineDuration, const CameraSensorHelper *sensorHelper) { lineDuration_ = lineDuration; - minExposureTime_ = minExposureTime; - maxExposureTime_ = maxExposureTime; maxFrameDuration_ = maxFrameDuration; minGain_ = minGain; maxGain_ = maxGain; sensorHelper_ = sensorHelper; + + minExposureTime_ = minExposureTime; + + /* + * Compute the maximum exposure time. + * + * If maxExposureTime is equal to minExposureTime then we use them + * to fix the exposure time. + * + * Otherwise, if the exposure can range between a min and max, use the + * maxFrameDuration minus the margin as upper limit for exposure + * (capped to the provided max exposure). + */ + auto margin = sensorHelper_->exposureMargin(); + if (!margin.has_value()) { + LOG(ExposureModeHelper, Warning) + << "Exposure margin not known. Default to 4"; + margin = { 4 }; + } + + maxExposureTime_ = minExposureTime != maxExposureTime + ? maxFrameDuration - margin.value() * lineDuration + : minExposureTime; } /** From patchwork Tue Oct 28 09:31:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24845 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E14E6BE080 for ; Tue, 28 Oct 2025 09:32:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 91F68607EE; Tue, 28 Oct 2025 10:32:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BWNDT1Yd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0400C6079C for ; Tue, 28 Oct 2025 10:32:11 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2E54B4B38; Tue, 28 Oct 2025 10:30:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643822; bh=RNUI2xtURg4B/VVj79hQSs2yG/gpcllwIsjZCRcUIcc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BWNDT1Ydlz35KnqnsllSTZX4TXGeXHNctOLZuNnoPgdqlTH+fHHK44gvgVM3BGHDx OO+S9weB+jUHHgYEkhoVNXAc7BJZ2vkTDZGxRxxlK5CQzrEFgJLSJB/D6hEKc5eXn/ VFh2Z9WOmnam82z5JGvzpHOVmYyDquo3IjADQM0E= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:53 +0100 Subject: [PATCH v2 07/10] ipa: libipa: agc: Set exposure limits with frame duration MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-7-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=9294; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=RNUI2xtURg4B/VVj79hQSs2yG/gpcllwIsjZCRcUIcc=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2VV35YcZuNFy8iUrKLvGkjOfGHuCuzymYKI gm2D2YnLp6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlQAKCRByNAaPFqFW PFZaD/9B6hiUApzGIQOgBdhJBME2by2fT6fdeIDYRx8ol7MuejEGQSr++xnCV8YX99N7U42txAU nK4U78eZoEzchvpjLeWkm8DJi6Hfg9CM8KS1/l/qcijnFeof5Oahm6CEudbzIvBS7PtHXXFlJ/j UNff0RuGOczKFyzgUNNrllj3PfLUuTq/b5sCvtNeuQTrItWKf9SJ+ZpHs79yoBoY70dHJPWjeaI 6aOxL9tW175NyctCL1pa9BYJZfy+x7vklv6BN+wc4mN1UMZw+oAutXOxkwESIGrtLyAbV/M+8AZ KBcJ6HqEXAvzfVESbP8ihM3lWlm9pSNqbjnK5XtL9YyEPQQ5NZkgZ0gkUBs0xB7DiyUlTtadr+H wH6yxdsl6vjl2tlYucx71BmcdyDTZcZwc2k2bHOD8WylVBn3zvCHKpeoTsc1sM2HOCklxF5dp3z hf+841rzC+H3jFd3Gm6KKUy4an+R6SDZCO7SYpVHslyTqSFHJSA5UMbfu46NYmhW/DPWdmT9P17 5PidDV7gcY/VrvB1cLicB2jWi8FZH5Rm1+ZwLXeWlPJf2n2uGBkHDFsLQkn7cDv++kSLMLLX8+3 pGeGYWXPsDuCft/RFf7aZHYTn0er6hiBPIGoXk22qRdpLlCZEBn1qZiLoulhEesaLeQcaS4+gwd uP+peSkutB2dJgQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Similarly to what now happens at configure() time, the AGC exposure limits updated at runtime with the setLimits() function are now computed using the maximum frame duration. Factor-out in the ExposureModeHelper class the exposure limits calculation from the configure() function so that it can be performed at setLimits() time too. Update the only IPA that handles limits update at runtime (RkISP1) to pass to the AGC the updated frame durations. Signed-off-by: Jacopo Mondi --- src/ipa/libipa/agc_mean_luminance.cpp | 5 ++- src/ipa/libipa/agc_mean_luminance.h | 3 +- src/ipa/libipa/exposure_mode_helper.cpp | 59 +++++++++++++++++++-------------- src/ipa/libipa/exposure_mode_helper.h | 5 ++- src/ipa/rkisp1/algorithms/agc.cpp | 8 ++--- 5 files changed, 48 insertions(+), 32 deletions(-) diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 195de2f2e93ce229b8047909b949e1d06ef8d5bf..1098865519ca9ca2679331edd9441720366c717d 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -453,6 +453,7 @@ int AgcMeanLuminance::parseTuningData(const YamlObject &tuningData) * \brief Set the ExposureModeHelper limits for this class * \param[in] minExposureTime Minimum exposure time to allow * \param[in] maxExposureTime Maximum ewposure time to allow + * \param[in] maxFrameDuration Maximum frame duration * \param[in] minGain Minimum gain to allow * \param[in] maxGain Maximum gain to allow * \param[in] constraints Additional constraints to apply @@ -462,11 +463,13 @@ int AgcMeanLuminance::parseTuningData(const YamlObject &tuningData) */ void AgcMeanLuminance::setLimits(utils::Duration minExposureTime, utils::Duration maxExposureTime, + utils::Duration maxFrameDuration, double minGain, double maxGain, std::vector constraints) { for (auto &[id, helper] : exposureModeHelpers_) - helper->setLimits(minExposureTime, maxExposureTime, minGain, maxGain); + helper->setLimits(minExposureTime, maxExposureTime, maxFrameDuration, + minGain, maxGain); additionalConstraints_ = std::move(constraints); } diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index c3d52c0b1cba23af14135dace8b3c13da4ae3e4b..0f7a4f53188d8d2759076c37d186efc088ddd279 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -61,7 +61,8 @@ public: } void setLimits(utils::Duration minExposureTime, utils::Duration maxExposureTime, - double minGain, double maxGain, std::vector constraints); + utils::Duration maxFrameDuration, double minGain, double maxGain, + std::vector constraints); std::map> constraintModes() { diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index 8f40290959e24294ead008c7228e2f67ffc5adf8..d1d94cf46d5ef2c077508fd94dfe9715fdbc6f03 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -79,6 +79,33 @@ ExposureModeHelper::ExposureModeHelper(const SpanexposureMargin(); + if (!margin.has_value()) { + LOG(ExposureModeHelper, Warning) + << "Exposure margin not known. Default to 4"; + margin = { 4 }; + } + + maxExposureTime_ = minExposureTime != maxExposureTime + ? maxFrameDuration - margin.value() * lineDuration_ + : minExposureTime; +} + /** * \brief Configure sensor details * \param[in] lineDuration The current line length of the sensor @@ -111,34 +138,14 @@ void ExposureModeHelper::configure(utils::Duration lineDuration, maxGain_ = maxGain; sensorHelper_ = sensorHelper; - minExposureTime_ = minExposureTime; - - /* - * Compute the maximum exposure time. - * - * If maxExposureTime is equal to minExposureTime then we use them - * to fix the exposure time. - * - * Otherwise, if the exposure can range between a min and max, use the - * maxFrameDuration minus the margin as upper limit for exposure - * (capped to the provided max exposure). - */ - auto margin = sensorHelper_->exposureMargin(); - if (!margin.has_value()) { - LOG(ExposureModeHelper, Warning) - << "Exposure margin not known. Default to 4"; - margin = { 4 }; - } - - maxExposureTime_ = minExposureTime != maxExposureTime - ? maxFrameDuration - margin.value() * lineDuration - : minExposureTime; + setMaxExposure(minExposureTime, maxExposureTime, maxFrameDuration); } /** * \brief Set the exposure time and gain limits * \param[in] minExposureTime The minimum exposure time supported * \param[in] maxExposureTime The maximum exposure time supported + * \param[in] maxFrameDuration The maximum frame duration * \param[in] minGain The minimum analogue gain supported * \param[in] maxGain The maximum analogue gain supported * @@ -150,15 +157,19 @@ void ExposureModeHelper::configure(utils::Duration lineDuration, * If the algorithm using the helpers needs to indicate that either exposure time * or analogue gain or both should be fixed it can do so by setting both the * minima and maxima to the same value. + * + * The exposure time limits are calculated using \a maxFrameDuration as the + * upper bound, the \a maxExposureTime paramter effectivelly only serves + * to indicate that the caller wants a fixed exposure value. */ void ExposureModeHelper::setLimits(utils::Duration minExposureTime, utils::Duration maxExposureTime, + utils::Duration maxFrameDuration, double minGain, double maxGain) { - minExposureTime_ = minExposureTime; - maxExposureTime_ = maxExposureTime; minGain_ = minGain; maxGain_ = maxGain; + setMaxExposure(minExposureTime, maxExposureTime, maxFrameDuration); } utils::Duration ExposureModeHelper::clampExposureTime(utils::Duration exposureTime, diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h index 8a7d2161912d318c2d1c5c8d009edc8a2d0c0427..b1de96d4a3a13e3771ff2162f11201e5f9253265 100644 --- a/src/ipa/libipa/exposure_mode_helper.h +++ b/src/ipa/libipa/exposure_mode_helper.h @@ -31,7 +31,7 @@ public: double minGain, double maxGain, const CameraSensorHelper *sensorHelper); void setLimits(utils::Duration minExposureTime, utils::Duration maxExposureTime, - double minGain, double maxGain); + utils::Duration maxFrameDuration, double minGain, double maxGain); std::tuple splitExposure(utils::Duration exposure) const; @@ -43,6 +43,9 @@ public: double maxGain() const { return maxGain_; } private: + void setMaxExposure(utils::Duration minExposureTime, + utils::Duration maxExposureTime, + utils::Duration maxFrameDuration); utils::Duration clampExposureTime(utils::Duration exposureTime, double *quantizationGain = nullptr) const; double clampGain(double gain, double *quantizationGain = nullptr) const; diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 121e75449a6985f951c50fdc85b0298026344297..96189254885c47d29b92f3fb40f0e86d10e2a7fc 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -579,9 +579,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, if (frameContext.agc.autoExposureEnabled) { minExposureTime = context.configuration.sensor.minExposureTime; - maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, - context.configuration.sensor.minExposureTime, - context.configuration.sensor.maxExposureTime); + maxExposureTime = context.configuration.sensor.maxExposureTime; } else { minExposureTime = context.configuration.sensor.lineDuration * frameContext.agc.exposure; @@ -600,8 +598,8 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, if (context.activeState.wdr.mode != controls::WdrOff) additionalConstraints.push_back(context.activeState.wdr.constraint); - setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, - std::move(additionalConstraints)); + setLimits(minExposureTime, maxExposureTime, frameContext.agc.maxFrameDuration, + minAnalogueGain, maxAnalogueGain, std::move(additionalConstraints)); /* * The Agc algorithm needs to know the effective exposure value that was From patchwork Tue Oct 28 09:31:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24846 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B41BBE080 for ; Tue, 28 Oct 2025 09:32:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 14933607F3; Tue, 28 Oct 2025 10:32:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AoBAfqZR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 08616607B6 for ; Tue, 28 Oct 2025 10:32:12 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 48722176B; Tue, 28 Oct 2025 10:30:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643823; bh=7tTe6peYOLaAek4d+BsUdN3DnG0br7JpENOXWV164eQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=AoBAfqZRYqkj0pvDvNX09cz48UPzcPOwPoqmCn23IHvcwaqsrkqsdUw1XcGqvqaTq T0hZCB6N+6wMDqFQ2ySb0kVxvMUzdo/fvnuVoioDvo2vWGjTwcd7+0pvFINc21G332 yyTxTl93SW3iKyxUintaJLIWQqZLvo6UGjXVeBj4= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:54 +0100 Subject: [PATCH v2 08/10] ipa: libipa: agc: Initialize a sensible frame duration MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-8-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12310; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=7tTe6peYOLaAek4d+BsUdN3DnG0br7JpENOXWV164eQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2WRutu0JDhEazAyVR3QUgWr4vZWUXN2IXrR WretAoppAKJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlgAKCRByNAaPFqFW PDM8D/4pyBxucuD/xUDwOSlBwPdLeQvTWy0rjcKWdIAbIP+Rgunr99R4cnRwWs2MAtju3v+n9HW +H46/toTvDIlWp0ccn/ZH+Gx3m6aX+Cs3vFXc6F/HXxKmx57z8ojRJSbYnIJx36RISh4d0jgUi/ esLss72dZb0tFRfKwvorTLAF6H5+o3fbDET1tFobIz4tso7LD6qA5Gon4hkcucGMiL20iAubzzk 4RW71jPYwIaZtttEPX0H4VeThb1En++ivOVgtjGnbrIjWO8xcN33wCVICbp3Rt94x/leeYNkXbc SkpvVGRN8MoU6a9F/ixsgq2pdaVijNoJys3ccPXhswZo8q/nTlCcQ4g1Xpw+OXbkXwWuAE5f66y dXjbh7tq1/JPXzXgbv517chYRNK0Vb/K+FCVjJ4uw4921zXxZp9xNxyKy4JGXD8eY+AoDqOoTOJ l+hMzL8RagubC/w+UqY7fIz19jqR9gT3q4zxAKIahcTA3QekinlpP6ovbuFXSq8TCrDYM9hs1ks P+7ANKfO1oDo8tNfzpLYvVjsINWHHr52cDigQ4qRxI3vGN4giY66vwKf20wvsGKg+442fd9P/+f LHuvoq1Uynpmx+OMRttoBwzBSJoBcURbwV3hqZI+EwvFyTUasBQknnHiOpK6wslkpRN/zCGmgKy ERte+Kujuhn8NOg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC algorithm implemented by AgcMeanLuminance splits the desired exposure between shutter time, analogue and digital gains. It tries to maximize the shutter time up to the specified limits and then increases the gains. When initializing the algorithm at configure() time the IPA modules have access to the Camera's absolute limits, which might include a very long maximum frame duration. If such duration is used to clamp the maximum exposure, the algorithm will maximize the shutter time, resulting in a very low frame rate. Try to pick a slighlty saner default of 30, 15 or 10 fps to limit the maximum shutter time with to provide a reasonable out-of-the-box behaviour for users. The new frame duration limit calculated by the AGC algorithm is passed back to the IPA module that shall use it to compute the right blankings to configure the camera with. At the moment, only the RkISP1 IPA actually controls the frame duration, and so it is the only IPA that needs to use the frame duration limit computed by the AGC algorithm. Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/agc.cpp | 4 +++- src/ipa/libipa/agc_mean_luminance.cpp | 20 ++++++++++++++------ src/ipa/libipa/agc_mean_luminance.h | 3 ++- src/ipa/libipa/exposure_mode_helper.cpp | 21 +++++++++++++++++---- src/ipa/libipa/exposure_mode_helper.h | 7 ++++--- src/ipa/mali-c55/algorithms/agc.cpp | 4 +++- src/ipa/rkisp1/algorithms/agc.cpp | 13 +++++++++---- 7 files changed, 52 insertions(+), 20 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 3a9cde22323719d73ac125179dbfcd5158b352ea..51aa4e51841ca8c6363c0211819a7d14cc3baec6 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -120,13 +120,15 @@ int Agc::configure(IPAContext &context, AgcMeanLuminance::AgcSensorConfiguration sensorConfig; sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = minExposureTime_; + sensorConfig.minFrameDuration = + std::chrono::microseconds(frameDurationLimits.min().get()); sensorConfig.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); sensorConfig.maxExposureTime = maxExposureTime_; sensorConfig.minAnalogueGain = minAnalogueGain_; sensorConfig.maxAnalogueGain = maxAnalogueGain_; - AgcMeanLuminance::configure(sensorConfig, context.camHelper.get()); + AgcMeanLuminance::configure(&sensorConfig, context.camHelper.get()); /* \todo Update AGC limits when FrameDurationLimits is passed in */ diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 1098865519ca9ca2679331edd9441720366c717d..36bdf76dca45c837d5ef4eb03244a43d1262c675 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -121,6 +121,11 @@ static constexpr double kMaxRelativeLuminanceTarget = 0.95; * \brief The sensor maximum exposure time in microseconds */ +/** + * \var AgcMeanLuminance::AgcSensorConfiguration::minFrameDuration + * \brief The sensor minimum exposure time in microseconds + */ + /** * \var AgcMeanLuminance::AgcSensorConfiguration::maxFrameDuration * \brief The sensor maximum frame duration in microseconds @@ -355,21 +360,24 @@ int AgcMeanLuminance::parseExposureModes(const YamlObject &tuningData) /** * \brief Configure the exposure mode helpers - * \param[in] config The sensor configuration + * \param[inout] config The sensor configuration * \param[in] sensorHelper The sensor helper * * This function configures the exposure mode helpers by providing them the * sensor configuration parameters and the sensor helper, so they can correctly * take quantization effects into account. + * + * The maximum frame duration passed in as a member of \a config is updated to + * the AGC algorithm startup value. */ -void AgcMeanLuminance::configure(const AgcSensorConfiguration &config, +void AgcMeanLuminance::configure(AgcSensorConfiguration *config, const CameraSensorHelper *sensorHelper) { for (auto &[id, helper] : exposureModeHelpers_) - helper->configure(config.lineDuration, - config.minExposureTime, config.maxExposureTime, - config.maxFrameDuration, - config.minAnalogueGain, config.maxAnalogueGain, + helper->configure(config->lineDuration, + config->minExposureTime, config->maxExposureTime, + config->minFrameDuration, &config->maxFrameDuration, + config->minAnalogueGain, config->maxAnalogueGain, sensorHelper); } diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index 0f7a4f53188d8d2759076c37d186efc088ddd279..b00772760035a477152d638809ac71fcbc66cb68 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -46,12 +46,13 @@ public: utils::Duration lineDuration; utils::Duration minExposureTime; utils::Duration maxExposureTime; + utils::Duration minFrameDuration; utils::Duration maxFrameDuration; double minAnalogueGain; double maxAnalogueGain; }; - void configure(const AgcSensorConfiguration &config, + void configure(AgcSensorConfiguration *config, const CameraSensorHelper *sensorHelper); int parseTuningData(const YamlObject &tuningData); diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index d1d94cf46d5ef2c077508fd94dfe9715fdbc6f03..46ba535c67d92bd384095b250b925cdc0da40b38 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -111,7 +111,8 @@ void ExposureModeHelper::setMaxExposure(utils::Duration minExposureTime, * \param[in] lineDuration The current line length of the sensor * \param[in] minExposureTime The minimum exposure time supported * \param[in] maxExposureTime The maximum exposure time supported - * \param[in] maxFrameDuration The maximum frame duration + * \param[in] minFrameDuration The minimum frame duration + * \param[inout] maxFrameDuration The maximum frame duration * \param[in] minGain The minimum analogue gain supported * \param[in] maxGain The maximum analogue gain supported * \param[in] sensorHelper The sensor helper @@ -128,17 +129,29 @@ void ExposureModeHelper::setMaxExposure(utils::Duration minExposureTime, void ExposureModeHelper::configure(utils::Duration lineDuration, utils::Duration minExposureTime, utils::Duration maxExposureTime, - utils::Duration maxFrameDuration, + utils::Duration minFrameDuration, + utils::Duration *maxFrameDuration, double minGain, double maxGain, const CameraSensorHelper *sensorHelper) { lineDuration_ = lineDuration; - maxFrameDuration_ = maxFrameDuration; minGain_ = minGain; maxGain_ = maxGain; sensorHelper_ = sensorHelper; - setMaxExposure(minExposureTime, maxExposureTime, maxFrameDuration); + static constexpr utils::Duration duration30fps = 33.333 * 1ms; + static constexpr utils::Duration duration15fps = 66.666 * 1ms; + static constexpr utils::Duration duration10fps = 100.00 * 1ms; + utils::Duration frameDuration = minFrameDuration < duration30fps + ? duration30fps + : minFrameDuration < duration15fps + ? duration15fps + : minFrameDuration < duration10fps + ? duration10fps : minFrameDuration; + frameDuration = std::min(frameDuration, *maxFrameDuration); + setMaxExposure(minExposureTime, maxExposureTime, frameDuration); + + maxFrameDuration_ = *maxFrameDuration = frameDuration; } /** diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h index b1de96d4a3a13e3771ff2162f11201e5f9253265..492d2787f75945fb487b351a60f13756caec66e0 100644 --- a/src/ipa/libipa/exposure_mode_helper.h +++ b/src/ipa/libipa/exposure_mode_helper.h @@ -27,11 +27,12 @@ public: ~ExposureModeHelper() = default; void configure(utils::Duration lineLength, utils::Duration minExposureTime, - utils::Duration maxExposureTime, utils::Duration maxFrameDuration, - double minGain, double maxGain, + utils::Duration maxExposureTime, utils::Duration minFrameDuration, + utils::Duration *maxFrameDuration, double minGain, double maxGain, const CameraSensorHelper *sensorHelper); void setLimits(utils::Duration minExposureTime, utils::Duration maxExposureTime, - utils::Duration maxFrameDuration, double minGain, double maxGain); + utils::Duration maxFrameDuration, + double minGain, double maxGain); std::tuple splitExposure(utils::Duration exposure) const; diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 5bfb4ab17ecb3814a10baec0f619267772ed8b3f..4e2fa4386be621c63d992b75d3d9efc525c3d69c 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -175,12 +175,14 @@ int Agc::configure(IPAContext &context, sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = context.configuration.agc.minShutterSpeed; sensorConfig.maxExposureTime = context.configuration.agc.maxShutterSpeed; + sensorConfig.minFrameDuration = + std::chrono::microseconds(frameDurationLimits.min().get()); sensorConfig.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); sensorConfig.minAnalogueGain = context.configuration.agc.minAnalogueGain; sensorConfig.maxAnalogueGain = context.configuration.agc.maxAnalogueGain; - AgcMeanLuminance::configure(sensorConfig, context.camHelper.get()); + AgcMeanLuminance::configure(&sensorConfig, context.camHelper.get()); /* \todo Update AGC limits when FrameDurationLimits is passed in */ diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 96189254885c47d29b92f3fb40f0e86d10e2a7fc..a983efbecbd84fa9e4af8b9642fbdda124ba0b9f 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -190,10 +190,13 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.activeState.agc.meteringMode = static_cast(meteringModes_.begin()->first); - /* Limit the frame duration to match current initialisation */ + /* + * Initialize frame duration with the camera limits and update it to + * the value computed by AgcMeanLuminance::configure(). + */ ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get()); - context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); + utils::Duration maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); context.configuration.agc.measureWindow.h_offs = 0; context.configuration.agc.measureWindow.v_offs = 0; @@ -204,12 +207,14 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = context.configuration.sensor.minExposureTime; sensorConfig.maxExposureTime = context.configuration.sensor.maxExposureTime; - sensorConfig.maxFrameDuration = context.activeState.agc.maxFrameDuration; + sensorConfig.minFrameDuration = context.activeState.agc.minFrameDuration; + sensorConfig.maxFrameDuration = maxFrameDuration; sensorConfig.minAnalogueGain = context.configuration.sensor.minAnalogueGain; sensorConfig.maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; - AgcMeanLuminance::configure(sensorConfig, context.camHelper.get()); + AgcMeanLuminance::configure(&sensorConfig, context.camHelper.get()); + context.activeState.agc.maxFrameDuration = sensorConfig.maxFrameDuration; context.activeState.agc.automatic.yTarget = effectiveYTarget(); resetFrameCount(); From patchwork Tue Oct 28 09:31:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24847 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BC56BBE080 for ; Tue, 28 Oct 2025 09:32:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 48622607AC; Tue, 28 Oct 2025 10:32:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KS4cXdge"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9FABA607BC for ; Tue, 28 Oct 2025 10:32:12 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E56281AED; Tue, 28 Oct 2025 10:30:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643824; bh=hv6EQVhzC+oNcaqjzg+vZ5gTG1/2NINcRSZSFYP4LNQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=KS4cXdgeN4+YeuxIn3/JmYo5LXr5bN0gQH10clDTENhawtWXf1ragUm2B30eeYV9H aHgNcCr64NlhSHbxv9smtHQ8Dpj8XRDWHlYJ9CgrWW0QbwAoZUrc2wcw/0qsesKIg1 U6AujbKaz74cCA6fPyMFrk1R+Haj5RA+BC4D2WTY= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:55 +0100 Subject: [PATCH v2 09/10] ipa: libipa: agc: Calculate frame duration in helper MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-9-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12672; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=hv6EQVhzC+oNcaqjzg+vZ5gTG1/2NINcRSZSFYP4LNQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2Ww/01hoPy0MQRh+fGyLkAMl8eaD8YXTAnf jUyWyQniV+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlgAKCRByNAaPFqFW PKbkEACYrJuWEGDzIDx+8oyZ7UrDYs/IF/5R/7cVYujZTV8pClS/Gvwgp6GTUsHs8CR3t8jkLLE CofcFwQcKUurrXpKMf+xuY5TnBqlnn8gJwGO6ppzSgTUUlgQZxnav/G8qvkGnmHo0TcwiEoMgx1 YXXpK4PiaGd4olQmkJXU3/jHb0MBM5M1LWSyPtNP7N2xRyI9hm85O0Y0mJQ+ANzP2HUqDMOgRZw hYsgRt/ZeBU5ttluMFljVZEB0kCnNu0TIsI2llEohNnY/nR+xoUS6Esvsodt8jNK/tALGAsw2+o 9laV9fc61hrXLFMbjMmRcJGK18M0ibdnTsEeEiYvMZnaHM0ky4c0aAcZclv6YT/Vw0dq7OQtepx 9KJ/zx6/2gSSb5d7udRayB2hNFBuWOKpbH3A+/3Um0/6KMMbx6ysV3AAFVecsNIDyOHe4qPn2Nv yw22kOR8lUKtq3vcrdKUxT7RHYaF8bPNT1Cg7eHBkLZALGndl1FOzN12VsD2EWjBVzvnjq83tnJ jQ0j8M7H3OY4qp0jPoGb9n4OONo8M5gGriGp4dfKh4Me3/dcxl8iWbO+UYUHSPCuKBkeEAZCGmG oF5npg+SnEq+o4Sjm85idkU2k0Q1iENfi2ssf5gGqdSabfeoW6yvADxaI4tLcjuFwGxiqlJcEOX sqOry4PNE0gn0NA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The RkISP1 IPA is the only IPA that derives from AgcMeanLuminance that populates the FrameDuration control. The way frame duration is calculated is by using the shutter time as returned by the Agc helpers and use that value to populate the FrameDuration metadata and calculate the vblank. This is however not correct as the exposure time shall be shorter than the frame duration by a sensor-specific margin. The AGC helpers are in charge of calculating the desired exposure and split it between shutter time and gains. They already use the exposure margin to limit the maximum shutter time to the maximum frame duration. Let the helpers calculate the frame duration and return it with shutter time and gains from calculateNewEv(). Also update the Mali C55 and IPU3 IPAs, however they do not currently handle the frame duration and never change vblank so they don't report the FrameDuration metadata (Mali C55) or return a fixed value (IPU3). Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/agc.cpp | 12 +++++++----- src/ipa/libipa/agc_mean_luminance.cpp | 6 +++--- src/ipa/libipa/agc_mean_luminance.h | 2 +- src/ipa/libipa/exposure_mode_helper.cpp | 33 +++++++++++++++++++++++---------- src/ipa/libipa/exposure_mode_helper.h | 5 ++++- src/ipa/mali-c55/algorithms/agc.cpp | 3 ++- src/ipa/rkisp1/algorithms/agc.cpp | 11 ++++++----- 7 files changed, 46 insertions(+), 26 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 51aa4e51841ca8c6363c0211819a7d14cc3baec6..34cb31a3ef4ff8a4774351e1d1a8b5a46aca33cc 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -236,8 +236,9 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, utils::Duration effectiveExposureValue = exposureTime * analogueGain; utils::Duration newExposureTime; + utils::Duration frameDuration; double aGain, qGain, dGain; - std::tie(newExposureTime, aGain, qGain, dGain) = + std::tie(newExposureTime, frameDuration, aGain, qGain, dGain) = calculateNewEv(context.activeState.agc.constraintMode, context.activeState.agc.exposureMode, hist, effectiveExposureValue); @@ -254,12 +255,13 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, metadata.set(controls::AnalogueGain, frameContext.sensor.gain); metadata.set(controls::ExposureTime, exposureTime.get()); - /* \todo Use VBlank value calculated from each frame exposure. */ + /* \todo Use frameDuration to calculate the right vblank. */ + uint32_t vTotal = context.configuration.sensor.size.height + context.configuration.sensor.defVBlank; - utils::Duration frameDuration = context.configuration.sensor.lineDuration - * vTotal; - metadata.set(controls::FrameDuration, frameDuration.get()); + utils::Duration defFrameDuration = context.configuration.sensor.lineDuration + * vTotal; + metadata.set(controls::FrameDuration, defFrameDuration.get()); } REGISTER_IPA_ALGORITHM(Agc, "Agc") diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 36bdf76dca45c837d5ef4eb03244a43d1262c675..3722602921d6f1716f4cfedcae84023e4bea02a6 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -646,10 +646,10 @@ utils::Duration AgcMeanLuminance::filterExposure(utils::Duration exposureValue) * exposure value is filtered to prevent rapid changes from frame to frame, and * divided into exposure time, analogue, quantization and digital gain. * - * \return Tuple of exposure time, analogue gain, quantization gain and digital - * gain + * \return Tuple of exposure time, frame duration analogue gain, quantization + * gain and digital gain */ -std::tuple +std::tuple AgcMeanLuminance::calculateNewEv(uint32_t constraintModeIndex, uint32_t exposureModeIndex, const Histogram &yHist, diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index b00772760035a477152d638809ac71fcbc66cb68..71c4951080c728e87530caf189fbd52f92b3d033 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -80,7 +80,7 @@ public: return controls_; } - std::tuple + std::tuple calculateNewEv(uint32_t constraintModeIndex, uint32_t exposureModeIndex, const Histogram &yHist, utils::Duration effectiveExposureValue); diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index 46ba535c67d92bd384095b250b925cdc0da40b38..e9a37a3771cdfbb35ad4c8a373d3189f8a468c7b 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -100,9 +100,10 @@ void ExposureModeHelper::setMaxExposure(utils::Duration minExposureTime, << "Exposure margin not known. Default to 4"; margin = { 4 }; } + exposureMargin_ = margin.value() * lineDuration_; maxExposureTime_ = minExposureTime != maxExposureTime - ? maxFrameDuration - margin.value() * lineDuration_ + ? maxFrameDuration - exposureMargin_ : minExposureTime; } @@ -152,6 +153,7 @@ void ExposureModeHelper::configure(utils::Duration lineDuration, setMaxExposure(minExposureTime, maxExposureTime, frameDuration); maxFrameDuration_ = *maxFrameDuration = frameDuration; + minFrameDuration_ = minFrameDuration; } /** @@ -207,6 +209,17 @@ double ExposureModeHelper::clampGain(double gain, double *quantizationGain) cons return sensorHelper_->quantizeGain(clamped, quantizationGain); } +utils::Duration ExposureModeHelper::frameDurationFromExposure(utils::Duration exposureTime) const +{ + /* + * The maximum exposure value has already been clamped to the maximum + * frame duration. Re-apply the exposure margin and make sure we don't + * go below the minium frame duration. + */ + utils::Duration frameDuration = exposureTime + exposureMargin_; + return std::max(frameDuration, minFrameDuration_); +} + /** * \brief Split exposure into exposure time and gain * \param[in] exposure Exposure value @@ -244,7 +257,7 @@ double ExposureModeHelper::clampGain(double gain, double *quantizationGain) cons * \return Tuple of exposure time, analogue gain, quantization gain and digital * gain */ -std::tuple +std::tuple ExposureModeHelper::splitExposure(utils::Duration exposure) const { ASSERT(maxExposureTime_); @@ -266,8 +279,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(minGain_, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } double stageGain = clampGain(1.0); @@ -292,8 +305,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(exposure / exposureTime, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } /* Clamp the exposureTime to stageExposureTime and regulate gain. */ @@ -302,8 +315,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(exposure / exposureTime, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } lastStageGain = stageGain; @@ -320,8 +333,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(exposure / exposureTime, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } /** diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h index 492d2787f75945fb487b351a60f13756caec66e0..8dc94cf808945e0ff3cea080046b1cafd453a014 100644 --- a/src/ipa/libipa/exposure_mode_helper.h +++ b/src/ipa/libipa/exposure_mode_helper.h @@ -34,7 +34,7 @@ public: utils::Duration maxFrameDuration, double minGain, double maxGain); - std::tuple + std::tuple splitExposure(utils::Duration exposure) const; utils::Duration minExposureTime() const { return minExposureTime_; } @@ -50,6 +50,7 @@ private: utils::Duration clampExposureTime(utils::Duration exposureTime, double *quantizationGain = nullptr) const; double clampGain(double gain, double *quantizationGain = nullptr) const; + utils::Duration frameDurationFromExposure(utils::Duration exposureTime) const; std::vector exposureTimes_; std::vector gains_; @@ -58,6 +59,8 @@ private: utils::Duration minExposureTime_; utils::Duration maxExposureTime_; utils::Duration maxFrameDuration_; + utils::Duration minFrameDuration_; + utils::Duration exposureMargin_; double minGain_; double maxGain_; const CameraSensorHelper *sensorHelper_; diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 4e2fa4386be621c63d992b75d3d9efc525c3d69c..6ecb18d959c7f978966d619beb43e1c2cbfeec5f 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -391,9 +391,10 @@ void Agc::process(IPAContext &context, utils::Duration currentShutter = exposure * configuration.sensor.lineDuration; utils::Duration effectiveExposureValue = currentShutter * totalGain; + utils::Duration frameDuration; utils::Duration shutterTime; double aGain, qGain, dGain; - std::tie(shutterTime, aGain, qGain, dGain) = + std::tie(shutterTime, frameDuration, aGain, qGain, dGain) = calculateNewEv(activeState.agc.constraintMode, activeState.agc.exposureMode, statistics_.yHist, effectiveExposureValue); diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index a983efbecbd84fa9e4af8b9642fbdda124ba0b9f..c8a34c3d1bf2ed71b9cf203e48c4b42910eea797 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -625,15 +625,17 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, setExposureCompensation(pow(2.0, frameContext.agc.exposureValue)); utils::Duration newExposureTime; + utils::Duration frameDuration; double aGain, qGain, dGain; - std::tie(newExposureTime, aGain, qGain, dGain) = + std::tie(newExposureTime, frameDuration, aGain, qGain, dGain) = calculateNewEv(frameContext.agc.constraintMode, frameContext.agc.exposureMode, hist, effectiveExposureValue); LOG(RkISP1Agc, Debug) - << "Divided up exposure time, analogue gain, quantization gain" - << " and digital gain are " << newExposureTime << ", " << aGain + << "Divided up exposure time, frame duration, analogue gain," + << "quantization gain and digital gain are " + << newExposureTime << ", " << frameDuration << ", " << aGain << ", " << qGain << " and " << dGain; IPAActiveState &activeState = context.activeState; @@ -646,8 +648,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * Expand the target frame duration so that we do not run faster than * the minimum frame duration when we have short exposures. */ - processFrameDuration(context, frameContext, - std::max(frameContext.agc.minFrameDuration, newExposureTime)); + processFrameDuration(context, frameContext, frameDuration); fillMetadata(context, frameContext, metadata); expMeans_ = {}; From patchwork Tue Oct 28 09:31:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24848 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A4199BE080 for ; Tue, 28 Oct 2025 09:32:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 577076079C; Tue, 28 Oct 2025 10:32:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UNDHDc3T"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CBC96079F for ; Tue, 28 Oct 2025 10:32:13 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 813741F0E; Tue, 28 Oct 2025 10:30:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643824; bh=djORRSqv0qvw0i3v7L45OrPwtKtCDgKQJPDSZ33ZI6E=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=UNDHDc3TUzMkur5avlMDRSWBHIYqaaHedvyGukAfbpZJI35qLltYyJWML2ptfCEN2 eMlDa5zdHZ4QFeMhYYwNqwsWj2ROzMN/RHXWYcIG2uATEpL1ESi1m8YC6/uQDw13X4 nkO3pE3Cv7QDm7ycN0VmOke2WeYOVxnX8fl6+RJw= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:56 +0100 Subject: [PATCH DNI v2 10/10] ipa: mali: Handle FrameDurationLimits MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-10-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8448; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=djORRSqv0qvw0i3v7L45OrPwtKtCDgKQJPDSZ33ZI6E=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2WUU8UnGdjl9pnH/U3cJj8U9CnYTtICrcPO MuyaMnKYCWJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlgAKCRByNAaPFqFW PPTjD/9xc7uC4H41/Rp/njYmXpmqLYsgU8pZ/DMtNA4hBpZYU4ZtDuE6kLkWvUmvKddyP2Axq/1 zg0hLgUQG4kOijilR1ddpmM9+rG0ziKk7RVxP/DBMCJTgPQMMlvrxmg/NOhzHPYp0ILq410yt9w 8mHKEx/wRiRAX5NVAmwB9DWnU3cwm4tSP388iPLdDisRNl8pL2hdNdWOq+NXeXLeJuNQwxzY6GD qOwE/VyIQYtTAuDk+HqZim9/Lie9ATcd0M5iNbwcX0mL/YiY17vs45C+9MPCII+5Fcxq+Z++BUG peXMuPBDYovMr4DnM4TbsuHt4gBJZmx5ZevcD6tMl1cN/Y9WsWzXSpJPrF/QXEfsTwLT0JZCxAd MQiO4Sz+oIyBZZRP3GfbGPO+UuDRroOqKsYE9KENAA+DLHQZnQpfX03T2+IOc0guJZOpLHyrQCM LDkqm87m5a4fGbfJL4Ii4fXydpy5/C15FUT93Cw2HRsX9J3isQM0WIC/OcPDIRIODlXMOF9XRgE l9bmgAzT3JNkGzzcmUJ8qqvdKHClX6nWba90OzhUDyRW6nKigekQYq2kaBawuBy46llojkeMS0X L1YBk4LjIBZOVRWCYWUXoLn9bUcIviTUBVJhgc70qDjPn8BJZtGgiK3QyeomaaNuy9CwvqrxMLT SIrB86VeUbIUSbw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the FrameDurationLimits control for the Mali C55 platform. The frame duration is: 1) Adjuted at configure() time to what the AGC algorithm computed 2) Stored at queueRequest() time in the active state and in the per-frame context 3) Computed at process() time by the AGC calculateNewEV() function and used to compute the desired vblank The VBLANK control is now handled by both the IPA and the pipeline handler and correctly programmed on the sensor. Signed-off-by: Jacopo Mondi --- src/ipa/mali-c55/algorithms/agc.cpp | 63 +++++++++++++++++++++++++++++++------ src/ipa/mali-c55/ipa_context.h | 6 ++++ src/ipa/mali-c55/mali-c55.cpp | 10 ++++-- 3 files changed, 67 insertions(+), 12 deletions(-) diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 6ecb18d959c7f978966d619beb43e1c2cbfeec5f..60224cc0e261709000901ab9a91c3566cbace6c5 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -170,21 +170,22 @@ int Agc::configure(IPAContext &context, context.activeState.agc.exposureMode = exposureModeHelpers().begin()->first; ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; + context.activeState.agc.minFrameDuration = + std::chrono::microseconds(frameDurationLimits.min().get()); + context.activeState.agc.maxFrameDuration = + std::chrono::microseconds(frameDurationLimits.max().get()); AgcMeanLuminance::AgcSensorConfiguration sensorConfig; sensorConfig.lineDuration = context.configuration.sensor.lineDuration; sensorConfig.minExposureTime = context.configuration.agc.minShutterSpeed; sensorConfig.maxExposureTime = context.configuration.agc.maxShutterSpeed; - sensorConfig.minFrameDuration = - std::chrono::microseconds(frameDurationLimits.min().get()); - sensorConfig.maxFrameDuration = - std::chrono::microseconds(frameDurationLimits.max().get()); + sensorConfig.minFrameDuration = context.activeState.agc.minFrameDuration; + sensorConfig.maxFrameDuration = context.activeState.agc.maxFrameDuration; sensorConfig.minAnalogueGain = context.configuration.agc.minAnalogueGain; sensorConfig.maxAnalogueGain = context.configuration.agc.maxAnalogueGain; AgcMeanLuminance::configure(&sensorConfig, context.camHelper.get()); - - /* \todo Update AGC limits when FrameDurationLimits is passed in */ + context.activeState.agc.maxFrameDuration = sensorConfig.maxFrameDuration; resetFrameCount(); @@ -212,6 +213,25 @@ void Agc::queueRequest(IPAContext &context, const uint32_t frame, << " AGC"; } + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); + if (frameDurationLimits) { + /* Limit the control value to the limits in ControlInfo */ + ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; + int64_t minFrameDuration = + std::clamp((*frameDurationLimits).front(), + limits.min().get(), + limits.max().get()); + int64_t maxFrameDuration = + std::clamp((*frameDurationLimits).back(), + limits.min().get(), + limits.max().get()); + + agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); + agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); + } + frameContext.agc.minFrameDuration = agc.minFrameDuration; + frameContext.agc.maxFrameDuration = agc.maxFrameDuration; + /* * If the automatic exposure and gain is enabled we have no further work * to do here... @@ -375,6 +395,21 @@ void Agc::process(IPAContext &context, return; } + /* + * Update the AGC limits using the frame duration. + * + * \todo Handle ExposureTime and AnalogueGain controls to support + * manual mode. + */ + utils::Duration minExposureTime = context.configuration.agc.minShutterSpeed; + utils::Duration maxExposureTime = context.configuration.agc.maxShutterSpeed; + utils::Duration maxFrameDuration = frameContext.agc.maxFrameDuration; + + setLimits(minExposureTime, maxExposureTime, maxFrameDuration, + context.configuration.agc.minAnalogueGain, + context.configuration.agc.maxAnalogueGain, + {}); + statistics_.parseStatistics(stats); context.activeState.agc.temperatureK = estimateCCT({ { statistics_.rHist.interQuantileMean(0, 1), statistics_.gHist.interQuantileMean(0, 1), @@ -402,13 +437,23 @@ void Agc::process(IPAContext &context, dGain = std::clamp(dGain, kMinDigitalGain, kMaxDigitalGain); LOG(MaliC55Agc, Debug) - << "Divided up shutter, analogue gain and digital gain are " - << shutterTime << ", " << aGain << " and " << dGain; + << "Divided up shutter, frame duration, analogue gain and digital gain are " + << shutterTime << ", " << frameDuration << ", " << aGain + << " and " << dGain; + + /* Use the frame duration to calculate the desired vblank. */ + utils::Duration lineDuration = configuration.sensor.lineDuration; + IPACameraSensorInfo &sensorInfo = context.sensorInfo; + + frameContext.agc.vblank = (frameDuration / lineDuration) + - sensorInfo.outputSize.height; - activeState.agc.automatic.exposure = shutterTime / configuration.sensor.lineDuration; + /* Populate the active state. */ + activeState.agc.automatic.exposure = shutterTime / lineDuration; activeState.agc.automatic.sensorGain = aGain; activeState.agc.automatic.ispGain = dGain; + metadata.set(controls::FrameDuration, frameDuration.get()); metadata.set(controls::ExposureTime, currentShutter.get()); metadata.set(controls::AnalogueGain, frameContext.agc.sensorGain); metadata.set(controls::DigitalGain, frameContext.agc.ispGain); diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index ecb2f79c0dca0e41166b88f2608c22aa72adcf8a..107e55a2dab325e8d360da1281b212aa052ec724 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -53,6 +53,8 @@ struct IPAActiveState { uint32_t constraintMode; uint32_t exposureMode; uint32_t temperatureK; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; } agc; struct { @@ -66,6 +68,9 @@ struct IPAFrameContext : public FrameContext { uint32_t exposure; double sensorGain; double ispGain; + uint32_t vblank; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; } agc; struct { @@ -80,6 +85,7 @@ struct IPAContext { { } + IPACameraSensorInfo sensorInfo; IPASessionConfiguration configuration; IPAActiveState activeState; diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index 12cad7374520db8c6fa5ca233a0ef33dc7b2f287..7454e4b0d7c8703398cdb716f9a52ea551784371 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -68,7 +68,7 @@ private: void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); - void setControls(); + void setControls(unsigned int frame); std::map buffers_; @@ -126,14 +126,17 @@ int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig if (ret) return ret; + context_.sensorInfo = ipaConfig.sensorInfo; updateControls(ipaConfig.sensorInfo, ipaConfig.sensorControls, ipaControls); return 0; } -void IPAMaliC55::setControls() +void IPAMaliC55::setControls(unsigned int frame) { + IPAFrameContext &frameContext = context_.frameContexts.get(frame); IPAActiveState &activeState = context_.activeState; + uint32_t vblank = frameContext.agc.vblank; uint32_t exposure; uint32_t gain; @@ -148,6 +151,7 @@ void IPAMaliC55::setControls() ControlList ctrls(sensorControls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); + ctrls.set(V4L2_CID_VBLANK, static_cast(vblank)); setSensorControls.emit(ctrls); } @@ -368,7 +372,7 @@ void IPAMaliC55::processStats(unsigned int request, unsigned int bufferId, algo->process(context_, request, frameContext, stats, metadata); } - setControls(); + setControls(request); statsProcessed.emit(request, metadata); }