From patchwork Tue Oct 28 09:31:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24839 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCB92BE080 for ; Tue, 28 Oct 2025 09:32:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 96A92607AC; Tue, 28 Oct 2025 10:32:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nBWJxX4T"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 940DD6078C for ; Tue, 28 Oct 2025 10:32:07 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 023BB1AED; Tue, 28 Oct 2025 10:30:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643819; bh=f1o89u6CDxBXPdsmF5cTswP56ZrdhO4N8GFN23yudhs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nBWJxX4Tc/xWe8mbaaphnE+P1I3KULZMgYla7m+K3sMl0UL1YYslcTJ9l9lxJFY0h AFRFLeJ7qBYx8Ftzd6DTFM5+rzu6jh//nEG5huuQADGElfO0x/ibbLIwZsIE+NR9cj 3xDce1UqQ/X67sJAq+mhwiEU+WH+BTY264JqTTi4= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:47 +0100 Subject: [PATCH v2 01/10] ipa: camera_sensor_helper: Introduce exposureMargin() MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-1-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=10952; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=f1o89u6CDxBXPdsmF5cTswP56ZrdhO4N8GFN23yudhs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2UmaJ3xkBUY+rbq9+3+XPWhBT9vYPyEaH36 ffqn+GoceqJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlAAKCRByNAaPFqFW PJzREADD0byAlRv4NwvJ7Yxx1UKyaFilxaAdffovMWqO7lM0MUMuJNyokDMN1zxKrVGBu9V9HCt +sSn0UWQNqXf2RdXo1TcHESjWShMgYh2tnojafpkb6AFdKqxh9zyxn8ovshi9eVzakGw/QNvxji gr0rymhEZUH8ZNQQ+jZbVhQQh8VW+dRG2UgSF7OdpZZYtn9QyE5mnBJe2Fu/44obDDAr1d0mAel CXyOoAPYSHNJW7hVk6NEeI0aK/Orte6t2I1SrorYCkfPCiqhuY1xRzoGrxArW+MI46i5VO2PLpC Uulgksuw6cwitUVBD9hPkg6MtYQfq73nwHjTZf0prZgstwgzmuYHf4wcQFsQRWUmvIT6MCRYhdH 6W0ZGrsprcgvOxK3RgWNmmd82m8P8Z47YGipL6rMkShC2LwBCWu6Wcp/5TsikwdTlwK4ImxJ7Jj UtLYr+iQy6Mxno/U8HMcxA1h2PH6dTbuDJZYN5owB9CH+BnNmoGyV/P8VSPSuJreMbXBqVl0bef C3odpikrGHHhRGSbZQKV1aPcXxbAQH37VktNeppgyNsMCHdGKvXf1L12yuG9iewuWe6XwYvK6j7 PNyKkSZEc1PtDQGPPTYBgMKfK76cbw9jeU3loiMcYbbD5khyWsFAllMHqnEMHE/UfCU0Jynt00S hqVDgA41rJcb86w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There is a margin between the maximum achievable exposure time and the frame length. The value is sensor-dependent and should be used to adjust the maximum achievable exposure time whenever the frame length changes. Introduce in the CameraSensorHelper an exposureMargin() function that returns a std::optional<> with the sensor-specific margin. Use a std::optional<> to easily identify CameraSensorHelper implementations that do not populate exposureMargin_ and warn about possibly sub-optimal results in exposure time calculations. This feature is already implemented in the RPi CamHelper class hierarchy with the name of frameIntegrationDiff. Populate the CameraSensorHelper instances with an exposureMargin_. The value of the exposure margin comes from the mainline driver version of each sensor, and it has been compared with the frameIntegrationDiff_ value in the rpi camera helpers. The only outliner is the Vd56g3 sensor, whose exposure margin from the driver differs from the RPi camera helper. Signed-off-by: Jacopo Mondi --- src/ipa/libipa/camera_sensor_helper.cpp | 72 +++++++++++++++++++++++++++++++++ src/ipa/libipa/camera_sensor_helper.h | 2 + 2 files changed, 74 insertions(+) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index ef3bd0d621e6bead2b3f8a7fe6988f642e4791f7..6d14405657903704d6ffee72eb336ec8b78db1cd 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -154,6 +154,18 @@ double CameraSensorHelper::quantizeGain(double _gain, double *quantizationGain) return g; } +/** + * \fn CameraSensorHelper::exposureMargin() + * \brief Fetch the integration time margin + * + * This function returns the number of lines that represent the minimum + * difference between the frame length and the maximum achievable integration + * time. If it is unknown an empty optional is returned. + * + * \return The minimum difference between the exposure time and the frame + * length in lines, or an empty optional + */ + /** * \struct CameraSensorHelper::AnalogueGainLinear * \brief Analogue gain constants for the linear gain model @@ -229,6 +241,13 @@ double CameraSensorHelper::quantizeGain(double _gain, double *quantizationGain) * sensor specific. Use this variable to store the values at init time. */ +/** + * \var CameraSensorHelper::exposureMargin_ + * \brief The smallest margin between the integration time and the frame lenght + * in lines + * \sa CameraSensorHelper::exposureMargin() + */ + /** * \class CameraSensorHelperFactoryBase * \brief Base class for camera sensor helper factories @@ -385,6 +404,7 @@ public: { /* Power-on default value: 168 at 12bits. */ blackLevel_ = 2688; + exposureMargin_ = 4; } uint32_t gainCode(double gain) const override @@ -474,6 +494,11 @@ REGISTER_CAMERA_SENSOR_HELPER("ar0144", CameraSensorHelperAr0144) class CameraSensorHelperAr0521 : public CameraSensorHelper { public: + CameraSensorHelperAr0521() + { + exposureMargin_ = 4; + } + uint32_t gainCode(double gain) const override { gain = std::clamp(gain, 1.0, 15.5); @@ -504,6 +529,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 100, 0, 0, 1024 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("gc05a2", CameraSensorHelperGc05a2) @@ -516,6 +542,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 100, 0, 0, 1024 }; + exposureMargin_ = 16; } }; REGISTER_CAMERA_SENSOR_HELPER("gc08a3", CameraSensorHelperGc08a3) @@ -526,6 +553,7 @@ public: CameraSensorHelperHm1246() { gain_ = AnalogueGainLinear{ 1, 16, 0, 16 }; + exposureMargin_ = 2; } }; REGISTER_CAMERA_SENSOR_HELPER("hm1246", CameraSensorHelperHm1246) @@ -538,6 +566,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 512, -1, 512 }; + exposureMargin_ = 10; } }; REGISTER_CAMERA_SENSOR_HELPER("imx214", CameraSensorHelperImx214) @@ -550,6 +579,7 @@ public: /* From datasheet: 64 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 256, -1, 256 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("imx219", CameraSensorHelperImx219) @@ -562,6 +592,7 @@ public: /* From datasheet: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 512, -1, 512 }; + exposureMargin_ = 10; } }; REGISTER_CAMERA_SENSOR_HELPER("imx258", CameraSensorHelperImx258) @@ -574,6 +605,7 @@ public: /* From datasheet: 0x32 at 10bits. */ blackLevel_ = 3200; gain_ = AnalogueGainLinear{ 0, 2048, -1, 2048 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("imx283", CameraSensorHelperImx283) @@ -586,6 +618,7 @@ public: /* From datasheet: 0xf0 at 12bits. */ blackLevel_ = 3840; gain_ = AnalogueGainExp{ 1.0, expGainDb(0.3) }; + exposureMargin_ = 2; } }; REGISTER_CAMERA_SENSOR_HELPER("imx290", CameraSensorHelperImx290) @@ -596,6 +629,11 @@ public: CameraSensorHelperImx296() { gain_ = AnalogueGainExp{ 1.0, expGainDb(0.1) }; + /* + * The driver doesn't apply any margin. Use the value + * in RPi's CamHelper. + */ + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("imx296", CameraSensorHelperImx296) @@ -613,6 +651,7 @@ public: /* From datasheet: 0x32 at 10bits. */ blackLevel_ = 3200; gain_ = AnalogueGainExp{ 1.0, expGainDb(0.3) }; + exposureMargin_ = 9; } }; REGISTER_CAMERA_SENSOR_HELPER("imx335", CameraSensorHelperImx335) @@ -623,6 +662,7 @@ public: CameraSensorHelperImx415() { gain_ = AnalogueGainExp{ 1.0, expGainDb(0.3) }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("imx415", CameraSensorHelperImx415) @@ -638,6 +678,7 @@ public: CameraSensorHelperImx477() { gain_ = AnalogueGainLinear{ 0, 1024, -1, 1024 }; + exposureMargin_ = 22; } }; REGISTER_CAMERA_SENSOR_HELPER("imx477", CameraSensorHelperImx477) @@ -652,6 +693,7 @@ public: * This has been validated with some empirical testing only. */ gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov2685", CameraSensorHelperOv2685) @@ -662,6 +704,7 @@ public: CameraSensorHelperOv2740() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov2740", CameraSensorHelperOv2740) @@ -674,6 +717,7 @@ public: /* From datasheet: 0x40 at 12bits. */ blackLevel_ = 1024; gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov4689", CameraSensorHelperOv4689) @@ -686,6 +730,14 @@ public: /* From datasheet: 0x10 at 10bits. */ blackLevel_ = 1024; gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + /* + * Very convoluted in the driver that however applies a margin + * of 4 lines when setting vts. + * + * cap_vts = cap_shutter + 4; + * ret = ov5640_set_vts(sensor, cap_vts); + */ + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5640", CameraSensorHelperOv5640) @@ -696,6 +748,7 @@ public: CameraSensorHelperOv5647() { gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5647", CameraSensorHelperOv5647) @@ -706,6 +759,7 @@ public: CameraSensorHelperOv5670() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5670", CameraSensorHelperOv5670) @@ -718,6 +772,7 @@ public: /* From Linux kernel driver: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5675", CameraSensorHelperOv5675) @@ -728,6 +783,7 @@ public: CameraSensorHelperOv5693() { gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov5693", CameraSensorHelperOv5693) @@ -738,6 +794,7 @@ public: CameraSensorHelperOv64a40() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 32; } }; REGISTER_CAMERA_SENSOR_HELPER("ov64a40", CameraSensorHelperOv64a40) @@ -754,6 +811,7 @@ public: * See: https://patchwork.linuxtv.org/project/linux-media/patch/20221106171129.166892-2-nicholas@rothemail.net/#142267 */ gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 4; } }; REGISTER_CAMERA_SENSOR_HELPER("ov8858", CameraSensorHelperOv8858) @@ -764,6 +822,7 @@ public: CameraSensorHelperOv8865() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov8865", CameraSensorHelperOv8865) @@ -774,6 +833,7 @@ public: CameraSensorHelperOv13858() { gain_ = AnalogueGainLinear{ 1, 0, 0, 128 }; + exposureMargin_ = 8; } }; REGISTER_CAMERA_SENSOR_HELPER("ov13858", CameraSensorHelperOv13858) @@ -786,6 +846,7 @@ public: /* From datasheet: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 32, -1, 32 }; + exposureMargin_ = 64; } }; REGISTER_CAMERA_SENSOR_HELPER("vd55g1", CameraSensorHelperVd55g1) @@ -798,6 +859,17 @@ public: /* From datasheet: 0x40 at 10bits. */ blackLevel_ = 4096; gain_ = AnalogueGainLinear{ 0, 32, -1, 32 }; + /* + * The mainline driver version has + * #define VD56G3_EXPOSURE_MARGIN 75 + * while the frameIntegrationDiff value in the RPi cam + * helper for this sensor has + * static constexpr int frameIntegrationDiff = 61; + * + * Trust the driver and report 75 which is also larger and + * hence "safer" + */ + exposureMargin_ = 75; } }; REGISTER_CAMERA_SENSOR_HELPER("vd56g3", CameraSensorHelperVd56g3) diff --git a/src/ipa/libipa/camera_sensor_helper.h b/src/ipa/libipa/camera_sensor_helper.h index bd3d0beec77f006b68fecf45eee850c5283fefa5..57dd4c37ed76733710f1673a98e82ae47475baa7 100644 --- a/src/ipa/libipa/camera_sensor_helper.h +++ b/src/ipa/libipa/camera_sensor_helper.h @@ -30,6 +30,7 @@ public: virtual uint32_t gainCode(double gain) const; virtual double gain(uint32_t gainCode) const; double quantizeGain(double gain, double *quantizationGain) const; + std::optional exposureMargin() const { return exposureMargin_; } protected: struct AnalogueGainLinear { @@ -46,6 +47,7 @@ protected: std::optional blackLevel_; std::variant gain_; + std::optional exposureMargin_; private: LIBCAMERA_DISABLE_COPY_AND_MOVE(CameraSensorHelper)