From patchwork Tue Oct 28 09:31:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 24847 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BC56BBE080 for ; Tue, 28 Oct 2025 09:32:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 48622607AC; Tue, 28 Oct 2025 10:32:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KS4cXdge"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9FABA607BC for ; Tue, 28 Oct 2025 10:32:12 +0100 (CET) Received: from [192.168.0.172] (mob-5-90-58-13.net.vodafone.it [5.90.58.13]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E56281AED; Tue, 28 Oct 2025 10:30:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761643824; bh=hv6EQVhzC+oNcaqjzg+vZ5gTG1/2NINcRSZSFYP4LNQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=KS4cXdgeN4+YeuxIn3/JmYo5LXr5bN0gQH10clDTENhawtWXf1ragUm2B30eeYV9H aHgNcCr64NlhSHbxv9smtHQ8Dpj8XRDWHlYJ9CgrWW0QbwAoZUrc2wcw/0qsesKIg1 U6AujbKaz74cCA6fPyMFrk1R+Haj5RA+BC4D2WTY= From: Jacopo Mondi Date: Tue, 28 Oct 2025 10:31:55 +0100 Subject: [PATCH v2 09/10] ipa: libipa: agc: Calculate frame duration in helper MIME-Version: 1.0 Message-Id: <20251028-exposure-limits-v2-9-a8b5a318323e@ideasonboard.com> References: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> In-Reply-To: <20251028-exposure-limits-v2-0-a8b5a318323e@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , Robert Mader , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12672; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=hv6EQVhzC+oNcaqjzg+vZ5gTG1/2NINcRSZSFYP4LNQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpAI2Ww/01hoPy0MQRh+fGyLkAMl8eaD8YXTAnf jUyWyQniV+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaQCNlgAKCRByNAaPFqFW PKbkEACYrJuWEGDzIDx+8oyZ7UrDYs/IF/5R/7cVYujZTV8pClS/Gvwgp6GTUsHs8CR3t8jkLLE CofcFwQcKUurrXpKMf+xuY5TnBqlnn8gJwGO6ppzSgTUUlgQZxnav/G8qvkGnmHo0TcwiEoMgx1 YXXpK4PiaGd4olQmkJXU3/jHb0MBM5M1LWSyPtNP7N2xRyI9hm85O0Y0mJQ+ANzP2HUqDMOgRZw hYsgRt/ZeBU5ttluMFljVZEB0kCnNu0TIsI2llEohNnY/nR+xoUS6Esvsodt8jNK/tALGAsw2+o 9laV9fc61hrXLFMbjMmRcJGK18M0ibdnTsEeEiYvMZnaHM0ky4c0aAcZclv6YT/Vw0dq7OQtepx 9KJ/zx6/2gSSb5d7udRayB2hNFBuWOKpbH3A+/3Um0/6KMMbx6ysV3AAFVecsNIDyOHe4qPn2Nv yw22kOR8lUKtq3vcrdKUxT7RHYaF8bPNT1Cg7eHBkLZALGndl1FOzN12VsD2EWjBVzvnjq83tnJ jQ0j8M7H3OY4qp0jPoGb9n4OONo8M5gGriGp4dfKh4Me3/dcxl8iWbO+UYUHSPCuKBkeEAZCGmG oF5npg+SnEq+o4Sjm85idkU2k0Q1iENfi2ssf5gGqdSabfeoW6yvADxaI4tLcjuFwGxiqlJcEOX sqOry4PNE0gn0NA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The RkISP1 IPA is the only IPA that derives from AgcMeanLuminance that populates the FrameDuration control. The way frame duration is calculated is by using the shutter time as returned by the Agc helpers and use that value to populate the FrameDuration metadata and calculate the vblank. This is however not correct as the exposure time shall be shorter than the frame duration by a sensor-specific margin. The AGC helpers are in charge of calculating the desired exposure and split it between shutter time and gains. They already use the exposure margin to limit the maximum shutter time to the maximum frame duration. Let the helpers calculate the frame duration and return it with shutter time and gains from calculateNewEv(). Also update the Mali C55 and IPU3 IPAs, however they do not currently handle the frame duration and never change vblank so they don't report the FrameDuration metadata (Mali C55) or return a fixed value (IPU3). Signed-off-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/agc.cpp | 12 +++++++----- src/ipa/libipa/agc_mean_luminance.cpp | 6 +++--- src/ipa/libipa/agc_mean_luminance.h | 2 +- src/ipa/libipa/exposure_mode_helper.cpp | 33 +++++++++++++++++++++++---------- src/ipa/libipa/exposure_mode_helper.h | 5 ++++- src/ipa/mali-c55/algorithms/agc.cpp | 3 ++- src/ipa/rkisp1/algorithms/agc.cpp | 11 ++++++----- 7 files changed, 46 insertions(+), 26 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 51aa4e51841ca8c6363c0211819a7d14cc3baec6..34cb31a3ef4ff8a4774351e1d1a8b5a46aca33cc 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -236,8 +236,9 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, utils::Duration effectiveExposureValue = exposureTime * analogueGain; utils::Duration newExposureTime; + utils::Duration frameDuration; double aGain, qGain, dGain; - std::tie(newExposureTime, aGain, qGain, dGain) = + std::tie(newExposureTime, frameDuration, aGain, qGain, dGain) = calculateNewEv(context.activeState.agc.constraintMode, context.activeState.agc.exposureMode, hist, effectiveExposureValue); @@ -254,12 +255,13 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, metadata.set(controls::AnalogueGain, frameContext.sensor.gain); metadata.set(controls::ExposureTime, exposureTime.get()); - /* \todo Use VBlank value calculated from each frame exposure. */ + /* \todo Use frameDuration to calculate the right vblank. */ + uint32_t vTotal = context.configuration.sensor.size.height + context.configuration.sensor.defVBlank; - utils::Duration frameDuration = context.configuration.sensor.lineDuration - * vTotal; - metadata.set(controls::FrameDuration, frameDuration.get()); + utils::Duration defFrameDuration = context.configuration.sensor.lineDuration + * vTotal; + metadata.set(controls::FrameDuration, defFrameDuration.get()); } REGISTER_IPA_ALGORITHM(Agc, "Agc") diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 36bdf76dca45c837d5ef4eb03244a43d1262c675..3722602921d6f1716f4cfedcae84023e4bea02a6 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -646,10 +646,10 @@ utils::Duration AgcMeanLuminance::filterExposure(utils::Duration exposureValue) * exposure value is filtered to prevent rapid changes from frame to frame, and * divided into exposure time, analogue, quantization and digital gain. * - * \return Tuple of exposure time, analogue gain, quantization gain and digital - * gain + * \return Tuple of exposure time, frame duration analogue gain, quantization + * gain and digital gain */ -std::tuple +std::tuple AgcMeanLuminance::calculateNewEv(uint32_t constraintModeIndex, uint32_t exposureModeIndex, const Histogram &yHist, diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index b00772760035a477152d638809ac71fcbc66cb68..71c4951080c728e87530caf189fbd52f92b3d033 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -80,7 +80,7 @@ public: return controls_; } - std::tuple + std::tuple calculateNewEv(uint32_t constraintModeIndex, uint32_t exposureModeIndex, const Histogram &yHist, utils::Duration effectiveExposureValue); diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp index 46ba535c67d92bd384095b250b925cdc0da40b38..e9a37a3771cdfbb35ad4c8a373d3189f8a468c7b 100644 --- a/src/ipa/libipa/exposure_mode_helper.cpp +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -100,9 +100,10 @@ void ExposureModeHelper::setMaxExposure(utils::Duration minExposureTime, << "Exposure margin not known. Default to 4"; margin = { 4 }; } + exposureMargin_ = margin.value() * lineDuration_; maxExposureTime_ = minExposureTime != maxExposureTime - ? maxFrameDuration - margin.value() * lineDuration_ + ? maxFrameDuration - exposureMargin_ : minExposureTime; } @@ -152,6 +153,7 @@ void ExposureModeHelper::configure(utils::Duration lineDuration, setMaxExposure(minExposureTime, maxExposureTime, frameDuration); maxFrameDuration_ = *maxFrameDuration = frameDuration; + minFrameDuration_ = minFrameDuration; } /** @@ -207,6 +209,17 @@ double ExposureModeHelper::clampGain(double gain, double *quantizationGain) cons return sensorHelper_->quantizeGain(clamped, quantizationGain); } +utils::Duration ExposureModeHelper::frameDurationFromExposure(utils::Duration exposureTime) const +{ + /* + * The maximum exposure value has already been clamped to the maximum + * frame duration. Re-apply the exposure margin and make sure we don't + * go below the minium frame duration. + */ + utils::Duration frameDuration = exposureTime + exposureMargin_; + return std::max(frameDuration, minFrameDuration_); +} + /** * \brief Split exposure into exposure time and gain * \param[in] exposure Exposure value @@ -244,7 +257,7 @@ double ExposureModeHelper::clampGain(double gain, double *quantizationGain) cons * \return Tuple of exposure time, analogue gain, quantization gain and digital * gain */ -std::tuple +std::tuple ExposureModeHelper::splitExposure(utils::Duration exposure) const { ASSERT(maxExposureTime_); @@ -266,8 +279,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(minGain_, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } double stageGain = clampGain(1.0); @@ -292,8 +305,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(exposure / exposureTime, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } /* Clamp the exposureTime to stageExposureTime and regulate gain. */ @@ -302,8 +315,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(exposure / exposureTime, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } lastStageGain = stageGain; @@ -320,8 +333,8 @@ ExposureModeHelper::splitExposure(utils::Duration exposure) const gain = clampGain(exposure / exposureTime, &quantGain2); quantGain *= quantGain2; - return { exposureTime, gain, quantGain, - exposure / (exposureTime * gain * quantGain) }; + return { exposureTime, frameDurationFromExposure(exposureTime), + gain, quantGain, exposure / (exposureTime * gain * quantGain) }; } /** diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h index 492d2787f75945fb487b351a60f13756caec66e0..8dc94cf808945e0ff3cea080046b1cafd453a014 100644 --- a/src/ipa/libipa/exposure_mode_helper.h +++ b/src/ipa/libipa/exposure_mode_helper.h @@ -34,7 +34,7 @@ public: utils::Duration maxFrameDuration, double minGain, double maxGain); - std::tuple + std::tuple splitExposure(utils::Duration exposure) const; utils::Duration minExposureTime() const { return minExposureTime_; } @@ -50,6 +50,7 @@ private: utils::Duration clampExposureTime(utils::Duration exposureTime, double *quantizationGain = nullptr) const; double clampGain(double gain, double *quantizationGain = nullptr) const; + utils::Duration frameDurationFromExposure(utils::Duration exposureTime) const; std::vector exposureTimes_; std::vector gains_; @@ -58,6 +59,8 @@ private: utils::Duration minExposureTime_; utils::Duration maxExposureTime_; utils::Duration maxFrameDuration_; + utils::Duration minFrameDuration_; + utils::Duration exposureMargin_; double minGain_; double maxGain_; const CameraSensorHelper *sensorHelper_; diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 4e2fa4386be621c63d992b75d3d9efc525c3d69c..6ecb18d959c7f978966d619beb43e1c2cbfeec5f 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -391,9 +391,10 @@ void Agc::process(IPAContext &context, utils::Duration currentShutter = exposure * configuration.sensor.lineDuration; utils::Duration effectiveExposureValue = currentShutter * totalGain; + utils::Duration frameDuration; utils::Duration shutterTime; double aGain, qGain, dGain; - std::tie(shutterTime, aGain, qGain, dGain) = + std::tie(shutterTime, frameDuration, aGain, qGain, dGain) = calculateNewEv(activeState.agc.constraintMode, activeState.agc.exposureMode, statistics_.yHist, effectiveExposureValue); diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index a983efbecbd84fa9e4af8b9642fbdda124ba0b9f..c8a34c3d1bf2ed71b9cf203e48c4b42910eea797 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -625,15 +625,17 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, setExposureCompensation(pow(2.0, frameContext.agc.exposureValue)); utils::Duration newExposureTime; + utils::Duration frameDuration; double aGain, qGain, dGain; - std::tie(newExposureTime, aGain, qGain, dGain) = + std::tie(newExposureTime, frameDuration, aGain, qGain, dGain) = calculateNewEv(frameContext.agc.constraintMode, frameContext.agc.exposureMode, hist, effectiveExposureValue); LOG(RkISP1Agc, Debug) - << "Divided up exposure time, analogue gain, quantization gain" - << " and digital gain are " << newExposureTime << ", " << aGain + << "Divided up exposure time, frame duration, analogue gain," + << "quantization gain and digital gain are " + << newExposureTime << ", " << frameDuration << ", " << aGain << ", " << qGain << " and " << dGain; IPAActiveState &activeState = context.activeState; @@ -646,8 +648,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * Expand the target frame duration so that we do not run faster than * the minimum frame duration when we have short exposures. */ - processFrameDuration(context, frameContext, - std::max(frameContext.agc.minFrameDuration, newExposureTime)); + processFrameDuration(context, frameContext, frameDuration); fillMetadata(context, frameContext, metadata); expMeans_ = {};