From patchwork Mon Aug 24 09:14:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28077 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 60260C3339 for ; Mon, 24 Aug 2026 09:15:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E22A16844A; Mon, 24 Aug 2026 11:15:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Dc6I2rhK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6BADB683F4 for ; Mon, 24 Aug 2026 11:14:23 +0200 (CEST) Received: from pb-laptop.local (185.221.143.32.nat.pool.zt.hu [185.221.143.32]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 13A87C24; Mon, 24 Aug 2026 11:12:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787562779; bh=ZTRnuRb0dMuVw8TJnZLY8aQY3VfNeTveyWdFK8PvmyM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Dc6I2rhK1bzpQtKYp6T0AtiiASJY5Zila01ffTbiEx8Bs03iTQ0SgU3/lQ/vI8v0v rYTxnRuYBbYCjKnEM8BXUL3ZvLghdG/7kKuL+GHQUC2ZymQVVABdy3TA8l3Ki/MpFo lXmJAnS1VIE9fpDxkE1Oh4w3+z3YbW2dZoPUTcmY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [PATCH v8 45/47] ipa: libipa: agc: Work without `CameraSensorHelper` Date: Mon, 24 Aug 2026 11:14:04 +0200 Message-ID: <20260824091407.502020-46-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260824091407.502020-1-barnabas.pocze@ideasonboard.com> References: <20260824091407.502020-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" `AgcMeanLuminance` can operate without a `CameraSensorHelper`, but in that case it assumes an "ideal" gain model, where gains are truly continuous, and a gain of x will apply a gain of exactly x to the result. This is not a good fit when only gain codes are available. So now that the agc algorithm from the simple ipa has been extracted (AgcMSV), let's use that in `AgcAlgorithm` to be able to always provide some level of automatic exposure/gain control. Even though the main use case for operating without a known gain model is empirically determining the gain model using the manual controls. Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/agc.cpp | 4 +- src/ipa/libipa/agc.cpp | 202 +++++++++++++++++++--------- src/ipa/libipa/agc.h | 10 +- src/ipa/mali-c55/algorithms/agc.cpp | 4 +- src/ipa/rkisp1/algorithms/agc.cpp | 4 +- 5 files changed, 148 insertions(+), 76 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 29a562d667..22846514be 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -66,8 +66,7 @@ Agc::Agc() */ int Agc::init(IPAContext &context, const ValueNode &tuningData) { - return agc_.init(tuningData, { - .sensor = context.camHelper.get(), + return agc_.init(tuningData, context.camHelper.get(), { .sensorInfo = context.sensorInfo, .sensorControls = context.sensorControls, .ctrlMap = context.ctrlMap, @@ -88,7 +87,6 @@ int Agc::configure(IPAContext &context, bdsGrid_ = context.configuration.grid.bdsGrid; return agc_.configure(context.configuration.agc, context.activeState.agc, { - .sensor = context.camHelper.get(), .sensorInfo = context.sensorInfo, .sensorControls = context.sensorControls, .ctrlMap = context.ctrlMap, diff --git a/src/ipa/libipa/agc.cpp b/src/ipa/libipa/agc.cpp index 1c096c984d..9f37e1b8ee 100644 --- a/src/ipa/libipa/agc.cpp +++ b/src/ipa/libipa/agc.cpp @@ -13,11 +13,13 @@ #include #include #include +#include #include #include #include +#include #include #include @@ -80,6 +82,9 @@ namespace agc { * \var Session::maxAnalogueGain * \brief Maximum analogue gain for the streaming session * + * \var Session::defAnalogueGain + * \brief Default analogue gain of the configured sensor + * * \var Session::minFrameDuration * \brief Minimum frame duration for the streaming session * @@ -222,10 +227,9 @@ namespace agc { * * The AgcAlgorithm class can be used to implement automatic exposure/gain * control in an IPA module, conforming to the prescribed Algorithm interface. - * Internally AgcMeanLuminance is used, this class merely concerns itself with - * processing the sensor properties, establishing limits, managing controls, - * providing the resulting metadata, and driving the actual algorithm in - * process(). + * Internally AgcMeanLuminance or AgcMSV is used, this class merely concerns + * itself with processing the sensor properties, establishing limits, managing + * controls, and providing the resulting metadata. * * Users should compose the agc::Session, agc::ActiveState, and agc::FrameContext * structures into their session configuration, active state, and frame contexts, @@ -241,9 +245,6 @@ namespace agc { * \struct AgcAlgorithm::ConfigurationParams * \brief Parameters for AgcAlgorithm::configure() * - * \var AgcAlgorithm::ConfigurationParams::sensor - * \brief CameraSensorHelper for the sensor - * * \var AgcAlgorithm::ConfigurationParams::sensorInfo * \brief Current configuration of the sensor * @@ -288,24 +289,35 @@ namespace agc { /** * \brief Load tuning data and configure * \param[in] tuningData The tuning data + * \param[in] sensor The camera sensor helper * \param[in] config The algorithm configuration * * This function loads the tuning data and configures the algorithm as if * by a call to configure(), in order to provide the available controls * in ConfigurationParams::ctrlMap. * - * The tuning data format is that of AgcMeanLuminance. Refer to - * AgcMeanLuminance::parseTuningData() for more details. + * If a CameraSensorHelper is provided in \a sensor, an instance of + * AgcMeanLuminance is instantiated and the tuning data is loaded by + * calling AgcMeanLuminance::parseTuningData(). Otherwise an instance + * of AgcMSV is instantiated, and in this case the tuning data is ignored. * * \return 0 on success, or a negative error code * * \sa Algorithm::init() */ -int AgcAlgorithm::init(const ValueNode &tuningData, const ConfigurationParams &config) +int AgcAlgorithm::init(const ValueNode &tuningData, CameraSensorHelper *sensor, + const ConfigurationParams &config) { - int ret = impl_.parseTuningData(tuningData); - if (ret) - return ret; + if (sensor) { + auto &impl = impl_.emplace(); + int ret = impl.parseTuningData(tuningData); + if (ret) + return ret; + } else { + impl_.emplace(); + } + + sensor_ = sensor; /* * The purpose of this `configure()` is merely to provide the @@ -361,10 +373,14 @@ int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, int32_t defExposure = v4l2Exposure.def().get(); /* Compute the analogue gain limits. */ + const auto extractGain = [&](const ControlValue &v) { + auto gainCode = v.get(); + return sensor_ ? sensor_->gain(gainCode) : gainCode; + }; const ControlInfo &v4l2Gain = config.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; - float minGain = config.sensor->gain(v4l2Gain.min().get()); - float maxGain = config.sensor->gain(v4l2Gain.max().get()); - float defGain = config.sensor->gain(v4l2Gain.def().get()); + float minGain = extractGain(v4l2Gain.min()); + float maxGain = extractGain(v4l2Gain.max()); + float defGain = extractGain(v4l2Gain.def()); LOG(Agc, Debug) << "exposure: [" << minExposure << ',' << maxExposure << "], " @@ -403,28 +419,21 @@ int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, session.maxExposureTime = maxExposure * session.lineDuration; session.minAnalogueGain = minGain; session.maxAnalogueGain = maxGain; + session.defAnalogueGain = defGain; session.minFrameDuration = std::chrono::microseconds(frameDurations[0]); session.maxFrameDuration = std::chrono::microseconds(frameDurations[1]); - impl_.configure(session.lineDuration, config.sensor); - impl_.resetFrameCount(); - /* Configure the default exposure and gain. */ state = {}; state.automatic.gain = session.minAnalogueGain; state.automatic.exposure = defExposure; state.automatic.quantizationGain = 1; state.automatic.digitalGain = 1; - state.automatic.yTarget = impl_.effectiveYTarget(0, 1); state.manual.gain = state.automatic.gain; state.manual.exposure = state.automatic.exposure; state.autoExposureEnabled = session.autoAllowed; state.autoGainEnabled = session.autoAllowed; state.exposureValue = 0; - state.constraintMode = - static_cast(impl_.constraintModes().begin()->first); - state.exposureMode = - static_cast(impl_.exposureModeHelpers().begin()->first); state.minFrameDuration = session.minFrameDuration; state.maxFrameDuration = session.maxFrameDuration; @@ -475,19 +484,52 @@ int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, add(controls::AnalogueGainMode, controls::AnalogueGainModeAuto, controls::AnalogueGainModeManual); - if (session.autoAllowed) { - config.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + std::visit(utils::overloaded{ + [&](AgcMSV &) { + /* No constraint/exposure mode support. */ + state.constraintMode = controls::AeConstraintModeEnum::ConstraintNormal; + state.exposureMode = controls::AeExposureModeEnum::ExposureNormal; - std::vector options; - for (const auto &[id, _] : impl_.constraintModes()) - options.emplace_back(id); - config.ctrlMap[&controls::AeConstraintMode] = ControlInfo(options); + state.automatic.yTarget = 0; /* Not supported. */ - options.clear(); - for (const auto &[id, _] : impl_.exposureModeHelpers()) - options.emplace_back(id); - config.ctrlMap[&controls::AeExposureMode] = ControlInfo(options); - } + if (!session.autoAllowed) + return; + + config.ctrlMap[&controls::AeConstraintMode] = ControlInfo( + std::array{ ControlValue(state.constraintMode) } + ); + + config.ctrlMap[&controls::AeExposureMode] = ControlInfo( + std::array{ ControlValue(state.exposureMode) } + ); + }, + [&](AgcMeanLuminance &impl) { + state.constraintMode = + static_cast(impl.constraintModes().begin()->first); + state.exposureMode = + static_cast(impl.exposureModeHelpers().begin()->first); + + state.automatic.yTarget = impl.effectiveYTarget(0, 1); + + impl.configure(session.lineDuration, sensor_); + impl.resetFrameCount(); + + if (!session.autoAllowed) + return; + + config.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + + std::vector options; + for (const auto &[id, _] : impl.constraintModes()) + options.emplace_back(id); + config.ctrlMap[&controls::AeConstraintMode] = ControlInfo(options); + + options.clear(); + for (const auto &[id, _] : impl.exposureModeHelpers()) + options.emplace_back(id); + config.ctrlMap[&controls::AeExposureMode] = ControlInfo(options); + }, + }, impl_); return 0; } @@ -718,36 +760,68 @@ void AgcAlgorithm::process(const agc::Session &session, agc::ActiveState &state, maxAnalogueGain = frameContext.gain; } - /* - * The Agc algorithm needs to know the effective exposure value that was - * applied to the sensor when the statistics were collected. - */ - utils::Duration effectiveExposureValue = - lineDuration * params->exposure * params->gain; - - impl_.setLimits(minExposureTime, maxExposureTime, - minAnalogueGain, maxAnalogueGain, - std::move(params->additionalConstraints)); - - const auto &newEv = impl_.calculateNewEv({ - .traits = params->traits, - .yHist = params->yHist, - .effectiveExposureValue = effectiveExposureValue, - .constraintModeIndex = frameContext.constraintMode, - .exposureModeIndex = frameContext.exposureMode, - .lux = params->lux, - .exposureCompensation = std::pow(2.0, frameContext.exposureValue), - }); - - /* Update the estimated exposure and gain. */ - state.automatic.exposure = newEv.exposureTime / lineDuration; - state.automatic.gain = newEv.analogueGain; - state.automatic.quantizationGain = newEv.quantizationGain; - state.automatic.digitalGain = newEv.digitalGain; - state.automatic.yTarget = newEv.yTarget; + std::visit(utils::overloaded{ + [&](AgcMSV& impl) { + impl.setLimits({ + .exposure = { + static_cast(minExposureTime / lineDuration), + static_cast(maxExposureTime / lineDuration), + }, + .gain = { + minAnalogueGain, + maxAnalogueGain, + }, + /* gain codes -> step size of 1 */ + .gainMinStep = 1, + /* assume default gain is close to 1.0 */ + .gain1 = session.defAnalogueGain, + }); + + const auto& newEv = impl.calculateNewEv({ + .yHist = params->yHist, + .exposure = params->exposure, + .gain = params->gain, + }); + + state.automatic.exposure = newEv.exposure; + state.automatic.gain = newEv.analogueGain; + }, + [&](AgcMeanLuminance& impl) { + /* + * The Agc algorithm needs to know the effective exposure + * value that was applied to the sensor when the statistics + * were collected. + */ + utils::Duration effectiveExposureValue = + lineDuration * params->exposure * params->gain; + + impl.setLimits(minExposureTime, maxExposureTime, + minAnalogueGain, maxAnalogueGain, + std::move(params->additionalConstraints)); + + const auto &newEv = impl.calculateNewEv({ + .traits = params->traits, + .yHist = params->yHist, + .effectiveExposureValue = effectiveExposureValue, + .constraintModeIndex = frameContext.constraintMode, + .exposureModeIndex = frameContext.exposureMode, + .lux = params->lux, + .exposureCompensation = std::pow(2.0, frameContext.exposureValue), + }); + + /* Update the estimated exposure and gain. */ + state.automatic.exposure = newEv.exposureTime / lineDuration; + state.automatic.gain = newEv.analogueGain; + state.automatic.quantizationGain = newEv.quantizationGain; + state.automatic.digitalGain = newEv.digitalGain; + state.automatic.yTarget = newEv.yTarget; + }, + }, impl_); + + const utils::Duration newExposureTime = state.automatic.exposure * lineDuration; LOG(Agc, Debug) - << "exposure-time: " << newEv.exposureTime << ", " + << "exposure-time: " << newExposureTime << ", " << "analogue-gain: " << state.automatic.gain << ", " << "quantization-gain: " << state.automatic.quantizationGain << ", " << "digital-gain: " << state.automatic.digitalGain; @@ -757,7 +831,7 @@ void AgcAlgorithm::process(const agc::Session &session, agc::ActiveState &state, * the minimum frame duration when we have short exposures. */ processFrameDuration(session, frameContext, - std::max(frameContext.minFrameDuration, newEv.exposureTime)); + std::max(frameContext.minFrameDuration, newExposureTime)); fillMetadata(session, frameContext, metadata); } diff --git a/src/ipa/libipa/agc.h b/src/ipa/libipa/agc.h index 7c5a7c3518..e820077e7a 100644 --- a/src/ipa/libipa/agc.h +++ b/src/ipa/libipa/agc.h @@ -11,6 +11,7 @@ #include #include #include +#include #include @@ -21,6 +22,7 @@ #include #include "agc_mean_luminance.h" +#include "agc_msv.h" #include "camera_sensor_helper.h" namespace libcamera { @@ -38,6 +40,7 @@ struct Session { utils::Duration maxExposureTime; double minAnalogueGain; double maxAnalogueGain; + double defAnalogueGain; utils::Duration minFrameDuration; utils::Duration maxFrameDuration; utils::Duration lineDuration; @@ -119,7 +122,6 @@ class AgcAlgorithm { public: struct ConfigurationParams { - const CameraSensorHelper *sensor; const IPACameraSensorInfo &sensorInfo; const ControlInfoMap &sensorControls; ControlInfoMap::Map &ctrlMap; @@ -135,7 +137,8 @@ public: double lux = 0; }; - int init(const ValueNode &tuningData, const ConfigurationParams &config); + int init(const ValueNode &tuningData, CameraSensorHelper *sensor, + const ConfigurationParams &config); int configure(agc::Session &session, agc::ActiveState &state, const ConfigurationParams &config); @@ -157,7 +160,8 @@ private: const agc::FrameContext &frameContext, ControlList &metadata); - AgcMeanLuminance impl_; + std::variant impl_; + CameraSensorHelper *sensor_ = nullptr; }; } /* namespace ipa */ diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 96e0a4c7e9..e977ef75bd 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -122,8 +122,7 @@ Agc::Agc() int Agc::init(IPAContext &context, const ValueNode &tuningData) { - return agc_.init(tuningData, { - .sensor = context.camHelper.get(), + return agc_.init(tuningData, context.camHelper.get(), { .sensorInfo = context.sensorInfo, .sensorControls = context.sensorControls, .ctrlMap = context.ctrlMap, @@ -138,7 +137,6 @@ int Agc::configure(IPAContext &context, return ret; ret = agc_.configure(context.configuration.agc, context.activeState.agc, { - .sensor = context.camHelper.get(), .sensorInfo = context.sensorInfo, .sensorControls = context.sensorControls, .ctrlMap = context.ctrlMap, diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index bb0345ca87..79a7474cb2 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -136,8 +136,7 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) { int ret; - ret = agc_.init(tuningData, { - .sensor = context.camHelper.get(), + ret = agc_.init(tuningData, context.camHelper.get(), { .sensorInfo = context.sensorInfo, .sensorControls = context.sensorControls, .ctrlMap = context.ctrlMap, @@ -163,7 +162,6 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { int ret = agc_.configure(context.configuration.agc, context.activeState.agc, { - .sensor = context.camHelper.get(), .sensorInfo = context.sensorInfo, .sensorControls = context.sensorControls, .ctrlMap = context.ctrlMap,