| Message ID | 20260824091407.502020-22-barnabas.pocze@ideasonboard.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series |
|
| Related | show |
Hi Barnabás, Quoting Barnabás Pőcze (2026-08-24 11:13:40) > Add a class that implements the `Algorithm` interface using `AgcMeanLuminance` > based on the rkisp1 `Agc` algorithm, with the following main adjustments: > > * the parameters for `process()` have been made optional to handle > the cases where statistics are not available; > * the "raw" capture check has been replaced with the "autoAllowed" > session parameter; > * the controls are only provided after `configure()`. > > Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> > --- On such big patches it is helpful to have a changelog for the patch itself. I think whatever is missing from this series can be added on top. So I'll repeat my tags form v5: Reviewed-by: Stefan Klug <stefan.klug@ideasonboard.com> Tested-by: Stefan Klug <stefan.klug@ideasonboard.com> Thanks for your work. Best regards, Stefan > src/ipa/libipa/agc.cpp | 729 +++++++++++++++++++++++++++++- > src/ipa/libipa/agc.h | 111 +++++ > src/ipa/rkisp1/algorithms/agc.cpp | 461 +++---------------- > src/ipa/rkisp1/algorithms/agc.h | 10 +- > src/ipa/rkisp1/algorithms/lux.cpp | 2 +- > src/ipa/rkisp1/ipa_context.cpp | 98 ---- > src/ipa/rkisp1/ipa_context.h | 47 +- > 7 files changed, 896 insertions(+), 562 deletions(-) > > diff --git a/src/ipa/libipa/agc.cpp b/src/ipa/libipa/agc.cpp > index 864b73d5f1..e311bfd0d6 100644 > --- a/src/ipa/libipa/agc.cpp > +++ b/src/ipa/libipa/agc.cpp > @@ -1,21 +1,41 @@ > /* SPDX-License-Identifier: LGPL-2.1-or-later */ > /* > - * Copyright (C) 2026 Ideas On Board > + * Copyright (C) 2021-2026 Ideas On Board > * > - * Auto exposure/gain algorithm for implementing the IPA-specific AGC algorithms > + * libIPA Agc algorithm > */ > > #include "agc.h" > > +#include <algorithm> > +#include <array> > +#include <chrono> > +#include <cmath> > +#include <optional> > +#include <ratio> > + > +#include <linux/v4l2-controls.h> > + > +#include <libcamera/base/log.h> > +#include <libcamera/base/span.h> > + > +#include <libcamera/control_ids.h> > +#include <libcamera/controls.h> > + > +#include <libcamera/ipa/core_ipa_interface.h> > + > /** > * \file agc.h > * \brief libipa AGC algorithm > */ > - > namespace libcamera { > > namespace ipa { > > +using namespace std::chrono_literals; > + > +LOG_DEFINE_CATEGORY(Agc) > + > namespace agc { > > /** > @@ -44,8 +64,711 @@ namespace agc { > * otherwise the gain value will be used directly. > */ > > +/** > + * \struct Session > + * \brief Session configuration for AgcAlgorithm > + * > + * \var Session::minExposureTime > + * \brief Minimum exposure time for the streaming session > + * > + * \var Session::maxExposureTime > + * \brief Maximum exposure time for the streaming session > + * > + * \var Session::minAnalogueGain > + * \brief Minimum analogue gain for the streaming session > + * > + * \var Session::maxAnalogueGain > + * \brief Maximum analogue gain for the streaming session > + * > + * \var Session::minFrameDuration > + * \brief Minimum frame duration for the streaming session > + * > + * \var Session::maxFrameDuration > + * \brief Maximum frame duration for the streaming session > + * > + * \var Session::lineDuration > + * \brief Line duration for the streaming session > + * > + * \var Session::sensor > + * \brief Details of the sensor configuration > + * > + * \var Session::sensor.outputSize > + * \brief Configured output size of the sensor > + * > + * \var Session::autoAllowed > + * \copybrief AgcAlgorithm::ConfigurationParams::autoAllowed > + * \sa AgcAlgorithm::ConfigurationParams::autoAllowed > + */ > + > +/** > + * \struct ActiveState > + * \brief Active state for AgcAlgorithm > + * > + * The \a automatic variables track the latest values computed by algorithm > + * based on the latest processed statistics. All other variables track the > + * consolidated controls requested in queued requests. > + * > + * \var ActiveState::manual > + * \brief Manual exposure time and analog gain (set through requests) > + * > + * \var ActiveState::manual.exposure > + * \brief Manual exposure time expressed as a number of lines as set by the > + * ExposureTime control > + * > + * \var ActiveState::manual.gain > + * \brief Manual analogue gain as set by the AnalogueGain control > + * > + * \var ActiveState::automatic > + * \brief Automatic exposure time and analog gain (computed by the algorithm) > + * > + * \var ActiveState::automatic.exposure > + * \brief Automatic exposure time expressed as a number of lines > + * > + * \var ActiveState::automatic.gain > + * \brief Automatic analogue gain multiplier > + * > + * \var ActiveState::automatic.quantizationGain > + * \brief Automatic quantization gain multiplier > + * > + * \var ActiveState::automatic.yTarget > + * \brief Automatically determined luminance target > + * > + * \var ActiveState::autoExposureEnabled > + * \brief Whether automatic exposure control is enabled by the ExposureTimeMode control > + * > + * \var ActiveState::autoGainEnabled > + * \brief Whether automatic gain control is enabled by the AnalogueGainMode control > + * > + * \var ActiveState::exposureValue > + * \brief Exposure value as set by the ExposureValue control > + * > + * \var ActiveState::constraintMode > + * \brief Constraint mode as set by the AeConstraintMode control > + * > + * \var ActiveState::exposureMode > + * \brief Exposure mode as set by the AeExposureMode control > + * > + * \var ActiveState::minFrameDuration > + * \brief Minimum frame duration as set by the FrameDurationLimits control > + * > + * \var ActiveState::maxFrameDuration > + * \brief Maximum frame duration as set by the FrameDurationLimits control > + */ > + > +/** > + * \struct FrameContext > + * \brief Per-frame context for AgcAlgorithm > + * > + * \var FrameContext::exposure > + * \brief Exposure time expressed as a number of lines computed by the algorithm > + * > + * \var FrameContext::gain > + * \brief Analogue gain multiplier computed by the algorithm > + * > + * The gain should be translated to the sensor specific gain code before applying. > + * > + * \var FrameContext::quantizationGain > + * \brief Quantization gain multiplier computed by the algorithm > + * > + * \var FrameContext::exposureValue > + * \brief Exposure value as set by the ExposureValue control > + * > + * \var FrameContext::yTarget > + * \brief Luminance target computed by the algorithm > + * > + * \var FrameContext::vblank > + * \brief Vertical blanking parameter computed by the algorithm > + * > + * \var FrameContext::autoExposureEnabled > + * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control > + * > + * \var FrameContext::autoGainEnabled > + * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control > + * > + * \var FrameContext::constraintMode > + * \brief Constraint mode as set by the AeConstraintMode control > + * > + * \var FrameContext::exposureMode > + * \brief Exposure mode as set by the AeExposureMode control > + * > + * \var FrameContext::minFrameDuration > + * \brief Minimum frame duration as set by the FrameDurationLimits control > + * > + * \var FrameContext::maxFrameDuration > + * \brief Maximum frame duration as set by the FrameDurationLimits control > + * > + * \var FrameContext::frameDuration > + * \brief The actual FrameDuration used by the algorithm for the frame > + * > + * \var FrameContext::autoExposureModeChange > + * \brief Indicate if autoExposureEnabled has changed from true in the previous > + * frame to false in the current frame, and no manual exposure value has been > + * supplied in the current frame > + * > + * \var FrameContext::autoGainModeChange > + * \brief Indicate if autoGainEnabled has changed from true in the previous > + * frame to false in the current frame, and no manual gain value has been > + * supplied in the current frame > + */ > + > } /* namespace agc */ > > +/** > + * \class AgcAlgorithm > + * \brief libIPA LSC algorithm algorithm > + * > + * The AgcAlgorithm class can be used to implement automatic exposure/gain > + * control in an IPA module, conforming to the prescribed Algorithm interface. > + * Internally AgcMeanLuminance is used, this class merely concerns itself with > + * processing the sensor properties, establishing limits, managing controls, > + * providing the resulting metadata, and driving the actual algorithm in > + * process(). > + * > + * Users should compose the agc::Session, agc::ActiveState, and agc::FrameContext > + * structures into their session configuration, active state, and frame contexts, > + * respectively. Furthermore, in their implementation of the Algorithm virtual > + * function, they should simply call the identically named member function of > + * AgcAlgorithm. > + * > + * \todo DigitalGain, DigitalGainMode > + * \todo Expand documentation > + */ > + > +/** > + * \struct AgcAlgorithm::ConfigurationParams > + * \brief Parameters for AgcAlgorithm::configure() > + * > + * \var AgcAlgorithm::ConfigurationParams::sensor > + * \brief CameraSensorHelper for the sensor > + * > + * \var AgcAlgorithm::ConfigurationParams::sensorInfo > + * \brief Current configuration of the sensor > + * > + * \var AgcAlgorithm::ConfigurationParams::sensorControls > + * \brief ControlInfoMap of the sensor > + * > + * \var AgcAlgorithm::ConfigurationParams::ctrlMap > + * \brief ControlInfoMap::Map to update with controls > + * > + * \var AgcAlgorithm::ConfigurationParams::autoAllowed > + * \brief Whether to enable auto controls > + * > + * If \a false, the algorithm is set up for manual exposure and gain > + * control only, without automatic adjustments. In this mode statistics > + * must not be provided to AgcAlgorithm::process(), and ExposureTimeMode > + * and AnalogueGainMode will only advertise manual control. > + */ > + > +/** > + * \struct AgcAlgorithm::ProcessParams > + * \brief Parameters for AgcAlgorithm::process() > + * > + * \var AgcAlgorithm::ProcessParams::traits > + * \brief Implementation of AgcMeanLuminance::Traits > + * > + * \var AgcAlgorithm::ProcessParams::yHist > + * \brief Luminance histogram of the frame > + * > + * \var AgcAlgorithm::ProcessParams::exposure > + * \brief Effective exposure of the frame > + * > + * \var AgcAlgorithm::ProcessParams::gain > + * \brief Effective gain of the frame > + * > + * \var AgcAlgorithm::ProcessParams::additionalConstraints > + * \brief Additional AgcMeanLuminance::AgcConstraints to apply > + * > + * \var AgcAlgorithm::ProcessParams::lux > + * \brief Effective lux value of the frame > + */ > + > +/** > + * \brief Load tuning data and configure > + * \param[in] tuningData The tuning data > + * \param[in] config The algorithm configuration > + * > + * This function loads the tuning data and configures the algorithm as if > + * by a call to configure(), in order to provide the available controls > + * in ConfigurationParams::ctrlMap. > + * > + * The tuning data format is that of AgcMeanLuminance. Refer to > + * AgcMeanLuminance::parseTuningData() for more details. > + * > + * \return 0 on success, or a negative error code > + * > + * \sa Algorithm::init() > + */ > +int AgcAlgorithm::init(const ValueNode &tuningData, const ConfigurationParams &config) > +{ > + int ret = impl_.parseTuningData(tuningData); > + if (ret) > + return ret; > + > + /* > + * The purpose of this `configure()` is merely to provide the > + * available controls in `config.ctrlMap`. > + * > + * \todo Remove it once IPA modules have been changed to > + * configure the algorithms during initialization. > + */ > + > + agc::Session dummySession; > + agc::ActiveState dummyState; > + > + return configure(dummySession, dummyState, config); > +} > + > +/** > + * \brief Initialize the session configuration and active state > + * \param[in] session The agc session configuration > + * \param[in] state The agc active state > + * \param[in] config The algorithm configuration > + * > + * This function initializes \a session and \a state based on the tuning > + * data loaded by init() and the configuration in \a config. > + * > + * It also updates ConfigurationParams::ctrlMap with the limits of the various > + * available agc-related controls. Users are expected to propagate these controls > + * to the camera. > + * > + * \return 0 on success, or a negative error code > + * > + * \sa Algorithm::configure() > + */ > +int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, > + const ConfigurationParams &config) > +{ > + session = {}; > + session.autoAllowed = config.autoAllowed; > + session.lineDuration = > + config.sensorInfo.minLineLength * 1.0s / config.sensorInfo.pixelRate; > + session.sensor.outputSize = config.sensorInfo.outputSize; > + > + const double lineDurationUs = session.lineDuration.get<std::micro>(); > + > + /* > + * Compute exposure time limits from the V4L2_CID_EXPOSURE control > + * limits and the line duration. > + */ > + > + const ControlInfo &v4l2Exposure = config.sensorControls.find(V4L2_CID_EXPOSURE)->second; > + int32_t minExposure = v4l2Exposure.min().get<int32_t>(); > + int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); > + int32_t defExposure = v4l2Exposure.def().get<int32_t>(); > + > + /* Compute the analogue gain limits. */ > + const ControlInfo &v4l2Gain = config.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; > + float minGain = config.sensor->gain(v4l2Gain.min().get<int32_t>()); > + float maxGain = config.sensor->gain(v4l2Gain.max().get<int32_t>()); > + float defGain = config.sensor->gain(v4l2Gain.def().get<int32_t>()); > + > + LOG(Agc, Debug) > + << "Exposure: [" << minExposure << ", " << maxExposure > + << "], gain: [" << minGain << ", " << maxGain << "]"; > + > + /* > + * Compute the frame duration limits. > + * > + * The frame length is computed assuming a fixed line length combined > + * with the vertical frame sizes. > + */ > + const ControlInfo &v4l2HBlank = config.sensorControls.find(V4L2_CID_HBLANK)->second; > + uint32_t hblank = v4l2HBlank.def().get<int32_t>(); > + uint32_t lineLength = config.sensorInfo.outputSize.width + hblank; > + > + const ControlInfo &v4l2VBlank = config.sensorControls.find(V4L2_CID_VBLANK)->second; > + std::array<uint32_t, 3> frameHeights{ > + v4l2VBlank.min().get<int32_t>() + config.sensorInfo.outputSize.height, > + v4l2VBlank.max().get<int32_t>() + config.sensorInfo.outputSize.height, > + v4l2VBlank.def().get<int32_t>() + config.sensorInfo.outputSize.height, > + }; > + > + std::array<int64_t, 3> frameDurations; > + for (unsigned int i = 0; i < frameHeights.size(); ++i) { > + uint64_t frameSize = lineLength * frameHeights[i]; > + frameDurations[i] = frameSize / (config.sensorInfo.pixelRate / 1000000U); > + } > + > + /* > + * When the AGC computes the new exposure values for a frame, it needs > + * to know the limits for exposure time and analogue gain. As it depends > + * on the sensor, update it with the controls. > + * > + * \todo take VBLANK into account for maximum exposure time > + */ > + session.minExposureTime = minExposure * session.lineDuration; > + session.maxExposureTime = maxExposure * session.lineDuration; > + session.minAnalogueGain = minGain; > + session.maxAnalogueGain = maxGain; > + session.minFrameDuration = std::chrono::microseconds(frameDurations[0]); > + session.maxFrameDuration = std::chrono::microseconds(frameDurations[1]); > + > + impl_.configure(session.lineDuration, config.sensor); > + impl_.setLimits(session.minExposureTime, session.maxExposureTime, > + session.minAnalogueGain, session.maxAnalogueGain, > + {}); > + impl_.resetFrameCount(); > + > + /* Configure the default exposure and gain. */ > + state = {}; > + state.automatic.gain = session.minAnalogueGain; > + state.automatic.exposure = 10ms / session.lineDuration; > + state.automatic.quantizationGain = 1; > + state.automatic.yTarget = impl_.effectiveYTarget(0, 1); > + state.manual.gain = state.automatic.gain; > + state.manual.exposure = state.automatic.exposure; > + state.autoExposureEnabled = session.autoAllowed; > + state.autoGainEnabled = session.autoAllowed; > + state.exposureValue = 0; > + state.constraintMode = > + static_cast<controls::AeConstraintModeEnum>(impl_.constraintModes().begin()->first); > + state.exposureMode = > + static_cast<controls::AeExposureModeEnum>(impl_.exposureModeHelpers().begin()->first); > + state.minFrameDuration = session.minFrameDuration; > + state.maxFrameDuration = session.maxFrameDuration; > + > + /* \todo Move this to the `Camera` class. */ > + config.ctrlMap[&controls::AeEnable] = ControlInfo{ > + false, session.autoAllowed, session.autoAllowed > + }; > + config.ctrlMap[&controls::AnalogueGain] = ControlInfo{ > + minGain, maxGain, defGain > + }; > + config.ctrlMap[&controls::ExposureTime] = ControlInfo{ > + static_cast<int32_t>(minExposure * lineDurationUs), > + static_cast<int32_t>(maxExposure * lineDurationUs), > + static_cast<int32_t>(defExposure * lineDurationUs), > + }; > + config.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ > + frameDurations[0], frameDurations[1], > + Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, > + }; > + config.ctrlMap[&controls::ExposureTimeMode] = ControlInfo{ > + {{ controls::ExposureTimeModeAuto, controls::ExposureTimeModeManual }}, > + controls::ExposureTimeModeAuto, > + }; > + config.ctrlMap[&controls::AnalogueGainMode] = ControlInfo{ > + {{ controls::AnalogueGainModeAuto, controls::AnalogueGainModeManual }}, > + controls::AnalogueGainModeAuto, > + }; > + config.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); > + config.ctrlMap.merge(impl_.controls()); > + > + return 0; > +} > + > +/** > + * \brief Queue a request > + * \param[in] session The agc session configuration > + * \param[in] state The agc active state > + * \param[in] frameContext The agc frame context > + * \param[in] controls The list of controls associated with a Request > + * > + * This functions processes the agc-related controls in \a controls for the frame > + * denoted by \a frameContext, and updates \a state and \a frameContext accordingly. > + * > + * \sa Algorithm::queueRequest() > + */ > +void AgcAlgorithm::queueRequest(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, const ControlList &controls) > +{ > + if (session.autoAllowed) { > + const auto &aeEnable = controls.get(controls::ExposureTimeMode); > + if (aeEnable && > + (*aeEnable == controls::ExposureTimeModeAuto) != state.autoExposureEnabled) { > + state.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); > + > + LOG(Agc, Debug) > + << (state.autoExposureEnabled ? "Enabling" : "Disabling") > + << " AGC (exposure)"; > + > + /* > + * If we go from auto -> manual with no manual control > + * set, use the last computed value, which we don't > + * know until prepare() so save this information. > + * > + * \todo Check the previous frame at prepare() time > + * instead of saving a flag here > + */ > + if (!state.autoExposureEnabled && !controls.get(controls::ExposureTime)) > + frameContext.autoExposureModeChange = true; > + } > + > + const auto &agEnable = controls.get(controls::AnalogueGainMode); > + if (agEnable && > + (*agEnable == controls::AnalogueGainModeAuto) != state.autoGainEnabled) { > + state.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); > + > + LOG(Agc, Debug) > + << (state.autoGainEnabled ? "Enabling" : "Disabling") > + << " AGC (gain)"; > + > + /* > + * If we go from auto -> manual with no manual control > + * set, use the last computed value, which we don't > + * know until prepare() so save this information. > + */ > + if (!state.autoGainEnabled && !controls.get(controls::AnalogueGain)) > + frameContext.autoGainModeChange = true; > + } > + } > + > + const auto &exposure = controls.get(controls::ExposureTime); > + if (exposure && !state.autoExposureEnabled) { > + state.manual.exposure = *exposure * 1.0us / session.lineDuration; > + > + LOG(Agc, Debug) << "Set exposure to " << state.manual.exposure; > + } > + > + const auto &gain = controls.get(controls::AnalogueGain); > + if (gain && !state.autoGainEnabled) { > + state.manual.gain = *gain; > + > + LOG(Agc, Debug) << "Set gain to " << state.manual.gain; > + } > + > + frameContext.autoExposureEnabled = state.autoExposureEnabled; > + frameContext.autoGainEnabled = state.autoGainEnabled; > + > + if (!frameContext.autoExposureEnabled) > + frameContext.exposure = state.manual.exposure; > + if (!frameContext.autoGainEnabled) > + frameContext.gain = state.manual.gain; > + > + if (!frameContext.autoExposureEnabled && !frameContext.autoGainEnabled) > + frameContext.quantizationGain = 1.0; > + > + const auto &exposureMode = controls.get(controls::AeExposureMode); > + if (exposureMode) > + state.exposureMode = > + static_cast<controls::AeExposureModeEnum>(*exposureMode); > + frameContext.exposureMode = state.exposureMode; > + > + const auto &constraintMode = controls.get(controls::AeConstraintMode); > + if (constraintMode) > + state.constraintMode = > + static_cast<controls::AeConstraintModeEnum>(*constraintMode); > + frameContext.constraintMode = state.constraintMode; > + > + const auto &exposureValue = controls.get(controls::ExposureValue); > + if (exposureValue) > + state.exposureValue = *exposureValue; > + frameContext.exposureValue = state.exposureValue; > + > + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); > + if (frameDurationLimits) { > + /* Limit the control value to the limits in ControlInfo */ > + state.minFrameDuration = std::clamp<utils::Duration>( > + std::chrono::microseconds((*frameDurationLimits).front()), > + session.minFrameDuration, session.maxFrameDuration); > + > + state.maxFrameDuration = std::clamp<utils::Duration>( > + std::chrono::microseconds((*frameDurationLimits).back()), > + session.minFrameDuration, session.maxFrameDuration); > + } > + frameContext.minFrameDuration = state.minFrameDuration; > + frameContext.maxFrameDuration = state.maxFrameDuration; > +} > + > +/** > + * \brief Prepare a frame > + * \param[in] state The agc active state > + * \param[in] frameContext The agc frame context > + * > + * This function prepares the parameters for the frame denoted by \a frameContext. > + * After a call to this function, the values of \ref agc::FrameContext::exposure > + * "frameContext.exposure" and \ref agc::FrameContext::gain "frameContext.gain" > + * will be finalized and may be used by the caller (see agc::prepareControls()). > + * > + * \todo Finalize \ref agc::FrameContext::vblank "frameContext.vblank" as well > + * > + * \sa Algorithm::prepare() > + */ > +void AgcAlgorithm::prepare(agc::ActiveState &state, agc::FrameContext &frameContext) > +{ > + uint32_t activeAutoExposure = state.automatic.exposure; > + double activeAutoGain = state.automatic.gain; > + double activeAutoQGain = state.automatic.quantizationGain; > + > + /* Populate exposure and gain in auto mode */ > + if (frameContext.autoExposureEnabled) { > + frameContext.exposure = activeAutoExposure; > + frameContext.quantizationGain = activeAutoQGain; > + } > + if (frameContext.autoGainEnabled) { > + frameContext.gain = activeAutoGain; > + frameContext.quantizationGain = activeAutoQGain; > + } > + > + /* > + * Populate manual exposure and gain from the active auto values when > + * transitioning from auto to manual > + */ > + if (!frameContext.autoExposureEnabled && frameContext.autoExposureModeChange) { > + state.manual.exposure = activeAutoExposure; > + frameContext.exposure = activeAutoExposure; > + } > + if (!frameContext.autoGainEnabled && frameContext.autoGainModeChange) { > + state.manual.gain = activeAutoGain; > + frameContext.gain = activeAutoGain; > + frameContext.quantizationGain = activeAutoQGain; > + } > + > + frameContext.yTarget = state.automatic.yTarget; > +} > + > +/** > + * \brief Process frame statistics > + * \param[in] session The agc session configuration > + * \param[in] state The agc active state > + * \param[in] frameContext The agc frame context > + * \param[in] params The algorithm parameters > + * \param[in] metadata The list of metadata > + * > + * This function processes the statistics for the completed frame denoted by > + * \a frameContext, runs the AGC implementation, and updates \a state appropriately. > + * This function also populates \a metadata for the completed frame. > + * > + * \a params must be omitted if the session was configured without "autoAllowed", > + * and it may be omitted even if auto control is enabled, for example, if the > + * statistics could not be delivered due to some ephemeral error. This ensures > + * that the algorithm state will not go out of sync, and that metadata is produced > + * as expected. > + * > + * Care must be taken to convert the platform specific statistics to the format > + * expected in \a params. See ProcessParams for the details. > + * > + * \sa Algorithm::process() > + */ > +void AgcAlgorithm::process(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, > + ControlList &metadata) > +{ > + if (!params) { > + processFrameDuration(session, frameContext, frameContext.minFrameDuration); > + fillMetadata(session, frameContext, metadata); > + return; > + } > + > + ASSERT(session.autoAllowed); > + > + const utils::Duration &lineDuration = session.lineDuration; > + > + /* > + * Set the AGC limits using the fixed exposure time and/or gain in > + * manual mode, or the sensor limits in auto mode. > + */ > + utils::Duration minExposureTime; > + utils::Duration maxExposureTime; > + double minAnalogueGain; > + double maxAnalogueGain; > + > + /* \todo This uses the configuration from an already completed frame. */ > + > + if (frameContext.autoExposureEnabled) { > + minExposureTime = session.minExposureTime; > + maxExposureTime = std::clamp(frameContext.maxFrameDuration, > + session.minExposureTime, > + session.maxExposureTime); > + } else { > + minExposureTime = lineDuration * frameContext.exposure; > + maxExposureTime = minExposureTime; > + } > + > + if (frameContext.autoGainEnabled) { > + minAnalogueGain = session.minAnalogueGain; > + maxAnalogueGain = session.maxAnalogueGain; > + } else { > + minAnalogueGain = frameContext.gain; > + maxAnalogueGain = frameContext.gain; > + } > + > + /* > + * The Agc algorithm needs to know the effective exposure value that was > + * applied to the sensor when the statistics were collected. > + */ > + utils::Duration effectiveExposureValue = > + lineDuration * params->exposure * params->gain; > + > + impl_.setLimits(minExposureTime, maxExposureTime, > + minAnalogueGain, maxAnalogueGain, > + std::move(params->additionalConstraints)); > + > + const auto &newEv = impl_.calculateNewEv({ > + .traits = params->traits, > + .yHist = params->yHist, > + .effectiveExposureValue = effectiveExposureValue, > + .constraintModeIndex = frameContext.constraintMode, > + .exposureModeIndex = frameContext.exposureMode, > + .lux = params->lux, > + .exposureCompensation = std::pow(2.0, frameContext.exposureValue), > + }); > + > + /* Update the estimated exposure and gain. */ > + state.automatic.exposure = newEv.exposureTime / lineDuration; > + state.automatic.gain = newEv.analogueGain; > + state.automatic.quantizationGain = newEv.quantizationGain; > + state.automatic.yTarget = newEv.yTarget; > + > + LOG(Agc, Debug) > + << "Divided up exposure time, analogue gain, quantization gain" > + << " and digital gain are " << newEv.exposureTime > + << ", " << state.automatic.gain << ", " << state.automatic.quantizationGain > + << " and " << newEv.digitalGain; > + > + /* > + * Expand the target frame duration so that we do not run faster than > + * the minimum frame duration when we have short exposures. > + */ > + processFrameDuration(session, frameContext, > + std::max(frameContext.minFrameDuration, newEv.exposureTime)); > + > + fillMetadata(session, frameContext, metadata); > +} > + > +/** > + * \brief Process frame duration and compute vblank > + * \param[in] session The session parameters > + * \param[in] frameContext The current frame context > + * \param[in] frameDuration The target frame duration > + * > + * Compute and populate vblank from the target frame duration. > + */ > +void AgcAlgorithm::processFrameDuration(const agc::Session &session, > + agc::FrameContext &frameContext, > + utils::Duration frameDuration) > +{ > + const utils::Duration &lineDuration = session.lineDuration; > + > + frameContext.vblank = > + (frameDuration / lineDuration) - session.sensor.outputSize.height; > + > + /* Update frame duration accounting for line length quantization. */ > + frameContext.frameDuration = > + (session.sensor.outputSize.height + frameContext.vblank) * lineDuration; > +} > + > +void AgcAlgorithm::fillMetadata(const agc::Session &session, > + const agc::FrameContext &frameContext, > + ControlList &metadata) > +{ > + > + metadata.set(controls::AnalogueGain, frameContext.gain); > + metadata.set(controls::ExposureTime, > + utils::Duration(session.lineDuration * frameContext.exposure).get<std::micro>()); > + metadata.set(controls::FrameDuration, frameContext.frameDuration.get<std::micro>()); > + metadata.set(controls::ExposureTimeMode, frameContext.autoExposureEnabled > + ? controls::ExposureTimeModeAuto > + : controls::ExposureTimeModeManual); > + metadata.set(controls::AnalogueGainMode, frameContext.autoGainEnabled > + ? controls::AnalogueGainModeAuto > + : controls::AnalogueGainModeManual); > + > + metadata.set(controls::AeExposureMode, frameContext.exposureMode); > + metadata.set(controls::AeConstraintMode, frameContext.constraintMode); > + metadata.set(controls::ExposureValue, frameContext.exposureValue); > +} > + > } /* namespace ipa */ > > } /* namespace libcamera */ > diff --git a/src/ipa/libipa/agc.h b/src/ipa/libipa/agc.h > index 5a67464086..06600f0208 100644 > --- a/src/ipa/libipa/agc.h > +++ b/src/ipa/libipa/agc.h > @@ -7,21 +7,87 @@ > > #pragma once > > +#include <optional> > #include <stdint.h> > #include <utility> > +#include <vector> > > #include <linux/v4l2-controls.h> > > +#include <libcamera/base/utils.h> > + > +#include <libcamera/control_ids.h> > #include <libcamera/controls.h> > +#include <libcamera/geometry.h> > > +#include "agc_mean_luminance.h" > #include "camera_sensor_helper.h" > > namespace libcamera { > > +struct IPACameraSensorInfo; > + > namespace ipa { > > +class Histogram; > + > namespace agc { > > +struct Session { > + utils::Duration minExposureTime; > + utils::Duration maxExposureTime; > + double minAnalogueGain; > + double maxAnalogueGain; > + utils::Duration minFrameDuration; > + utils::Duration maxFrameDuration; > + utils::Duration lineDuration; > + > + struct { > + Size outputSize; > + } sensor; > + > + bool autoAllowed; > +}; > + > +struct ActiveState { > + struct { > + uint32_t exposure; > + double gain; > + } manual; > + struct { > + uint32_t exposure; > + double gain; > + double quantizationGain; > + double yTarget; > + } automatic; > + > + bool autoExposureEnabled; > + bool autoGainEnabled; > + double exposureValue; > + controls::AeConstraintModeEnum constraintMode; > + controls::AeExposureModeEnum exposureMode; > + utils::Duration minFrameDuration; > + utils::Duration maxFrameDuration; > +}; > + > +struct FrameContext { > + uint32_t exposure; > + double gain; > + double quantizationGain; > + double exposureValue; > + double yTarget; > + uint32_t vblank; > + bool autoExposureEnabled; > + bool autoGainEnabled; > + controls::AeConstraintModeEnum constraintMode; > + controls::AeExposureModeEnum exposureMode; > + utils::Duration minFrameDuration; > + utils::Duration maxFrameDuration; > + utils::Duration frameDuration; > + bool autoExposureModeChange; > + bool autoGainModeChange; > +}; > + > [[nodiscard]] > inline std::pair<uint32_t, double> > extractControls(const ControlList &controls, const CameraSensorHelper *sensor) > @@ -48,6 +114,51 @@ prepareControls(ControlList &controls, const CameraSensorHelper *sensor, > > } /* namespace agc */ > > +class AgcAlgorithm > +{ > +public: > + struct ConfigurationParams { > + const CameraSensorHelper *sensor; > + const IPACameraSensorInfo &sensorInfo; > + const ControlInfoMap &sensorControls; > + ControlInfoMap::Map &ctrlMap; > + bool autoAllowed = true; > + }; > + > + struct ProcessParams { > + const AgcMeanLuminance::Traits &traits; > + const Histogram &yHist; > + uint32_t exposure; > + double gain; > + std::vector<AgcMeanLuminance::AgcConstraint> &&additionalConstraints = {}; > + double lux = 0; > + }; > + > + int init(const ValueNode &tuningData, const ConfigurationParams &config); > + > + int configure(agc::Session &session, agc::ActiveState &state, > + const ConfigurationParams &config); > + > + void queueRequest(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, const ControlList &controls); > + > + void prepare(agc::ActiveState &state, agc::FrameContext &frameContext); > + > + void process(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, > + ControlList &metadata); > + > +private: > + void processFrameDuration(const agc::Session &session, > + agc::FrameContext &frameContext, > + utils::Duration frameDuration); > + void fillMetadata(const agc::Session &session, > + const agc::FrameContext &frameContext, > + ControlList &metadata); > + > + AgcMeanLuminance impl_; > +}; > + > } /* namespace ipa */ > > } /* namespace libcamera */ > diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp > index 7230232d24..bb0345ca87 100644 > --- a/src/ipa/rkisp1/algorithms/agc.cpp > +++ b/src/ipa/rkisp1/algorithms/agc.cpp > @@ -8,10 +8,7 @@ > #include "agc.h" > > #include <algorithm> > -#include <array> > -#include <chrono> > #include <cmath> > -#include <tuple> > #include <vector> > > #include <libcamera/base/log.h> > @@ -36,89 +33,6 @@ namespace ipa::rkisp1::algorithms { > > LOG_DEFINE_CATEGORY(RkISP1Agc) > > -namespace { > - > -void reconfigure(IPAContext &context) > -{ > - context.configuration.sensor.lineDuration = > - context.sensorInfo.minLineLength * 1.0s / context.sensorInfo.pixelRate; > - > - double lineDurationUs = context.configuration.sensor.lineDuration.get<std::micro>(); > - > - /* > - * Compute exposure time limits from the V4L2_CID_EXPOSURE control > - * limits and the line duration. > - */ > - > - const ControlInfo &v4l2Exposure = context.sensorControls.find(V4L2_CID_EXPOSURE)->second; > - int32_t minExposure = v4l2Exposure.min().get<int32_t>(); > - int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); > - int32_t defExposure = v4l2Exposure.def().get<int32_t>(); > - context.ctrlMap[&controls::ExposureTime] = ControlInfo{ > - static_cast<int32_t>(minExposure * lineDurationUs), > - static_cast<int32_t>(maxExposure * lineDurationUs), > - static_cast<int32_t>(defExposure * lineDurationUs), > - }; > - > - /* Compute the analogue gain limits. */ > - const ControlInfo &v4l2Gain = context.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; > - float minGain = context.camHelper->gain(v4l2Gain.min().get<int32_t>()); > - float maxGain = context.camHelper->gain(v4l2Gain.max().get<int32_t>()); > - float defGain = context.camHelper->gain(v4l2Gain.def().get<int32_t>()); > - context.ctrlMap[&controls::AnalogueGain] = ControlInfo{ > - minGain, > - maxGain, > - defGain, > - }; > - > - LOG(RkISP1Agc, Debug) > - << "Exposure: [" << minExposure << ", " << maxExposure > - << "], gain: [" << minGain << ", " << maxGain << "]"; > - > - /* > - * Compute the frame duration limits. > - * > - * The frame length is computed assuming a fixed line length combined > - * with the vertical frame sizes. > - */ > - const ControlInfo &v4l2HBlank = context.sensorControls.find(V4L2_CID_HBLANK)->second; > - uint32_t hblank = v4l2HBlank.def().get<int32_t>(); > - uint32_t lineLength = context.sensorInfo.outputSize.width + hblank; > - > - const ControlInfo &v4l2VBlank = context.sensorControls.find(V4L2_CID_VBLANK)->second; > - std::array<uint32_t, 3> frameHeights{ > - v4l2VBlank.min().get<int32_t>() + context.sensorInfo.outputSize.height, > - v4l2VBlank.max().get<int32_t>() + context.sensorInfo.outputSize.height, > - v4l2VBlank.def().get<int32_t>() + context.sensorInfo.outputSize.height, > - }; > - > - std::array<int64_t, 3> frameDurations; > - for (unsigned int i = 0; i < frameHeights.size(); ++i) { > - uint64_t frameSize = lineLength * frameHeights[i]; > - frameDurations[i] = frameSize / (context.sensorInfo.pixelRate / 1000000U); > - } > - > - context.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ > - frameDurations[0], > - frameDurations[1], > - Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, > - }; > - > - /* > - * When the AGC computes the new exposure values for a frame, it needs > - * to know the limits for exposure time and analogue gain. As it depends > - * on the sensor, update it with the controls. > - * > - * \todo take VBLANK into account for maximum exposure time > - */ > - context.configuration.sensor.minExposureTime = minExposure * context.configuration.sensor.lineDuration; > - context.configuration.sensor.maxExposureTime = maxExposure * context.configuration.sensor.lineDuration; > - context.configuration.sensor.minAnalogueGain = minGain; > - context.configuration.sensor.maxAnalogueGain = maxGain; > -} > - > -} /* namespace */ > - > /** > * \class Agc > * \brief A mean-based auto-exposure algorithm > @@ -222,7 +136,12 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) > { > int ret; > > - ret = agc_.parseTuningData(tuningData); > + ret = agc_.init(tuningData, { > + .sensor = context.camHelper.get(), > + .sensorInfo = context.sensorInfo, > + .sensorControls = context.sensorControls, > + .ctrlMap = context.ctrlMap, > + }); > if (ret) > return ret; > > @@ -231,21 +150,6 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) > if (ret) > return ret; > > - context.ctrlMap[&controls::ExposureTimeMode] = > - ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), > - ControlValue(controls::ExposureTimeModeManual) } }, > - ControlValue(controls::ExposureTimeModeAuto)); > - context.ctrlMap[&controls::AnalogueGainMode] = > - ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), > - ControlValue(controls::AnalogueGainModeManual) } }, > - ControlValue(controls::AnalogueGainModeAuto)); > - /* \todo Move this to the Camera class */ > - context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true, true); > - context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); > - context.ctrlMap.merge(agc_.controls()); > - > - reconfigure(context); > - > return 0; > } > > @@ -258,47 +162,24 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) > */ > int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) > { > - reconfigure(context); > - > - /* Configure the default exposure and gain. */ > - context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; > - context.activeState.agc.automatic.exposure = > - 10ms / context.configuration.sensor.lineDuration; > - context.activeState.agc.automatic.quantizationGain = 1.0; > - context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; > - context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; > - context.activeState.agc.autoExposureEnabled = !context.configuration.raw; > - context.activeState.agc.autoGainEnabled = !context.configuration.raw; > - context.activeState.agc.exposureValue = 0.0; > - > - context.activeState.agc.constraintMode = > - static_cast<controls::AeConstraintModeEnum>(agc_.constraintModes().begin()->first); > - context.activeState.agc.exposureMode = > - static_cast<controls::AeExposureModeEnum>(agc_.exposureModeHelpers().begin()->first); > + int ret = agc_.configure(context.configuration.agc, context.activeState.agc, { > + .sensor = context.camHelper.get(), > + .sensorInfo = context.sensorInfo, > + .sensorControls = context.sensorControls, > + .ctrlMap = context.ctrlMap, > + .autoAllowed = !context.configuration.raw, > + }); > + if (ret) > + return ret; > + > context.activeState.agc.meteringMode = > static_cast<controls::AeMeteringModeEnum>(meteringModes_.begin()->first); > > - /* Limit the frame duration to match current initialisation */ > - ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; > - context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get<int64_t>()); > - context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get<int64_t>()); > - > context.configuration.agc.measureWindow.h_offs = 0; > context.configuration.agc.measureWindow.v_offs = 0; > context.configuration.agc.measureWindow.h_size = configInfo.outputSize.width; > context.configuration.agc.measureWindow.v_size = configInfo.outputSize.height; > > - agc_.configure(context.configuration.sensor.lineDuration, context.camHelper.get()); > - > - agc_.setLimits(context.configuration.sensor.minExposureTime, > - context.configuration.sensor.maxExposureTime, > - context.configuration.sensor.minAnalogueGain, > - context.configuration.sensor.maxAnalogueGain, {}); > - > - context.activeState.agc.automatic.yTarget = agc_.effectiveYTarget(0, 1); > - > - agc_.resetFrameCount(); > - > return 0; > } > > @@ -312,73 +193,7 @@ void Agc::queueRequest(IPAContext &context, > { > auto &agc = context.activeState.agc; > > - if (!context.configuration.raw) { > - const auto &aeEnable = controls.get(controls::ExposureTimeMode); > - if (aeEnable && > - (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { > - agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); > - > - LOG(RkISP1Agc, Debug) > - << (agc.autoExposureEnabled ? "Enabling" : "Disabling") > - << " AGC (exposure)"; > - > - /* > - * If we go from auto -> manual with no manual control > - * set, use the last computed value, which we don't > - * know until prepare() so save this information. > - * > - * \todo Check the previous frame at prepare() time > - * instead of saving a flag here > - */ > - if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) > - frameContext.agc.autoExposureModeChange = true; > - } > - > - const auto &agEnable = controls.get(controls::AnalogueGainMode); > - if (agEnable && > - (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { > - agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); > - > - LOG(RkISP1Agc, Debug) > - << (agc.autoGainEnabled ? "Enabling" : "Disabling") > - << " AGC (gain)"; > - /* > - * If we go from auto -> manual with no manual control > - * set, use the last computed value, which we don't > - * know until prepare() so save this information. > - */ > - if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) > - frameContext.agc.autoGainModeChange = true; > - } > - } > - > - const auto &exposure = controls.get(controls::ExposureTime); > - if (exposure && !agc.autoExposureEnabled) { > - agc.manual.exposure = *exposure * 1.0us > - / context.configuration.sensor.lineDuration; > - > - LOG(RkISP1Agc, Debug) > - << "Set exposure to " << agc.manual.exposure; > - } > - > - const auto &gain = controls.get(controls::AnalogueGain); > - if (gain && !agc.autoGainEnabled) { > - agc.manual.gain = *gain; > - > - LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; > - } > - > - frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; > - frameContext.agc.autoGainEnabled = agc.autoGainEnabled; > - > - if (!frameContext.agc.autoExposureEnabled) > - frameContext.agc.exposure = agc.manual.exposure; > - if (!frameContext.agc.autoGainEnabled) > - frameContext.agc.gain = agc.manual.gain; > - > - if (!frameContext.agc.autoExposureEnabled && > - !frameContext.agc.autoGainEnabled) > - frameContext.agc.quantizationGain = 1.0; > + agc_.queueRequest(context.configuration.agc, agc, frameContext.agc, controls); > > const auto &meteringMode = controls.get(controls::AeMeteringMode); > if (meteringMode) { > @@ -387,42 +202,6 @@ void Agc::queueRequest(IPAContext &context, > static_cast<controls::AeMeteringModeEnum>(*meteringMode); > } > frameContext.agc.meteringMode = agc.meteringMode; > - > - const auto &exposureMode = controls.get(controls::AeExposureMode); > - if (exposureMode) > - agc.exposureMode = > - static_cast<controls::AeExposureModeEnum>(*exposureMode); > - frameContext.agc.exposureMode = agc.exposureMode; > - > - const auto &constraintMode = controls.get(controls::AeConstraintMode); > - if (constraintMode) > - agc.constraintMode = > - static_cast<controls::AeConstraintModeEnum>(*constraintMode); > - frameContext.agc.constraintMode = agc.constraintMode; > - > - const auto &exposureValue = controls.get(controls::ExposureValue); > - if (exposureValue) > - agc.exposureValue = *exposureValue; > - frameContext.agc.exposureValue = agc.exposureValue; > - > - const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); > - if (frameDurationLimits) { > - /* Limit the control value to the limits in ControlInfo */ > - ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; > - int64_t minFrameDuration = > - std::clamp((*frameDurationLimits).front(), > - limits.min().get<int64_t>(), > - limits.max().get<int64_t>()); > - int64_t maxFrameDuration = > - std::clamp((*frameDurationLimits).back(), > - limits.min().get<int64_t>(), > - limits.max().get<int64_t>()); > - > - agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); > - agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); > - } > - frameContext.agc.minFrameDuration = agc.minFrameDuration; > - frameContext.agc.maxFrameDuration = agc.maxFrameDuration; > } > > /** > @@ -431,41 +210,13 @@ void Agc::queueRequest(IPAContext &context, > void Agc::prepare(IPAContext &context, const uint32_t frame, > IPAFrameContext &frameContext, RkISP1Params *params) > { > - uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; > - double activeAutoGain = context.activeState.agc.automatic.gain; > - double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; > - > - /* Populate exposure and gain in auto mode */ > - if (frameContext.agc.autoExposureEnabled) { > - frameContext.agc.exposure = activeAutoExposure; > - frameContext.agc.quantizationGain = activeAutoQGain; > - } > - if (frameContext.agc.autoGainEnabled) { > - frameContext.agc.gain = activeAutoGain; > - frameContext.agc.quantizationGain = activeAutoQGain; > - } > - > - /* > - * Populate manual exposure and gain from the active auto values when > - * transitioning from auto to manual > - */ > - if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { > - context.activeState.agc.manual.exposure = activeAutoExposure; > - frameContext.agc.exposure = activeAutoExposure; > - } > - if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { > - context.activeState.agc.manual.gain = activeAutoGain; > - frameContext.agc.gain = activeAutoGain; > - frameContext.agc.quantizationGain = activeAutoQGain; > - } > + agc_.prepare(context.activeState.agc, frameContext.agc); > > if (context.configuration.compress.supported) { > frameContext.compress.enable = true; > frameContext.compress.gain = frameContext.agc.quantizationGain; > } > > - frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; > - > if (frame > 0 && !frameContext.agc.updateMetering) > return; > > @@ -521,50 +272,6 @@ void Agc::prepare(IPAContext &context, const uint32_t frame, > static_cast<rkisp1_cif_isp_histogram_mode>(hstConfig->mode)); > } > > -void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, > - ControlList &metadata) > -{ > - utils::Duration exposureTime = context.configuration.sensor.lineDuration > - * frameContext.sensor.exposure; > - metadata.set(controls::AnalogueGain, frameContext.sensor.gain); > - metadata.set(controls::ExposureTime, exposureTime.get<std::micro>()); > - metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get<std::micro>()); > - metadata.set(controls::ExposureTimeMode, > - frameContext.agc.autoExposureEnabled > - ? controls::ExposureTimeModeAuto > - : controls::ExposureTimeModeManual); > - metadata.set(controls::AnalogueGainMode, > - frameContext.agc.autoGainEnabled > - ? controls::AnalogueGainModeAuto > - : controls::AnalogueGainModeManual); > - > - metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); > - metadata.set(controls::AeExposureMode, frameContext.agc.exposureMode); > - metadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode); > - metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); > -} > - > -/** > - * \brief Process frame duration and compute vblank > - * \param[in] context The shared IPA context > - * \param[in] frameContext The current frame context > - * \param[in] frameDuration The target frame duration > - * > - * Compute and populate vblank from the target frame duration. > - */ > -void Agc::processFrameDuration(IPAContext &context, > - IPAFrameContext &frameContext, > - utils::Duration frameDuration) > -{ > - IPACameraSensorInfo &sensorInfo = context.sensorInfo; > - utils::Duration lineDuration = context.configuration.sensor.lineDuration; > - > - frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; > - > - /* Update frame duration accounting for line length quantization. */ > - frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; > -} > - > namespace { > > class AgcTraits final : public AgcMeanLuminance::Traits > @@ -638,21 +345,6 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, > IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats, > ControlList &metadata) > { > - if (!stats) { > - processFrameDuration(context, frameContext, > - frameContext.agc.minFrameDuration); > - fillMetadata(context, frameContext, metadata); > - return; > - } > - > - if (!(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP)) { > - fillMetadata(context, frameContext, metadata); > - LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; > - return; > - } > - > - const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; > - > /* > * \todo Verify that the exposure and gain applied by the sensor for > * this frame match what has been requested. This isn't a hard > @@ -661,95 +353,46 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, > * we receive), but is important in manual mode. > */ > > - const rkisp1_cif_isp_stat *params = &stats->params; > + const rkisp1_cif_isp_stat *params = nullptr; > > - /* > - * Set the AGC limits using the fixed exposure time and/or gain in > - * manual mode, or the sensor limits in auto mode. > - */ > - utils::Duration minExposureTime; > - utils::Duration maxExposureTime; > - double minAnalogueGain; > - double maxAnalogueGain; > - > - if (frameContext.agc.autoExposureEnabled) { > - minExposureTime = context.configuration.sensor.minExposureTime; > - maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, > - context.configuration.sensor.minExposureTime, > - context.configuration.sensor.maxExposureTime); > - } else { > - minExposureTime = context.configuration.sensor.lineDuration > - * frameContext.agc.exposure; > - maxExposureTime = minExposureTime; > + if (stats) { > + if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) > + params = &stats->params; > + else > + LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; > } > > - if (frameContext.agc.autoGainEnabled) { > - minAnalogueGain = context.configuration.sensor.minAnalogueGain; > - maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; > + if (params) { > + std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; > + if (context.activeState.wdr.mode != controls::WdrOff) > + additionalConstraints.push_back(context.activeState.wdr.constraint); > + > + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {{ > + .traits = AgcTraits{ > + { params->ae.exp_mean, context.hw.numAeCells }, > + meteringModes_.at(frameContext.agc.meteringMode), > + }, > + .yHist = { > + /* The lower 4 bits are fractional and meant to be discarded. */ > + { params->hist.hist_bins, context.hw.numHistogramBins }, > + [](uint32_t x) { return x >> 4; }, > + }, > + .exposure = frameContext.sensor.exposure, > + /* > + * Include the quantization gain if it was applied. Do not use > + * compress.gain because it will include gains that shall not be > + * reported to the user when HDR is implemented. > + */ > + .gain = frameContext.sensor.gain > + * (frameContext.compress.enable ? frameContext.agc.quantizationGain : 1), > + .additionalConstraints = std::move(additionalConstraints), > + .lux = frameContext.lux.lux, > + }}, metadata); > } else { > - minAnalogueGain = frameContext.agc.gain; > - maxAnalogueGain = frameContext.agc.gain; > + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {}, metadata); > } > > - std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; > - if (context.activeState.wdr.mode != controls::WdrOff) > - additionalConstraints.push_back(context.activeState.wdr.constraint); > - > - agc_.setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, > - std::move(additionalConstraints)); > - > - /* > - * The Agc algorithm needs to know the effective exposure value that was > - * applied to the sensor when the statistics were collected. > - */ > - utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; > - double analogueGain = frameContext.sensor.gain; > - utils::Duration effectiveExposureValue = exposureTime * analogueGain; > - > - /* > - * Include the quantization gain if it was applied. Do not use > - * compress.gain because it will include gains that shall not be > - * reported to the user when HDR is implemented. > - */ > - if (frameContext.compress.enable) > - effectiveExposureValue *= frameContext.agc.quantizationGain; > - > - /* The lower 4 bits are fractional and meant to be discarded. */ > - Histogram hist({ params->hist.hist_bins, context.hw.numHistogramBins }, > - [](uint32_t x) { return x >> 4; }); > - > - const auto &newEv = agc_.calculateNewEv({ > - .traits = AgcTraits{ > - { params->ae.exp_mean, context.hw.numAeCells }, > - meteringModes_.at(frameContext.agc.meteringMode), > - }, > - .yHist = hist, > - .effectiveExposureValue = effectiveExposureValue, > - .constraintModeIndex = frameContext.agc.constraintMode, > - .exposureModeIndex = frameContext.agc.exposureMode, > - .lux = frameContext.lux.lux, > - .exposureCompensation = pow(2.0, frameContext.agc.exposureValue), > - }); > - > - LOG(RkISP1Agc, Debug) > - << "Divided up exposure time, analogue gain, quantization gain" > - << " and digital gain are " << newEv.exposureTime << ", " << newEv.analogueGain > - << ", " << newEv.quantizationGain << " and " << newEv.digitalGain; > - > - IPAActiveState &activeState = context.activeState; > - /* Update the estimated exposure and gain. */ > - activeState.agc.automatic.exposure = newEv.exposureTime / lineDuration; > - activeState.agc.automatic.gain = newEv.analogueGain; > - activeState.agc.automatic.quantizationGain = newEv.quantizationGain; > - activeState.agc.automatic.yTarget = newEv.yTarget; > - /* > - * Expand the target frame duration so that we do not run faster than > - * the minimum frame duration when we have short exposures. > - */ > - processFrameDuration(context, frameContext, > - std::max(frameContext.agc.minFrameDuration, newEv.exposureTime)); > - > - fillMetadata(context, frameContext, metadata); > + metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); > } > > REGISTER_IPA_ALGORITHM(Agc, "Agc") > diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h > index 0527ca0d5f..3a4d7bc546 100644 > --- a/src/ipa/rkisp1/algorithms/agc.h > +++ b/src/ipa/rkisp1/algorithms/agc.h > @@ -14,7 +14,7 @@ > > #include <libcamera/geometry.h> > > -#include "libipa/agc_mean_luminance.h" > +#include "libipa/agc.h" > > #include "algorithm.h" > > @@ -47,14 +47,8 @@ private: > uint8_t computeHistogramPredivider(const Size &size, > enum rkisp1_cif_isp_histogram_mode mode); > > - void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, > - ControlList &metadata); > - void processFrameDuration(IPAContext &context, > - IPAFrameContext &frameContext, > - utils::Duration frameDuration); > - > std::map<int32_t, std::vector<uint8_t>> meteringModes_; > - AgcMeanLuminance agc_; > + AgcAlgorithm agc_; > }; > > } /* namespace ipa::rkisp1::algorithms */ > diff --git a/src/ipa/rkisp1/algorithms/lux.cpp b/src/ipa/rkisp1/algorithms/lux.cpp > index 86e46c492f..ce6928a55d 100644 > --- a/src/ipa/rkisp1/algorithms/lux.cpp > +++ b/src/ipa/rkisp1/algorithms/lux.cpp > @@ -74,7 +74,7 @@ void Lux::process(IPAContext &context, > if (!stats) > return; > > - utils::Duration exposureTime = context.configuration.sensor.lineDuration * > + utils::Duration exposureTime = context.configuration.agc.lineDuration * > frameContext.sensor.exposure; > double gain = frameContext.sensor.gain; > > diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp > index 1f94afda6b..47691674ad 100644 > --- a/src/ipa/rkisp1/ipa_context.cpp > +++ b/src/ipa/rkisp1/ipa_context.cpp > @@ -86,21 +86,6 @@ namespace libcamera::ipa::rkisp1 { > * \var IPASessionConfiguration::sensor > * \brief Sensor-specific configuration of the IPA > * > - * \var IPASessionConfiguration::sensor.minExposureTime > - * \brief Minimum exposure time supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.maxExposureTime > - * \brief Maximum exposure time supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.minAnalogueGain > - * \brief Minimum analogue gain supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.maxAnalogueGain > - * \brief Maximum analogue gain supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.lineDuration > - * \brief Line duration in microseconds > - * > * \var IPASessionConfiguration::sensor.size > * \brief Sensor output resolution > */ > @@ -147,49 +132,8 @@ namespace libcamera::ipa::rkisp1 { > * \var IPAActiveState::agc > * \brief State for the Automatic Gain Control algorithm > * > - * The \a automatic variables track the latest values computed by algorithm > - * based on the latest processed statistics. All other variables track the > - * consolidated controls requested in queued requests. > - * > - * \struct IPAActiveState::agc.manual > - * \brief Manual exposure time and analog gain (set through requests) > - * > - * \var IPAActiveState::agc.manual.exposure > - * \brief Manual exposure time expressed as a number of lines as set by the > - * ExposureTime control > - * > - * \var IPAActiveState::agc.manual.gain > - * \brief Manual analogue gain as set by the AnalogueGain control > - * > - * \struct IPAActiveState::agc.automatic > - * \brief Automatic exposure time and analog gain (computed by the algorithm) > - * > - * \var IPAActiveState::agc.automatic.exposure > - * \brief Automatic exposure time expressed as a number of lines > - * > - * \var IPAActiveState::agc.automatic.gain > - * \brief Automatic analogue gain multiplier > - * > - * \var IPAActiveState::agc.autoExposureEnabled > - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control > - * > - * \var IPAActiveState::agc.autoGainEnabled > - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control > - * > - * \var IPAActiveState::agc.constraintMode > - * \brief Constraint mode as set by the AeConstraintMode control > - * > - * \var IPAActiveState::agc.exposureMode > - * \brief Exposure mode as set by the AeExposureMode control > - * > * \var IPAActiveState::agc.meteringMode > * \brief Metering mode as set by the AeMeteringMode control > - * > - * \var IPAActiveState::agc.minFrameDuration > - * \brief Minimum frame duration as set by the FrameDurationLimits control > - * > - * \var IPAActiveState::agc.maxFrameDuration > - * \brief Maximum frame duration as set by the FrameDurationLimits control > */ > > /** > @@ -314,53 +258,11 @@ namespace libcamera::ipa::rkisp1 { > * the vertical blanking period is determined to maintain a consistent frame > * rate matched to the FrameDurationLimits as set by the user. > * > - * \var IPAFrameContext::agc.exposure > - * \brief Exposure time expressed as a number of lines computed by the algorithm > - * > - * \var IPAFrameContext::agc.gain > - * \brief Analogue gain multiplier computed by the algorithm > - * > - * The gain should be adapted to the sensor specific gain code before applying. > - * > - * \var IPAFrameContext::agc.vblank > - * \brief Vertical blanking parameter computed by the algorithm > - * > - * \var IPAFrameContext::agc.autoExposureEnabled > - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control > - * > - * \var IPAFrameContext::agc.autoGainEnabled > - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control > - * > - * \var IPAFrameContext::agc.constraintMode > - * \brief Constraint mode as set by the AeConstraintMode control > - * > - * \var IPAFrameContext::agc.exposureMode > - * \brief Exposure mode as set by the AeExposureMode control > - * > * \var IPAFrameContext::agc.meteringMode > * \brief Metering mode as set by the AeMeteringMode control > * > - * \var IPAFrameContext::agc.minFrameDuration > - * \brief Minimum frame duration as set by the FrameDurationLimits control > - * > - * \var IPAFrameContext::agc.maxFrameDuration > - * \brief Maximum frame duration as set by the FrameDurationLimits control > - * > - * \var IPAFrameContext::agc.frameDuration > - * \brief The actual FrameDuration used by the algorithm for the frame > - * > * \var IPAFrameContext::agc.updateMetering > * \brief Indicate if new ISP AGC metering parameters need to be applied > - * > - * \var IPAFrameContext::agc.autoExposureModeChange > - * \brief Indicate if autoExposureEnabled has changed from true in the previous > - * frame to false in the current frame, and no manual exposure value has been > - * supplied in the current frame. > - * > - * \var IPAFrameContext::agc.autoGainModeChange > - * \brief Indicate if autoGainEnabled has changed from true in the previous > - * frame to false in the current frame, and no manual gain value has been > - * supplied in the current frame. > */ > > /** > diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h > index cd213dd991..cc07bb9462 100644 > --- a/src/ipa/rkisp1/ipa_context.h > +++ b/src/ipa/rkisp1/ipa_context.h > @@ -24,7 +24,7 @@ > #include "libcamera/internal/matrix.h" > #include "libcamera/internal/vector.h" > > -#include "libipa/agc_mean_luminance.h" > +#include "libipa/agc.h" > #include "libipa/awb.h" > #include "libipa/camera_sensor_helper.h" > #include "libipa/ccm.h" > @@ -57,7 +57,7 @@ struct RKISP1AwbSession { > }; > > struct IPASessionConfiguration { > - struct { > + struct Agc : agc::Session { > struct rkisp1_cif_isp_window measureWindow; > } agc; > > @@ -68,12 +68,6 @@ struct IPASessionConfiguration { > } compress; > > struct { > - utils::Duration minExposureTime; > - utils::Duration maxExposureTime; > - double minAnalogueGain; > - double maxAnalogueGain; > - > - utils::Duration lineDuration; > Size size; > } sensor; > > @@ -82,26 +76,8 @@ struct IPASessionConfiguration { > }; > > struct IPAActiveState { > - struct { > - struct { > - uint32_t exposure; > - double gain; > - } manual; > - struct { > - uint32_t exposure; > - double gain; > - double quantizationGain; > - double yTarget; > - } automatic; > - > - bool autoExposureEnabled; > - bool autoGainEnabled; > - double exposureValue; > - controls::AeConstraintModeEnum constraintMode; > - controls::AeExposureModeEnum exposureMode; > + struct Agc : agc::ActiveState { > controls::AeMeteringModeEnum meteringMode; > - utils::Duration minFrameDuration; > - utils::Duration maxFrameDuration; > } agc; > > ipa::awb::ActiveState awb; > @@ -145,24 +121,9 @@ struct IPAActiveState { > }; > > struct IPAFrameContext : public FrameContext { > - struct { > - uint32_t exposure; > - double gain; > - double exposureValue; > - double quantizationGain; > - uint32_t vblank; > - double yTarget; > - bool autoExposureEnabled; > - bool autoGainEnabled; > - controls::AeConstraintModeEnum constraintMode; > - controls::AeExposureModeEnum exposureMode; > + struct Agc : agc::FrameContext { > controls::AeMeteringModeEnum meteringMode; > - utils::Duration minFrameDuration; > - utils::Duration maxFrameDuration; > - utils::Duration frameDuration; > bool updateMetering; > - bool autoExposureModeChange; > - bool autoGainModeChange; > } agc; > > ipa::awb::FrameContext awb; > -- > 2.55.0 >
Hi Barnabás On Mon, Aug 24, 2026 at 11:13:40AM +0200, Barnabás Pőcze wrote: > Add a class that implements the `Algorithm` interface using `AgcMeanLuminance` > based on the rkisp1 `Agc` algorithm, with the following main adjustments: > > * the parameters for `process()` have been made optional to handle > the cases where statistics are not available; > * the "raw" capture check has been replaced with the "autoAllowed" > session parameter; > * the controls are only provided after `configure()`. > > Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> > --- > src/ipa/libipa/agc.cpp | 729 +++++++++++++++++++++++++++++- > src/ipa/libipa/agc.h | 111 +++++ > src/ipa/rkisp1/algorithms/agc.cpp | 461 +++---------------- > src/ipa/rkisp1/algorithms/agc.h | 10 +- > src/ipa/rkisp1/algorithms/lux.cpp | 2 +- > src/ipa/rkisp1/ipa_context.cpp | 98 ---- > src/ipa/rkisp1/ipa_context.h | 47 +- > 7 files changed, 896 insertions(+), 562 deletions(-) > > diff --git a/src/ipa/libipa/agc.cpp b/src/ipa/libipa/agc.cpp > index 864b73d5f1..e311bfd0d6 100644 > --- a/src/ipa/libipa/agc.cpp > +++ b/src/ipa/libipa/agc.cpp > @@ -1,21 +1,41 @@ > /* SPDX-License-Identifier: LGPL-2.1-or-later */ > /* > - * Copyright (C) 2026 Ideas On Board > + * Copyright (C) 2021-2026 Ideas On Board > * > - * Auto exposure/gain algorithm for implementing the IPA-specific AGC algorithms > + * libIPA Agc algorithm Introduced in the previous patch and changed here ? > */ > > #include "agc.h" > > +#include <algorithm> > +#include <array> > +#include <chrono> > +#include <cmath> > +#include <optional> > +#include <ratio> > + > +#include <linux/v4l2-controls.h> > + > +#include <libcamera/base/log.h> > +#include <libcamera/base/span.h> > + > +#include <libcamera/control_ids.h> > +#include <libcamera/controls.h> > + > +#include <libcamera/ipa/core_ipa_interface.h> > + > /** > * \file agc.h > * \brief libipa AGC algorithm > */ > - Intentional ? > namespace libcamera { > > namespace ipa { > > +using namespace std::chrono_literals; > + > +LOG_DEFINE_CATEGORY(Agc) > + > namespace agc { > > /** > @@ -44,8 +64,711 @@ namespace agc { > * otherwise the gain value will be used directly. > */ > > +/** > + * \struct Session > + * \brief Session configuration for AgcAlgorithm > + * > + * \var Session::minExposureTime > + * \brief Minimum exposure time for the streaming session > + * > + * \var Session::maxExposureTime > + * \brief Maximum exposure time for the streaming session > + * > + * \var Session::minAnalogueGain > + * \brief Minimum analogue gain for the streaming session > + * > + * \var Session::maxAnalogueGain > + * \brief Maximum analogue gain for the streaming session > + * > + * \var Session::minFrameDuration > + * \brief Minimum frame duration for the streaming session > + * > + * \var Session::maxFrameDuration > + * \brief Maximum frame duration for the streaming session > + * > + * \var Session::lineDuration > + * \brief Line duration for the streaming session > + * > + * \var Session::sensor > + * \brief Details of the sensor configuration > + * > + * \var Session::sensor.outputSize > + * \brief Configured output size of the sensor > + * > + * \var Session::autoAllowed > + * \copybrief AgcAlgorithm::ConfigurationParams::autoAllowed > + * \sa AgcAlgorithm::ConfigurationParams::autoAllowed > + */ > + > +/** > + * \struct ActiveState > + * \brief Active state for AgcAlgorithm > + * > + * The \a automatic variables track the latest values computed by algorithm > + * based on the latest processed statistics. All other variables track the > + * consolidated controls requested in queued requests. > + * > + * \var ActiveState::manual > + * \brief Manual exposure time and analog gain (set through requests) > + * > + * \var ActiveState::manual.exposure > + * \brief Manual exposure time expressed as a number of lines as set by the > + * ExposureTime control > + * > + * \var ActiveState::manual.gain > + * \brief Manual analogue gain as set by the AnalogueGain control > + * > + * \var ActiveState::automatic > + * \brief Automatic exposure time and analog gain (computed by the algorithm) > + * > + * \var ActiveState::automatic.exposure > + * \brief Automatic exposure time expressed as a number of lines > + * > + * \var ActiveState::automatic.gain > + * \brief Automatic analogue gain multiplier > + * > + * \var ActiveState::automatic.quantizationGain > + * \brief Automatic quantization gain multiplier > + * > + * \var ActiveState::automatic.yTarget > + * \brief Automatically determined luminance target > + * > + * \var ActiveState::autoExposureEnabled > + * \brief Whether automatic exposure control is enabled by the ExposureTimeMode control > + * > + * \var ActiveState::autoGainEnabled > + * \brief Whether automatic gain control is enabled by the AnalogueGainMode control > + * > + * \var ActiveState::exposureValue > + * \brief Exposure value as set by the ExposureValue control > + * > + * \var ActiveState::constraintMode > + * \brief Constraint mode as set by the AeConstraintMode control > + * > + * \var ActiveState::exposureMode > + * \brief Exposure mode as set by the AeExposureMode control > + * > + * \var ActiveState::minFrameDuration > + * \brief Minimum frame duration as set by the FrameDurationLimits control > + * > + * \var ActiveState::maxFrameDuration > + * \brief Maximum frame duration as set by the FrameDurationLimits control > + */ > + > +/** > + * \struct FrameContext > + * \brief Per-frame context for AgcAlgorithm > + * > + * \var FrameContext::exposure > + * \brief Exposure time expressed as a number of lines computed by the algorithm > + * > + * \var FrameContext::gain > + * \brief Analogue gain multiplier computed by the algorithm > + * > + * The gain should be translated to the sensor specific gain code before applying. > + * > + * \var FrameContext::quantizationGain > + * \brief Quantization gain multiplier computed by the algorithm > + * > + * \var FrameContext::exposureValue > + * \brief Exposure value as set by the ExposureValue control > + * > + * \var FrameContext::yTarget > + * \brief Luminance target computed by the algorithm > + * > + * \var FrameContext::vblank > + * \brief Vertical blanking parameter computed by the algorithm > + * > + * \var FrameContext::autoExposureEnabled > + * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control > + * > + * \var FrameContext::autoGainEnabled > + * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control > + * > + * \var FrameContext::constraintMode > + * \brief Constraint mode as set by the AeConstraintMode control > + * > + * \var FrameContext::exposureMode > + * \brief Exposure mode as set by the AeExposureMode control > + * > + * \var FrameContext::minFrameDuration > + * \brief Minimum frame duration as set by the FrameDurationLimits control > + * > + * \var FrameContext::maxFrameDuration > + * \brief Maximum frame duration as set by the FrameDurationLimits control > + * > + * \var FrameContext::frameDuration > + * \brief The actual FrameDuration used by the algorithm for the frame > + * > + * \var FrameContext::autoExposureModeChange > + * \brief Indicate if autoExposureEnabled has changed from true in the previous > + * frame to false in the current frame, and no manual exposure value has been > + * supplied in the current frame > + * > + * \var FrameContext::autoGainModeChange > + * \brief Indicate if autoGainEnabled has changed from true in the previous > + * frame to false in the current frame, and no manual gain value has been > + * supplied in the current frame > + */ > + > } /* namespace agc */ > > +/** > + * \class AgcAlgorithm > + * \brief libIPA LSC algorithm algorithm > + * > + * The AgcAlgorithm class can be used to implement automatic exposure/gain > + * control in an IPA module, conforming to the prescribed Algorithm interface. > + * Internally AgcMeanLuminance is used, this class merely concerns itself with > + * processing the sensor properties, establishing limits, managing controls, > + * providing the resulting metadata, and driving the actual algorithm in > + * process(). > + * > + * Users should compose the agc::Session, agc::ActiveState, and agc::FrameContext > + * structures into their session configuration, active state, and frame contexts, > + * respectively. Furthermore, in their implementation of the Algorithm virtual > + * function, they should simply call the identically named member function of > + * AgcAlgorithm. > + * > + * \todo DigitalGain, DigitalGainMode > + * \todo Expand documentation > + */ > + > +/** > + * \struct AgcAlgorithm::ConfigurationParams > + * \brief Parameters for AgcAlgorithm::configure() > + * > + * \var AgcAlgorithm::ConfigurationParams::sensor > + * \brief CameraSensorHelper for the sensor > + * > + * \var AgcAlgorithm::ConfigurationParams::sensorInfo > + * \brief Current configuration of the sensor > + * > + * \var AgcAlgorithm::ConfigurationParams::sensorControls > + * \brief ControlInfoMap of the sensor > + * > + * \var AgcAlgorithm::ConfigurationParams::ctrlMap > + * \brief ControlInfoMap::Map to update with controls > + * > + * \var AgcAlgorithm::ConfigurationParams::autoAllowed > + * \brief Whether to enable auto controls > + * > + * If \a false, the algorithm is set up for manual exposure and gain > + * control only, without automatic adjustments. In this mode statistics > + * must not be provided to AgcAlgorithm::process(), and ExposureTimeMode > + * and AnalogueGainMode will only advertise manual control. > + */ > + > +/** > + * \struct AgcAlgorithm::ProcessParams > + * \brief Parameters for AgcAlgorithm::process() > + * > + * \var AgcAlgorithm::ProcessParams::traits > + * \brief Implementation of AgcMeanLuminance::Traits > + * > + * \var AgcAlgorithm::ProcessParams::yHist > + * \brief Luminance histogram of the frame > + * > + * \var AgcAlgorithm::ProcessParams::exposure > + * \brief Effective exposure of the frame > + * > + * \var AgcAlgorithm::ProcessParams::gain > + * \brief Effective gain of the frame > + * > + * \var AgcAlgorithm::ProcessParams::additionalConstraints > + * \brief Additional AgcMeanLuminance::AgcConstraints to apply > + * > + * \var AgcAlgorithm::ProcessParams::lux > + * \brief Effective lux value of the frame > + */ > + > +/** > + * \brief Load tuning data and configure > + * \param[in] tuningData The tuning data > + * \param[in] config The algorithm configuration > + * > + * This function loads the tuning data and configures the algorithm as if > + * by a call to configure(), in order to provide the available controls is "as if by" intentional ? > + * in ConfigurationParams::ctrlMap. > + * > + * The tuning data format is that of AgcMeanLuminance. Refer to is "the one" or "that" ? > + * AgcMeanLuminance::parseTuningData() for more details. > + * > + * \return 0 on success, or a negative error code > + * > + * \sa Algorithm::init() > + */ > +int AgcAlgorithm::init(const ValueNode &tuningData, const ConfigurationParams &config) > +{ > + int ret = impl_.parseTuningData(tuningData); > + if (ret) > + return ret; > + > + /* > + * The purpose of this `configure()` is merely to provide the > + * available controls in `config.ctrlMap`. > + * > + * \todo Remove it once IPA modules have been changed to > + * configure the algorithms during initialization. > + */ > + > + agc::Session dummySession; > + agc::ActiveState dummyState; > + This was different in the previous version, right ? > + return configure(dummySession, dummyState, config); > +} > + > +/** > + * \brief Initialize the session configuration and active state > + * \param[in] session The agc session configuration > + * \param[in] state The agc active state > + * \param[in] config The algorithm configuration > + * > + * This function initializes \a session and \a state based on the tuning > + * data loaded by init() and the configuration in \a config. > + * > + * It also updates ConfigurationParams::ctrlMap with the limits of the various > + * available agc-related controls. Users are expected to propagate these controls > + * to the camera. > + * > + * \return 0 on success, or a negative error code > + * > + * \sa Algorithm::configure() > + */ > +int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, > + const ConfigurationParams &config) > +{ > + session = {}; > + session.autoAllowed = config.autoAllowed; > + session.lineDuration = > + config.sensorInfo.minLineLength * 1.0s / config.sensorInfo.pixelRate; > + session.sensor.outputSize = config.sensorInfo.outputSize; > + > + const double lineDurationUs = session.lineDuration.get<std::micro>(); > + > + /* > + * Compute exposure time limits from the V4L2_CID_EXPOSURE control > + * limits and the line duration. > + */ > + > + const ControlInfo &v4l2Exposure = config.sensorControls.find(V4L2_CID_EXPOSURE)->second; > + int32_t minExposure = v4l2Exposure.min().get<int32_t>(); > + int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); > + int32_t defExposure = v4l2Exposure.def().get<int32_t>(); > + > + /* Compute the analogue gain limits. */ > + const ControlInfo &v4l2Gain = config.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; > + float minGain = config.sensor->gain(v4l2Gain.min().get<int32_t>()); > + float maxGain = config.sensor->gain(v4l2Gain.max().get<int32_t>()); > + float defGain = config.sensor->gain(v4l2Gain.def().get<int32_t>()); > + > + LOG(Agc, Debug) > + << "Exposure: [" << minExposure << ", " << maxExposure > + << "], gain: [" << minGain << ", " << maxGain << "]"; > + > + /* > + * Compute the frame duration limits. > + * > + * The frame length is computed assuming a fixed line length combined > + * with the vertical frame sizes. > + */ > + const ControlInfo &v4l2HBlank = config.sensorControls.find(V4L2_CID_HBLANK)->second; > + uint32_t hblank = v4l2HBlank.def().get<int32_t>(); > + uint32_t lineLength = config.sensorInfo.outputSize.width + hblank; > + > + const ControlInfo &v4l2VBlank = config.sensorControls.find(V4L2_CID_VBLANK)->second; > + std::array<uint32_t, 3> frameHeights{ > + v4l2VBlank.min().get<int32_t>() + config.sensorInfo.outputSize.height, > + v4l2VBlank.max().get<int32_t>() + config.sensorInfo.outputSize.height, > + v4l2VBlank.def().get<int32_t>() + config.sensorInfo.outputSize.height, > + }; > + > + std::array<int64_t, 3> frameDurations; > + for (unsigned int i = 0; i < frameHeights.size(); ++i) { > + uint64_t frameSize = lineLength * frameHeights[i]; > + frameDurations[i] = frameSize / (config.sensorInfo.pixelRate / 1000000U); > + } > + > + /* > + * When the AGC computes the new exposure values for a frame, it needs > + * to know the limits for exposure time and analogue gain. As it depends > + * on the sensor, update it with the controls. > + * > + * \todo take VBLANK into account for maximum exposure time > + */ > + session.minExposureTime = minExposure * session.lineDuration; > + session.maxExposureTime = maxExposure * session.lineDuration; > + session.minAnalogueGain = minGain; > + session.maxAnalogueGain = maxGain; > + session.minFrameDuration = std::chrono::microseconds(frameDurations[0]); > + session.maxFrameDuration = std::chrono::microseconds(frameDurations[1]); > + > + impl_.configure(session.lineDuration, config.sensor); > + impl_.setLimits(session.minExposureTime, session.maxExposureTime, > + session.minAnalogueGain, session.maxAnalogueGain, > + {}); > + impl_.resetFrameCount(); > + > + /* Configure the default exposure and gain. */ > + state = {}; > + state.automatic.gain = session.minAnalogueGain; > + state.automatic.exposure = 10ms / session.lineDuration; > + state.automatic.quantizationGain = 1; > + state.automatic.yTarget = impl_.effectiveYTarget(0, 1); > + state.manual.gain = state.automatic.gain; > + state.manual.exposure = state.automatic.exposure; > + state.autoExposureEnabled = session.autoAllowed; > + state.autoGainEnabled = session.autoAllowed; > + state.exposureValue = 0; > + state.constraintMode = > + static_cast<controls::AeConstraintModeEnum>(impl_.constraintModes().begin()->first); > + state.exposureMode = > + static_cast<controls::AeExposureModeEnum>(impl_.exposureModeHelpers().begin()->first); > + state.minFrameDuration = session.minFrameDuration; > + state.maxFrameDuration = session.maxFrameDuration; > + > + /* \todo Move this to the `Camera` class. */ > + config.ctrlMap[&controls::AeEnable] = ControlInfo{ > + false, session.autoAllowed, session.autoAllowed > + }; > + config.ctrlMap[&controls::AnalogueGain] = ControlInfo{ > + minGain, maxGain, defGain > + }; > + config.ctrlMap[&controls::ExposureTime] = ControlInfo{ > + static_cast<int32_t>(minExposure * lineDurationUs), > + static_cast<int32_t>(maxExposure * lineDurationUs), > + static_cast<int32_t>(defExposure * lineDurationUs), > + }; > + config.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ > + frameDurations[0], frameDurations[1], > + Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, > + }; > + config.ctrlMap[&controls::ExposureTimeMode] = ControlInfo{ > + {{ controls::ExposureTimeModeAuto, controls::ExposureTimeModeManual }}, > + controls::ExposureTimeModeAuto, > + }; > + config.ctrlMap[&controls::AnalogueGainMode] = ControlInfo{ > + {{ controls::AnalogueGainModeAuto, controls::AnalogueGainModeManual }}, > + controls::AnalogueGainModeAuto, > + }; > + config.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); > + config.ctrlMap.merge(impl_.controls()); > + > + return 0; > +} > + > +/** > + * \brief Queue a request > + * \param[in] session The agc session configuration > + * \param[in] state The agc active state > + * \param[in] frameContext The agc frame context > + * \param[in] controls The list of controls associated with a Request > + * > + * This functions processes the agc-related controls in \a controls for the frame > + * denoted by \a frameContext, and updates \a state and \a frameContext accordingly. > + * Reflow to 80 cols > + * \sa Algorithm::queueRequest() > + */ > +void AgcAlgorithm::queueRequest(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, const ControlList &controls) > +{ > + if (session.autoAllowed) { > + const auto &aeEnable = controls.get(controls::ExposureTimeMode); > + if (aeEnable && > + (*aeEnable == controls::ExposureTimeModeAuto) != state.autoExposureEnabled) { > + state.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); > + > + LOG(Agc, Debug) > + << (state.autoExposureEnabled ? "Enabling" : "Disabling") > + << " AGC (exposure)"; > + > + /* > + * If we go from auto -> manual with no manual control > + * set, use the last computed value, which we don't > + * know until prepare() so save this information. > + * > + * \todo Check the previous frame at prepare() time > + * instead of saving a flag here > + */ > + if (!state.autoExposureEnabled && !controls.get(controls::ExposureTime)) > + frameContext.autoExposureModeChange = true; > + } > + > + const auto &agEnable = controls.get(controls::AnalogueGainMode); > + if (agEnable && > + (*agEnable == controls::AnalogueGainModeAuto) != state.autoGainEnabled) { > + state.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); > + > + LOG(Agc, Debug) > + << (state.autoGainEnabled ? "Enabling" : "Disabling") > + << " AGC (gain)"; > + > + /* > + * If we go from auto -> manual with no manual control > + * set, use the last computed value, which we don't > + * know until prepare() so save this information. > + */ > + if (!state.autoGainEnabled && !controls.get(controls::AnalogueGain)) > + frameContext.autoGainModeChange = true; > + } > + } > + > + const auto &exposure = controls.get(controls::ExposureTime); > + if (exposure && !state.autoExposureEnabled) { > + state.manual.exposure = *exposure * 1.0us / session.lineDuration; > + > + LOG(Agc, Debug) << "Set exposure to " << state.manual.exposure; > + } > + > + const auto &gain = controls.get(controls::AnalogueGain); > + if (gain && !state.autoGainEnabled) { > + state.manual.gain = *gain; > + > + LOG(Agc, Debug) << "Set gain to " << state.manual.gain; > + } > + > + frameContext.autoExposureEnabled = state.autoExposureEnabled; > + frameContext.autoGainEnabled = state.autoGainEnabled; > + > + if (!frameContext.autoExposureEnabled) > + frameContext.exposure = state.manual.exposure; > + if (!frameContext.autoGainEnabled) > + frameContext.gain = state.manual.gain; > + > + if (!frameContext.autoExposureEnabled && !frameContext.autoGainEnabled) > + frameContext.quantizationGain = 1.0; > + > + const auto &exposureMode = controls.get(controls::AeExposureMode); > + if (exposureMode) > + state.exposureMode = > + static_cast<controls::AeExposureModeEnum>(*exposureMode); > + frameContext.exposureMode = state.exposureMode; > + > + const auto &constraintMode = controls.get(controls::AeConstraintMode); > + if (constraintMode) > + state.constraintMode = > + static_cast<controls::AeConstraintModeEnum>(*constraintMode); > + frameContext.constraintMode = state.constraintMode; > + > + const auto &exposureValue = controls.get(controls::ExposureValue); > + if (exposureValue) > + state.exposureValue = *exposureValue; > + frameContext.exposureValue = state.exposureValue; > + > + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); > + if (frameDurationLimits) { > + /* Limit the control value to the limits in ControlInfo */ > + state.minFrameDuration = std::clamp<utils::Duration>( > + std::chrono::microseconds((*frameDurationLimits).front()), > + session.minFrameDuration, session.maxFrameDuration); > + > + state.maxFrameDuration = std::clamp<utils::Duration>( > + std::chrono::microseconds((*frameDurationLimits).back()), > + session.minFrameDuration, session.maxFrameDuration); > + } > + frameContext.minFrameDuration = state.minFrameDuration; > + frameContext.maxFrameDuration = state.maxFrameDuration; > +} > + > +/** > + * \brief Prepare a frame > + * \param[in] state The agc active state > + * \param[in] frameContext The agc frame context > + * > + * This function prepares the parameters for the frame denoted by \a frameContext. > + * After a call to this function, the values of \ref agc::FrameContext::exposure > + * "frameContext.exposure" and \ref agc::FrameContext::gain "frameContext.gain" > + * will be finalized and may be used by the caller (see agc::prepareControls()). > + * > + * \todo Finalize \ref agc::FrameContext::vblank "frameContext.vblank" as well > + * > + * \sa Algorithm::prepare() Here and in the below function, reflow to 80 cols. With these minors fixed Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> Thanks j > + */ > +void AgcAlgorithm::prepare(agc::ActiveState &state, agc::FrameContext &frameContext) > +{ > + uint32_t activeAutoExposure = state.automatic.exposure; > + double activeAutoGain = state.automatic.gain; > + double activeAutoQGain = state.automatic.quantizationGain; > + > + /* Populate exposure and gain in auto mode */ > + if (frameContext.autoExposureEnabled) { > + frameContext.exposure = activeAutoExposure; > + frameContext.quantizationGain = activeAutoQGain; > + } > + if (frameContext.autoGainEnabled) { > + frameContext.gain = activeAutoGain; > + frameContext.quantizationGain = activeAutoQGain; > + } > + > + /* > + * Populate manual exposure and gain from the active auto values when > + * transitioning from auto to manual > + */ > + if (!frameContext.autoExposureEnabled && frameContext.autoExposureModeChange) { > + state.manual.exposure = activeAutoExposure; > + frameContext.exposure = activeAutoExposure; > + } > + if (!frameContext.autoGainEnabled && frameContext.autoGainModeChange) { > + state.manual.gain = activeAutoGain; > + frameContext.gain = activeAutoGain; > + frameContext.quantizationGain = activeAutoQGain; > + } > + > + frameContext.yTarget = state.automatic.yTarget; > +} > + > +/** > + * \brief Process frame statistics > + * \param[in] session The agc session configuration > + * \param[in] state The agc active state > + * \param[in] frameContext The agc frame context > + * \param[in] params The algorithm parameters > + * \param[in] metadata The list of metadata > + * > + * This function processes the statistics for the completed frame denoted by > + * \a frameContext, runs the AGC implementation, and updates \a state appropriately. > + * This function also populates \a metadata for the completed frame. > + * > + * \a params must be omitted if the session was configured without "autoAllowed", > + * and it may be omitted even if auto control is enabled, for example, if the > + * statistics could not be delivered due to some ephemeral error. This ensures > + * that the algorithm state will not go out of sync, and that metadata is produced > + * as expected. > + * > + * Care must be taken to convert the platform specific statistics to the format > + * expected in \a params. See ProcessParams for the details. > + * > + * \sa Algorithm::process() > + */ > +void AgcAlgorithm::process(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, > + ControlList &metadata) > +{ > + if (!params) { > + processFrameDuration(session, frameContext, frameContext.minFrameDuration); > + fillMetadata(session, frameContext, metadata); > + return; > + } > + > + ASSERT(session.autoAllowed); > + > + const utils::Duration &lineDuration = session.lineDuration; > + > + /* > + * Set the AGC limits using the fixed exposure time and/or gain in > + * manual mode, or the sensor limits in auto mode. > + */ > + utils::Duration minExposureTime; > + utils::Duration maxExposureTime; > + double minAnalogueGain; > + double maxAnalogueGain; > + > + /* \todo This uses the configuration from an already completed frame. */ > + > + if (frameContext.autoExposureEnabled) { > + minExposureTime = session.minExposureTime; > + maxExposureTime = std::clamp(frameContext.maxFrameDuration, > + session.minExposureTime, > + session.maxExposureTime); > + } else { > + minExposureTime = lineDuration * frameContext.exposure; > + maxExposureTime = minExposureTime; > + } > + > + if (frameContext.autoGainEnabled) { > + minAnalogueGain = session.minAnalogueGain; > + maxAnalogueGain = session.maxAnalogueGain; > + } else { > + minAnalogueGain = frameContext.gain; > + maxAnalogueGain = frameContext.gain; > + } > + > + /* > + * The Agc algorithm needs to know the effective exposure value that was > + * applied to the sensor when the statistics were collected. > + */ > + utils::Duration effectiveExposureValue = > + lineDuration * params->exposure * params->gain; > + > + impl_.setLimits(minExposureTime, maxExposureTime, > + minAnalogueGain, maxAnalogueGain, > + std::move(params->additionalConstraints)); > + > + const auto &newEv = impl_.calculateNewEv({ > + .traits = params->traits, > + .yHist = params->yHist, > + .effectiveExposureValue = effectiveExposureValue, > + .constraintModeIndex = frameContext.constraintMode, > + .exposureModeIndex = frameContext.exposureMode, > + .lux = params->lux, > + .exposureCompensation = std::pow(2.0, frameContext.exposureValue), > + }); > + > + /* Update the estimated exposure and gain. */ > + state.automatic.exposure = newEv.exposureTime / lineDuration; > + state.automatic.gain = newEv.analogueGain; > + state.automatic.quantizationGain = newEv.quantizationGain; > + state.automatic.yTarget = newEv.yTarget; > + > + LOG(Agc, Debug) > + << "Divided up exposure time, analogue gain, quantization gain" > + << " and digital gain are " << newEv.exposureTime > + << ", " << state.automatic.gain << ", " << state.automatic.quantizationGain > + << " and " << newEv.digitalGain; > + > + /* > + * Expand the target frame duration so that we do not run faster than > + * the minimum frame duration when we have short exposures. > + */ > + processFrameDuration(session, frameContext, > + std::max(frameContext.minFrameDuration, newEv.exposureTime)); > + > + fillMetadata(session, frameContext, metadata); > +} > + > +/** > + * \brief Process frame duration and compute vblank > + * \param[in] session The session parameters > + * \param[in] frameContext The current frame context > + * \param[in] frameDuration The target frame duration > + * > + * Compute and populate vblank from the target frame duration. > + */ > +void AgcAlgorithm::processFrameDuration(const agc::Session &session, > + agc::FrameContext &frameContext, > + utils::Duration frameDuration) > +{ > + const utils::Duration &lineDuration = session.lineDuration; > + > + frameContext.vblank = > + (frameDuration / lineDuration) - session.sensor.outputSize.height; > + > + /* Update frame duration accounting for line length quantization. */ > + frameContext.frameDuration = > + (session.sensor.outputSize.height + frameContext.vblank) * lineDuration; > +} > + > +void AgcAlgorithm::fillMetadata(const agc::Session &session, > + const agc::FrameContext &frameContext, > + ControlList &metadata) > +{ > + > + metadata.set(controls::AnalogueGain, frameContext.gain); > + metadata.set(controls::ExposureTime, > + utils::Duration(session.lineDuration * frameContext.exposure).get<std::micro>()); > + metadata.set(controls::FrameDuration, frameContext.frameDuration.get<std::micro>()); > + metadata.set(controls::ExposureTimeMode, frameContext.autoExposureEnabled > + ? controls::ExposureTimeModeAuto > + : controls::ExposureTimeModeManual); > + metadata.set(controls::AnalogueGainMode, frameContext.autoGainEnabled > + ? controls::AnalogueGainModeAuto > + : controls::AnalogueGainModeManual); > + > + metadata.set(controls::AeExposureMode, frameContext.exposureMode); > + metadata.set(controls::AeConstraintMode, frameContext.constraintMode); > + metadata.set(controls::ExposureValue, frameContext.exposureValue); > +} > + > } /* namespace ipa */ > > } /* namespace libcamera */ > diff --git a/src/ipa/libipa/agc.h b/src/ipa/libipa/agc.h > index 5a67464086..06600f0208 100644 > --- a/src/ipa/libipa/agc.h > +++ b/src/ipa/libipa/agc.h > @@ -7,21 +7,87 @@ > > #pragma once > > +#include <optional> > #include <stdint.h> > #include <utility> > +#include <vector> > > #include <linux/v4l2-controls.h> > > +#include <libcamera/base/utils.h> > + > +#include <libcamera/control_ids.h> > #include <libcamera/controls.h> > +#include <libcamera/geometry.h> > > +#include "agc_mean_luminance.h" > #include "camera_sensor_helper.h" > > namespace libcamera { > > +struct IPACameraSensorInfo; > + > namespace ipa { > > +class Histogram; > + > namespace agc { > > +struct Session { > + utils::Duration minExposureTime; > + utils::Duration maxExposureTime; > + double minAnalogueGain; > + double maxAnalogueGain; > + utils::Duration minFrameDuration; > + utils::Duration maxFrameDuration; > + utils::Duration lineDuration; > + > + struct { > + Size outputSize; > + } sensor; > + > + bool autoAllowed; > +}; > + > +struct ActiveState { > + struct { > + uint32_t exposure; > + double gain; > + } manual; > + struct { > + uint32_t exposure; > + double gain; > + double quantizationGain; > + double yTarget; > + } automatic; > + > + bool autoExposureEnabled; > + bool autoGainEnabled; > + double exposureValue; > + controls::AeConstraintModeEnum constraintMode; > + controls::AeExposureModeEnum exposureMode; > + utils::Duration minFrameDuration; > + utils::Duration maxFrameDuration; > +}; > + > +struct FrameContext { > + uint32_t exposure; > + double gain; > + double quantizationGain; > + double exposureValue; > + double yTarget; > + uint32_t vblank; > + bool autoExposureEnabled; > + bool autoGainEnabled; > + controls::AeConstraintModeEnum constraintMode; > + controls::AeExposureModeEnum exposureMode; > + utils::Duration minFrameDuration; > + utils::Duration maxFrameDuration; > + utils::Duration frameDuration; > + bool autoExposureModeChange; > + bool autoGainModeChange; > +}; > + > [[nodiscard]] > inline std::pair<uint32_t, double> > extractControls(const ControlList &controls, const CameraSensorHelper *sensor) > @@ -48,6 +114,51 @@ prepareControls(ControlList &controls, const CameraSensorHelper *sensor, > > } /* namespace agc */ > > +class AgcAlgorithm > +{ > +public: > + struct ConfigurationParams { > + const CameraSensorHelper *sensor; > + const IPACameraSensorInfo &sensorInfo; > + const ControlInfoMap &sensorControls; > + ControlInfoMap::Map &ctrlMap; > + bool autoAllowed = true; > + }; > + > + struct ProcessParams { > + const AgcMeanLuminance::Traits &traits; > + const Histogram &yHist; > + uint32_t exposure; > + double gain; > + std::vector<AgcMeanLuminance::AgcConstraint> &&additionalConstraints = {}; > + double lux = 0; > + }; > + > + int init(const ValueNode &tuningData, const ConfigurationParams &config); > + > + int configure(agc::Session &session, agc::ActiveState &state, > + const ConfigurationParams &config); > + > + void queueRequest(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, const ControlList &controls); > + > + void prepare(agc::ActiveState &state, agc::FrameContext &frameContext); > + > + void process(const agc::Session &session, agc::ActiveState &state, > + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, > + ControlList &metadata); > + > +private: > + void processFrameDuration(const agc::Session &session, > + agc::FrameContext &frameContext, > + utils::Duration frameDuration); > + void fillMetadata(const agc::Session &session, > + const agc::FrameContext &frameContext, > + ControlList &metadata); > + > + AgcMeanLuminance impl_; > +}; > + > } /* namespace ipa */ > > } /* namespace libcamera */ > diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp > index 7230232d24..bb0345ca87 100644 > --- a/src/ipa/rkisp1/algorithms/agc.cpp > +++ b/src/ipa/rkisp1/algorithms/agc.cpp > @@ -8,10 +8,7 @@ > #include "agc.h" > > #include <algorithm> > -#include <array> > -#include <chrono> > #include <cmath> > -#include <tuple> > #include <vector> > > #include <libcamera/base/log.h> > @@ -36,89 +33,6 @@ namespace ipa::rkisp1::algorithms { > > LOG_DEFINE_CATEGORY(RkISP1Agc) > > -namespace { > - > -void reconfigure(IPAContext &context) > -{ > - context.configuration.sensor.lineDuration = > - context.sensorInfo.minLineLength * 1.0s / context.sensorInfo.pixelRate; > - > - double lineDurationUs = context.configuration.sensor.lineDuration.get<std::micro>(); > - > - /* > - * Compute exposure time limits from the V4L2_CID_EXPOSURE control > - * limits and the line duration. > - */ > - > - const ControlInfo &v4l2Exposure = context.sensorControls.find(V4L2_CID_EXPOSURE)->second; > - int32_t minExposure = v4l2Exposure.min().get<int32_t>(); > - int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); > - int32_t defExposure = v4l2Exposure.def().get<int32_t>(); > - context.ctrlMap[&controls::ExposureTime] = ControlInfo{ > - static_cast<int32_t>(minExposure * lineDurationUs), > - static_cast<int32_t>(maxExposure * lineDurationUs), > - static_cast<int32_t>(defExposure * lineDurationUs), > - }; > - > - /* Compute the analogue gain limits. */ > - const ControlInfo &v4l2Gain = context.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; > - float minGain = context.camHelper->gain(v4l2Gain.min().get<int32_t>()); > - float maxGain = context.camHelper->gain(v4l2Gain.max().get<int32_t>()); > - float defGain = context.camHelper->gain(v4l2Gain.def().get<int32_t>()); > - context.ctrlMap[&controls::AnalogueGain] = ControlInfo{ > - minGain, > - maxGain, > - defGain, > - }; > - > - LOG(RkISP1Agc, Debug) > - << "Exposure: [" << minExposure << ", " << maxExposure > - << "], gain: [" << minGain << ", " << maxGain << "]"; > - > - /* > - * Compute the frame duration limits. > - * > - * The frame length is computed assuming a fixed line length combined > - * with the vertical frame sizes. > - */ > - const ControlInfo &v4l2HBlank = context.sensorControls.find(V4L2_CID_HBLANK)->second; > - uint32_t hblank = v4l2HBlank.def().get<int32_t>(); > - uint32_t lineLength = context.sensorInfo.outputSize.width + hblank; > - > - const ControlInfo &v4l2VBlank = context.sensorControls.find(V4L2_CID_VBLANK)->second; > - std::array<uint32_t, 3> frameHeights{ > - v4l2VBlank.min().get<int32_t>() + context.sensorInfo.outputSize.height, > - v4l2VBlank.max().get<int32_t>() + context.sensorInfo.outputSize.height, > - v4l2VBlank.def().get<int32_t>() + context.sensorInfo.outputSize.height, > - }; > - > - std::array<int64_t, 3> frameDurations; > - for (unsigned int i = 0; i < frameHeights.size(); ++i) { > - uint64_t frameSize = lineLength * frameHeights[i]; > - frameDurations[i] = frameSize / (context.sensorInfo.pixelRate / 1000000U); > - } > - > - context.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ > - frameDurations[0], > - frameDurations[1], > - Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, > - }; > - > - /* > - * When the AGC computes the new exposure values for a frame, it needs > - * to know the limits for exposure time and analogue gain. As it depends > - * on the sensor, update it with the controls. > - * > - * \todo take VBLANK into account for maximum exposure time > - */ > - context.configuration.sensor.minExposureTime = minExposure * context.configuration.sensor.lineDuration; > - context.configuration.sensor.maxExposureTime = maxExposure * context.configuration.sensor.lineDuration; > - context.configuration.sensor.minAnalogueGain = minGain; > - context.configuration.sensor.maxAnalogueGain = maxGain; > -} > - > -} /* namespace */ > - > /** > * \class Agc > * \brief A mean-based auto-exposure algorithm > @@ -222,7 +136,12 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) > { > int ret; > > - ret = agc_.parseTuningData(tuningData); > + ret = agc_.init(tuningData, { > + .sensor = context.camHelper.get(), > + .sensorInfo = context.sensorInfo, > + .sensorControls = context.sensorControls, > + .ctrlMap = context.ctrlMap, > + }); > if (ret) > return ret; > > @@ -231,21 +150,6 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) > if (ret) > return ret; > > - context.ctrlMap[&controls::ExposureTimeMode] = > - ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), > - ControlValue(controls::ExposureTimeModeManual) } }, > - ControlValue(controls::ExposureTimeModeAuto)); > - context.ctrlMap[&controls::AnalogueGainMode] = > - ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), > - ControlValue(controls::AnalogueGainModeManual) } }, > - ControlValue(controls::AnalogueGainModeAuto)); > - /* \todo Move this to the Camera class */ > - context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true, true); > - context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); > - context.ctrlMap.merge(agc_.controls()); > - > - reconfigure(context); > - > return 0; > } > > @@ -258,47 +162,24 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) > */ > int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) > { > - reconfigure(context); > - > - /* Configure the default exposure and gain. */ > - context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; > - context.activeState.agc.automatic.exposure = > - 10ms / context.configuration.sensor.lineDuration; > - context.activeState.agc.automatic.quantizationGain = 1.0; > - context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; > - context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; > - context.activeState.agc.autoExposureEnabled = !context.configuration.raw; > - context.activeState.agc.autoGainEnabled = !context.configuration.raw; > - context.activeState.agc.exposureValue = 0.0; > - > - context.activeState.agc.constraintMode = > - static_cast<controls::AeConstraintModeEnum>(agc_.constraintModes().begin()->first); > - context.activeState.agc.exposureMode = > - static_cast<controls::AeExposureModeEnum>(agc_.exposureModeHelpers().begin()->first); > + int ret = agc_.configure(context.configuration.agc, context.activeState.agc, { > + .sensor = context.camHelper.get(), > + .sensorInfo = context.sensorInfo, > + .sensorControls = context.sensorControls, > + .ctrlMap = context.ctrlMap, > + .autoAllowed = !context.configuration.raw, > + }); > + if (ret) > + return ret; > + > context.activeState.agc.meteringMode = > static_cast<controls::AeMeteringModeEnum>(meteringModes_.begin()->first); > > - /* Limit the frame duration to match current initialisation */ > - ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; > - context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get<int64_t>()); > - context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get<int64_t>()); > - > context.configuration.agc.measureWindow.h_offs = 0; > context.configuration.agc.measureWindow.v_offs = 0; > context.configuration.agc.measureWindow.h_size = configInfo.outputSize.width; > context.configuration.agc.measureWindow.v_size = configInfo.outputSize.height; > > - agc_.configure(context.configuration.sensor.lineDuration, context.camHelper.get()); > - > - agc_.setLimits(context.configuration.sensor.minExposureTime, > - context.configuration.sensor.maxExposureTime, > - context.configuration.sensor.minAnalogueGain, > - context.configuration.sensor.maxAnalogueGain, {}); > - > - context.activeState.agc.automatic.yTarget = agc_.effectiveYTarget(0, 1); > - > - agc_.resetFrameCount(); > - > return 0; > } > > @@ -312,73 +193,7 @@ void Agc::queueRequest(IPAContext &context, > { > auto &agc = context.activeState.agc; > > - if (!context.configuration.raw) { > - const auto &aeEnable = controls.get(controls::ExposureTimeMode); > - if (aeEnable && > - (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { > - agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); > - > - LOG(RkISP1Agc, Debug) > - << (agc.autoExposureEnabled ? "Enabling" : "Disabling") > - << " AGC (exposure)"; > - > - /* > - * If we go from auto -> manual with no manual control > - * set, use the last computed value, which we don't > - * know until prepare() so save this information. > - * > - * \todo Check the previous frame at prepare() time > - * instead of saving a flag here > - */ > - if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) > - frameContext.agc.autoExposureModeChange = true; > - } > - > - const auto &agEnable = controls.get(controls::AnalogueGainMode); > - if (agEnable && > - (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { > - agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); > - > - LOG(RkISP1Agc, Debug) > - << (agc.autoGainEnabled ? "Enabling" : "Disabling") > - << " AGC (gain)"; > - /* > - * If we go from auto -> manual with no manual control > - * set, use the last computed value, which we don't > - * know until prepare() so save this information. > - */ > - if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) > - frameContext.agc.autoGainModeChange = true; > - } > - } > - > - const auto &exposure = controls.get(controls::ExposureTime); > - if (exposure && !agc.autoExposureEnabled) { > - agc.manual.exposure = *exposure * 1.0us > - / context.configuration.sensor.lineDuration; > - > - LOG(RkISP1Agc, Debug) > - << "Set exposure to " << agc.manual.exposure; > - } > - > - const auto &gain = controls.get(controls::AnalogueGain); > - if (gain && !agc.autoGainEnabled) { > - agc.manual.gain = *gain; > - > - LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; > - } > - > - frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; > - frameContext.agc.autoGainEnabled = agc.autoGainEnabled; > - > - if (!frameContext.agc.autoExposureEnabled) > - frameContext.agc.exposure = agc.manual.exposure; > - if (!frameContext.agc.autoGainEnabled) > - frameContext.agc.gain = agc.manual.gain; > - > - if (!frameContext.agc.autoExposureEnabled && > - !frameContext.agc.autoGainEnabled) > - frameContext.agc.quantizationGain = 1.0; > + agc_.queueRequest(context.configuration.agc, agc, frameContext.agc, controls); > > const auto &meteringMode = controls.get(controls::AeMeteringMode); > if (meteringMode) { > @@ -387,42 +202,6 @@ void Agc::queueRequest(IPAContext &context, > static_cast<controls::AeMeteringModeEnum>(*meteringMode); > } > frameContext.agc.meteringMode = agc.meteringMode; > - > - const auto &exposureMode = controls.get(controls::AeExposureMode); > - if (exposureMode) > - agc.exposureMode = > - static_cast<controls::AeExposureModeEnum>(*exposureMode); > - frameContext.agc.exposureMode = agc.exposureMode; > - > - const auto &constraintMode = controls.get(controls::AeConstraintMode); > - if (constraintMode) > - agc.constraintMode = > - static_cast<controls::AeConstraintModeEnum>(*constraintMode); > - frameContext.agc.constraintMode = agc.constraintMode; > - > - const auto &exposureValue = controls.get(controls::ExposureValue); > - if (exposureValue) > - agc.exposureValue = *exposureValue; > - frameContext.agc.exposureValue = agc.exposureValue; > - > - const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); > - if (frameDurationLimits) { > - /* Limit the control value to the limits in ControlInfo */ > - ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; > - int64_t minFrameDuration = > - std::clamp((*frameDurationLimits).front(), > - limits.min().get<int64_t>(), > - limits.max().get<int64_t>()); > - int64_t maxFrameDuration = > - std::clamp((*frameDurationLimits).back(), > - limits.min().get<int64_t>(), > - limits.max().get<int64_t>()); > - > - agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); > - agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); > - } > - frameContext.agc.minFrameDuration = agc.minFrameDuration; > - frameContext.agc.maxFrameDuration = agc.maxFrameDuration; > } > > /** > @@ -431,41 +210,13 @@ void Agc::queueRequest(IPAContext &context, > void Agc::prepare(IPAContext &context, const uint32_t frame, > IPAFrameContext &frameContext, RkISP1Params *params) > { > - uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; > - double activeAutoGain = context.activeState.agc.automatic.gain; > - double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; > - > - /* Populate exposure and gain in auto mode */ > - if (frameContext.agc.autoExposureEnabled) { > - frameContext.agc.exposure = activeAutoExposure; > - frameContext.agc.quantizationGain = activeAutoQGain; > - } > - if (frameContext.agc.autoGainEnabled) { > - frameContext.agc.gain = activeAutoGain; > - frameContext.agc.quantizationGain = activeAutoQGain; > - } > - > - /* > - * Populate manual exposure and gain from the active auto values when > - * transitioning from auto to manual > - */ > - if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { > - context.activeState.agc.manual.exposure = activeAutoExposure; > - frameContext.agc.exposure = activeAutoExposure; > - } > - if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { > - context.activeState.agc.manual.gain = activeAutoGain; > - frameContext.agc.gain = activeAutoGain; > - frameContext.agc.quantizationGain = activeAutoQGain; > - } > + agc_.prepare(context.activeState.agc, frameContext.agc); > > if (context.configuration.compress.supported) { > frameContext.compress.enable = true; > frameContext.compress.gain = frameContext.agc.quantizationGain; > } > > - frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; > - > if (frame > 0 && !frameContext.agc.updateMetering) > return; > > @@ -521,50 +272,6 @@ void Agc::prepare(IPAContext &context, const uint32_t frame, > static_cast<rkisp1_cif_isp_histogram_mode>(hstConfig->mode)); > } > > -void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, > - ControlList &metadata) > -{ > - utils::Duration exposureTime = context.configuration.sensor.lineDuration > - * frameContext.sensor.exposure; > - metadata.set(controls::AnalogueGain, frameContext.sensor.gain); > - metadata.set(controls::ExposureTime, exposureTime.get<std::micro>()); > - metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get<std::micro>()); > - metadata.set(controls::ExposureTimeMode, > - frameContext.agc.autoExposureEnabled > - ? controls::ExposureTimeModeAuto > - : controls::ExposureTimeModeManual); > - metadata.set(controls::AnalogueGainMode, > - frameContext.agc.autoGainEnabled > - ? controls::AnalogueGainModeAuto > - : controls::AnalogueGainModeManual); > - > - metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); > - metadata.set(controls::AeExposureMode, frameContext.agc.exposureMode); > - metadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode); > - metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); > -} > - > -/** > - * \brief Process frame duration and compute vblank > - * \param[in] context The shared IPA context > - * \param[in] frameContext The current frame context > - * \param[in] frameDuration The target frame duration > - * > - * Compute and populate vblank from the target frame duration. > - */ > -void Agc::processFrameDuration(IPAContext &context, > - IPAFrameContext &frameContext, > - utils::Duration frameDuration) > -{ > - IPACameraSensorInfo &sensorInfo = context.sensorInfo; > - utils::Duration lineDuration = context.configuration.sensor.lineDuration; > - > - frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; > - > - /* Update frame duration accounting for line length quantization. */ > - frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; > -} > - > namespace { > > class AgcTraits final : public AgcMeanLuminance::Traits > @@ -638,21 +345,6 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, > IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats, > ControlList &metadata) > { > - if (!stats) { > - processFrameDuration(context, frameContext, > - frameContext.agc.minFrameDuration); > - fillMetadata(context, frameContext, metadata); > - return; > - } > - > - if (!(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP)) { > - fillMetadata(context, frameContext, metadata); > - LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; > - return; > - } > - > - const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; > - > /* > * \todo Verify that the exposure and gain applied by the sensor for > * this frame match what has been requested. This isn't a hard > @@ -661,95 +353,46 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, > * we receive), but is important in manual mode. > */ > > - const rkisp1_cif_isp_stat *params = &stats->params; > + const rkisp1_cif_isp_stat *params = nullptr; > > - /* > - * Set the AGC limits using the fixed exposure time and/or gain in > - * manual mode, or the sensor limits in auto mode. > - */ > - utils::Duration minExposureTime; > - utils::Duration maxExposureTime; > - double minAnalogueGain; > - double maxAnalogueGain; > - > - if (frameContext.agc.autoExposureEnabled) { > - minExposureTime = context.configuration.sensor.minExposureTime; > - maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, > - context.configuration.sensor.minExposureTime, > - context.configuration.sensor.maxExposureTime); > - } else { > - minExposureTime = context.configuration.sensor.lineDuration > - * frameContext.agc.exposure; > - maxExposureTime = minExposureTime; > + if (stats) { > + if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) > + params = &stats->params; > + else > + LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; > } > > - if (frameContext.agc.autoGainEnabled) { > - minAnalogueGain = context.configuration.sensor.minAnalogueGain; > - maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; > + if (params) { > + std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; > + if (context.activeState.wdr.mode != controls::WdrOff) > + additionalConstraints.push_back(context.activeState.wdr.constraint); > + > + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {{ > + .traits = AgcTraits{ > + { params->ae.exp_mean, context.hw.numAeCells }, > + meteringModes_.at(frameContext.agc.meteringMode), > + }, > + .yHist = { > + /* The lower 4 bits are fractional and meant to be discarded. */ > + { params->hist.hist_bins, context.hw.numHistogramBins }, > + [](uint32_t x) { return x >> 4; }, > + }, > + .exposure = frameContext.sensor.exposure, > + /* > + * Include the quantization gain if it was applied. Do not use > + * compress.gain because it will include gains that shall not be > + * reported to the user when HDR is implemented. > + */ > + .gain = frameContext.sensor.gain > + * (frameContext.compress.enable ? frameContext.agc.quantizationGain : 1), > + .additionalConstraints = std::move(additionalConstraints), > + .lux = frameContext.lux.lux, > + }}, metadata); > } else { > - minAnalogueGain = frameContext.agc.gain; > - maxAnalogueGain = frameContext.agc.gain; > + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {}, metadata); > } > > - std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; > - if (context.activeState.wdr.mode != controls::WdrOff) > - additionalConstraints.push_back(context.activeState.wdr.constraint); > - > - agc_.setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, > - std::move(additionalConstraints)); > - > - /* > - * The Agc algorithm needs to know the effective exposure value that was > - * applied to the sensor when the statistics were collected. > - */ > - utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; > - double analogueGain = frameContext.sensor.gain; > - utils::Duration effectiveExposureValue = exposureTime * analogueGain; > - > - /* > - * Include the quantization gain if it was applied. Do not use > - * compress.gain because it will include gains that shall not be > - * reported to the user when HDR is implemented. > - */ > - if (frameContext.compress.enable) > - effectiveExposureValue *= frameContext.agc.quantizationGain; > - > - /* The lower 4 bits are fractional and meant to be discarded. */ > - Histogram hist({ params->hist.hist_bins, context.hw.numHistogramBins }, > - [](uint32_t x) { return x >> 4; }); > - > - const auto &newEv = agc_.calculateNewEv({ > - .traits = AgcTraits{ > - { params->ae.exp_mean, context.hw.numAeCells }, > - meteringModes_.at(frameContext.agc.meteringMode), > - }, > - .yHist = hist, > - .effectiveExposureValue = effectiveExposureValue, > - .constraintModeIndex = frameContext.agc.constraintMode, > - .exposureModeIndex = frameContext.agc.exposureMode, > - .lux = frameContext.lux.lux, > - .exposureCompensation = pow(2.0, frameContext.agc.exposureValue), > - }); > - > - LOG(RkISP1Agc, Debug) > - << "Divided up exposure time, analogue gain, quantization gain" > - << " and digital gain are " << newEv.exposureTime << ", " << newEv.analogueGain > - << ", " << newEv.quantizationGain << " and " << newEv.digitalGain; > - > - IPAActiveState &activeState = context.activeState; > - /* Update the estimated exposure and gain. */ > - activeState.agc.automatic.exposure = newEv.exposureTime / lineDuration; > - activeState.agc.automatic.gain = newEv.analogueGain; > - activeState.agc.automatic.quantizationGain = newEv.quantizationGain; > - activeState.agc.automatic.yTarget = newEv.yTarget; > - /* > - * Expand the target frame duration so that we do not run faster than > - * the minimum frame duration when we have short exposures. > - */ > - processFrameDuration(context, frameContext, > - std::max(frameContext.agc.minFrameDuration, newEv.exposureTime)); > - > - fillMetadata(context, frameContext, metadata); > + metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); > } > > REGISTER_IPA_ALGORITHM(Agc, "Agc") > diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h > index 0527ca0d5f..3a4d7bc546 100644 > --- a/src/ipa/rkisp1/algorithms/agc.h > +++ b/src/ipa/rkisp1/algorithms/agc.h > @@ -14,7 +14,7 @@ > > #include <libcamera/geometry.h> > > -#include "libipa/agc_mean_luminance.h" > +#include "libipa/agc.h" > > #include "algorithm.h" > > @@ -47,14 +47,8 @@ private: > uint8_t computeHistogramPredivider(const Size &size, > enum rkisp1_cif_isp_histogram_mode mode); > > - void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, > - ControlList &metadata); > - void processFrameDuration(IPAContext &context, > - IPAFrameContext &frameContext, > - utils::Duration frameDuration); > - > std::map<int32_t, std::vector<uint8_t>> meteringModes_; > - AgcMeanLuminance agc_; > + AgcAlgorithm agc_; > }; > > } /* namespace ipa::rkisp1::algorithms */ > diff --git a/src/ipa/rkisp1/algorithms/lux.cpp b/src/ipa/rkisp1/algorithms/lux.cpp > index 86e46c492f..ce6928a55d 100644 > --- a/src/ipa/rkisp1/algorithms/lux.cpp > +++ b/src/ipa/rkisp1/algorithms/lux.cpp > @@ -74,7 +74,7 @@ void Lux::process(IPAContext &context, > if (!stats) > return; > > - utils::Duration exposureTime = context.configuration.sensor.lineDuration * > + utils::Duration exposureTime = context.configuration.agc.lineDuration * > frameContext.sensor.exposure; > double gain = frameContext.sensor.gain; > > diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp > index 1f94afda6b..47691674ad 100644 > --- a/src/ipa/rkisp1/ipa_context.cpp > +++ b/src/ipa/rkisp1/ipa_context.cpp > @@ -86,21 +86,6 @@ namespace libcamera::ipa::rkisp1 { > * \var IPASessionConfiguration::sensor > * \brief Sensor-specific configuration of the IPA > * > - * \var IPASessionConfiguration::sensor.minExposureTime > - * \brief Minimum exposure time supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.maxExposureTime > - * \brief Maximum exposure time supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.minAnalogueGain > - * \brief Minimum analogue gain supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.maxAnalogueGain > - * \brief Maximum analogue gain supported with the sensor > - * > - * \var IPASessionConfiguration::sensor.lineDuration > - * \brief Line duration in microseconds > - * > * \var IPASessionConfiguration::sensor.size > * \brief Sensor output resolution > */ > @@ -147,49 +132,8 @@ namespace libcamera::ipa::rkisp1 { > * \var IPAActiveState::agc > * \brief State for the Automatic Gain Control algorithm > * > - * The \a automatic variables track the latest values computed by algorithm > - * based on the latest processed statistics. All other variables track the > - * consolidated controls requested in queued requests. > - * > - * \struct IPAActiveState::agc.manual > - * \brief Manual exposure time and analog gain (set through requests) > - * > - * \var IPAActiveState::agc.manual.exposure > - * \brief Manual exposure time expressed as a number of lines as set by the > - * ExposureTime control > - * > - * \var IPAActiveState::agc.manual.gain > - * \brief Manual analogue gain as set by the AnalogueGain control > - * > - * \struct IPAActiveState::agc.automatic > - * \brief Automatic exposure time and analog gain (computed by the algorithm) > - * > - * \var IPAActiveState::agc.automatic.exposure > - * \brief Automatic exposure time expressed as a number of lines > - * > - * \var IPAActiveState::agc.automatic.gain > - * \brief Automatic analogue gain multiplier > - * > - * \var IPAActiveState::agc.autoExposureEnabled > - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control > - * > - * \var IPAActiveState::agc.autoGainEnabled > - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control > - * > - * \var IPAActiveState::agc.constraintMode > - * \brief Constraint mode as set by the AeConstraintMode control > - * > - * \var IPAActiveState::agc.exposureMode > - * \brief Exposure mode as set by the AeExposureMode control > - * > * \var IPAActiveState::agc.meteringMode > * \brief Metering mode as set by the AeMeteringMode control > - * > - * \var IPAActiveState::agc.minFrameDuration > - * \brief Minimum frame duration as set by the FrameDurationLimits control > - * > - * \var IPAActiveState::agc.maxFrameDuration > - * \brief Maximum frame duration as set by the FrameDurationLimits control > */ > > /** > @@ -314,53 +258,11 @@ namespace libcamera::ipa::rkisp1 { > * the vertical blanking period is determined to maintain a consistent frame > * rate matched to the FrameDurationLimits as set by the user. > * > - * \var IPAFrameContext::agc.exposure > - * \brief Exposure time expressed as a number of lines computed by the algorithm > - * > - * \var IPAFrameContext::agc.gain > - * \brief Analogue gain multiplier computed by the algorithm > - * > - * The gain should be adapted to the sensor specific gain code before applying. > - * > - * \var IPAFrameContext::agc.vblank > - * \brief Vertical blanking parameter computed by the algorithm > - * > - * \var IPAFrameContext::agc.autoExposureEnabled > - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control > - * > - * \var IPAFrameContext::agc.autoGainEnabled > - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control > - * > - * \var IPAFrameContext::agc.constraintMode > - * \brief Constraint mode as set by the AeConstraintMode control > - * > - * \var IPAFrameContext::agc.exposureMode > - * \brief Exposure mode as set by the AeExposureMode control > - * > * \var IPAFrameContext::agc.meteringMode > * \brief Metering mode as set by the AeMeteringMode control > * > - * \var IPAFrameContext::agc.minFrameDuration > - * \brief Minimum frame duration as set by the FrameDurationLimits control > - * > - * \var IPAFrameContext::agc.maxFrameDuration > - * \brief Maximum frame duration as set by the FrameDurationLimits control > - * > - * \var IPAFrameContext::agc.frameDuration > - * \brief The actual FrameDuration used by the algorithm for the frame > - * > * \var IPAFrameContext::agc.updateMetering > * \brief Indicate if new ISP AGC metering parameters need to be applied > - * > - * \var IPAFrameContext::agc.autoExposureModeChange > - * \brief Indicate if autoExposureEnabled has changed from true in the previous > - * frame to false in the current frame, and no manual exposure value has been > - * supplied in the current frame. > - * > - * \var IPAFrameContext::agc.autoGainModeChange > - * \brief Indicate if autoGainEnabled has changed from true in the previous > - * frame to false in the current frame, and no manual gain value has been > - * supplied in the current frame. > */ > > /** > diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h > index cd213dd991..cc07bb9462 100644 > --- a/src/ipa/rkisp1/ipa_context.h > +++ b/src/ipa/rkisp1/ipa_context.h > @@ -24,7 +24,7 @@ > #include "libcamera/internal/matrix.h" > #include "libcamera/internal/vector.h" > > -#include "libipa/agc_mean_luminance.h" > +#include "libipa/agc.h" > #include "libipa/awb.h" > #include "libipa/camera_sensor_helper.h" > #include "libipa/ccm.h" > @@ -57,7 +57,7 @@ struct RKISP1AwbSession { > }; > > struct IPASessionConfiguration { > - struct { > + struct Agc : agc::Session { > struct rkisp1_cif_isp_window measureWindow; > } agc; > > @@ -68,12 +68,6 @@ struct IPASessionConfiguration { > } compress; > > struct { > - utils::Duration minExposureTime; > - utils::Duration maxExposureTime; > - double minAnalogueGain; > - double maxAnalogueGain; > - > - utils::Duration lineDuration; > Size size; > } sensor; > > @@ -82,26 +76,8 @@ struct IPASessionConfiguration { > }; > > struct IPAActiveState { > - struct { > - struct { > - uint32_t exposure; > - double gain; > - } manual; > - struct { > - uint32_t exposure; > - double gain; > - double quantizationGain; > - double yTarget; > - } automatic; > - > - bool autoExposureEnabled; > - bool autoGainEnabled; > - double exposureValue; > - controls::AeConstraintModeEnum constraintMode; > - controls::AeExposureModeEnum exposureMode; > + struct Agc : agc::ActiveState { > controls::AeMeteringModeEnum meteringMode; > - utils::Duration minFrameDuration; > - utils::Duration maxFrameDuration; > } agc; > > ipa::awb::ActiveState awb; > @@ -145,24 +121,9 @@ struct IPAActiveState { > }; > > struct IPAFrameContext : public FrameContext { > - struct { > - uint32_t exposure; > - double gain; > - double exposureValue; > - double quantizationGain; > - uint32_t vblank; > - double yTarget; > - bool autoExposureEnabled; > - bool autoGainEnabled; > - controls::AeConstraintModeEnum constraintMode; > - controls::AeExposureModeEnum exposureMode; > + struct Agc : agc::FrameContext { > controls::AeMeteringModeEnum meteringMode; > - utils::Duration minFrameDuration; > - utils::Duration maxFrameDuration; > - utils::Duration frameDuration; > bool updateMetering; > - bool autoExposureModeChange; > - bool autoGainModeChange; > } agc; > > ipa::awb::FrameContext awb; > -- > 2.55.0 >
2026. 08. 26. 16:17 keltezéssel, Jacopo Mondi írta: > Hi Barnabás > > On Mon, Aug 24, 2026 at 11:13:40AM +0200, Barnabás Pőcze wrote: >> Add a class that implements the `Algorithm` interface using `AgcMeanLuminance` >> based on the rkisp1 `Agc` algorithm, with the following main adjustments: >> >> * the parameters for `process()` have been made optional to handle >> the cases where statistics are not available; >> * the "raw" capture check has been replaced with the "autoAllowed" >> session parameter; >> * the controls are only provided after `configure()`. >> >> Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> >> --- >> src/ipa/libipa/agc.cpp | 729 +++++++++++++++++++++++++++++- >> src/ipa/libipa/agc.h | 111 +++++ >> src/ipa/rkisp1/algorithms/agc.cpp | 461 +++---------------- >> src/ipa/rkisp1/algorithms/agc.h | 10 +- >> src/ipa/rkisp1/algorithms/lux.cpp | 2 +- >> src/ipa/rkisp1/ipa_context.cpp | 98 ---- >> src/ipa/rkisp1/ipa_context.h | 47 +- >> 7 files changed, 896 insertions(+), 562 deletions(-) >> >> diff --git a/src/ipa/libipa/agc.cpp b/src/ipa/libipa/agc.cpp >> index 864b73d5f1..e311bfd0d6 100644 >> --- a/src/ipa/libipa/agc.cpp >> +++ b/src/ipa/libipa/agc.cpp >> @@ -1,21 +1,41 @@ >> /* SPDX-License-Identifier: LGPL-2.1-or-later */ >> /* >> - * Copyright (C) 2026 Ideas On Board >> + * Copyright (C) 2021-2026 Ideas On Board >> * >> - * Auto exposure/gain algorithm for implementing the IPA-specific AGC algorithms >> + * libIPA Agc algorithm > > Introduced in the previous patch and changed here ? I have changed the previous commit. > >> */ >> >> #include "agc.h" >> >> +#include <algorithm> >> +#include <array> >> +#include <chrono> >> +#include <cmath> >> +#include <optional> >> +#include <ratio> >> + >> +#include <linux/v4l2-controls.h> >> + >> +#include <libcamera/base/log.h> >> +#include <libcamera/base/span.h> >> + >> +#include <libcamera/control_ids.h> >> +#include <libcamera/controls.h> >> + >> +#include <libcamera/ipa/core_ipa_interface.h> >> + >> /** >> * \file agc.h >> * \brief libipa AGC algorithm >> */ >> - > > Intentional ? > No. >> namespace libcamera { >> >> namespace ipa { >> >> +using namespace std::chrono_literals; >> + >> +LOG_DEFINE_CATEGORY(Agc) >> + >> namespace agc { >> >> /** >> @@ -44,8 +64,711 @@ namespace agc { >> * otherwise the gain value will be used directly. >> */ >> >> +/** >> + * \struct Session >> + * \brief Session configuration for AgcAlgorithm >> + * >> + * \var Session::minExposureTime >> + * \brief Minimum exposure time for the streaming session >> + * >> + * \var Session::maxExposureTime >> + * \brief Maximum exposure time for the streaming session >> + * >> + * \var Session::minAnalogueGain >> + * \brief Minimum analogue gain for the streaming session >> + * >> + * \var Session::maxAnalogueGain >> + * \brief Maximum analogue gain for the streaming session >> + * >> + * \var Session::minFrameDuration >> + * \brief Minimum frame duration for the streaming session >> + * >> + * \var Session::maxFrameDuration >> + * \brief Maximum frame duration for the streaming session >> + * >> + * \var Session::lineDuration >> + * \brief Line duration for the streaming session >> + * >> + * \var Session::sensor >> + * \brief Details of the sensor configuration >> + * >> + * \var Session::sensor.outputSize >> + * \brief Configured output size of the sensor >> + * >> + * \var Session::autoAllowed >> + * \copybrief AgcAlgorithm::ConfigurationParams::autoAllowed >> + * \sa AgcAlgorithm::ConfigurationParams::autoAllowed >> + */ >> + >> +/** >> + * \struct ActiveState >> + * \brief Active state for AgcAlgorithm >> + * >> + * The \a automatic variables track the latest values computed by algorithm >> + * based on the latest processed statistics. All other variables track the >> + * consolidated controls requested in queued requests. >> + * >> + * \var ActiveState::manual >> + * \brief Manual exposure time and analog gain (set through requests) >> + * >> + * \var ActiveState::manual.exposure >> + * \brief Manual exposure time expressed as a number of lines as set by the >> + * ExposureTime control >> + * >> + * \var ActiveState::manual.gain >> + * \brief Manual analogue gain as set by the AnalogueGain control >> + * >> + * \var ActiveState::automatic >> + * \brief Automatic exposure time and analog gain (computed by the algorithm) >> + * >> + * \var ActiveState::automatic.exposure >> + * \brief Automatic exposure time expressed as a number of lines >> + * >> + * \var ActiveState::automatic.gain >> + * \brief Automatic analogue gain multiplier >> + * >> + * \var ActiveState::automatic.quantizationGain >> + * \brief Automatic quantization gain multiplier >> + * >> + * \var ActiveState::automatic.yTarget >> + * \brief Automatically determined luminance target >> + * >> + * \var ActiveState::autoExposureEnabled >> + * \brief Whether automatic exposure control is enabled by the ExposureTimeMode control >> + * >> + * \var ActiveState::autoGainEnabled >> + * \brief Whether automatic gain control is enabled by the AnalogueGainMode control >> + * >> + * \var ActiveState::exposureValue >> + * \brief Exposure value as set by the ExposureValue control >> + * >> + * \var ActiveState::constraintMode >> + * \brief Constraint mode as set by the AeConstraintMode control >> + * >> + * \var ActiveState::exposureMode >> + * \brief Exposure mode as set by the AeExposureMode control >> + * >> + * \var ActiveState::minFrameDuration >> + * \brief Minimum frame duration as set by the FrameDurationLimits control >> + * >> + * \var ActiveState::maxFrameDuration >> + * \brief Maximum frame duration as set by the FrameDurationLimits control >> + */ >> + >> +/** >> + * \struct FrameContext >> + * \brief Per-frame context for AgcAlgorithm >> + * >> + * \var FrameContext::exposure >> + * \brief Exposure time expressed as a number of lines computed by the algorithm >> + * >> + * \var FrameContext::gain >> + * \brief Analogue gain multiplier computed by the algorithm >> + * >> + * The gain should be translated to the sensor specific gain code before applying. >> + * >> + * \var FrameContext::quantizationGain >> + * \brief Quantization gain multiplier computed by the algorithm >> + * >> + * \var FrameContext::exposureValue >> + * \brief Exposure value as set by the ExposureValue control >> + * >> + * \var FrameContext::yTarget >> + * \brief Luminance target computed by the algorithm >> + * >> + * \var FrameContext::vblank >> + * \brief Vertical blanking parameter computed by the algorithm >> + * >> + * \var FrameContext::autoExposureEnabled >> + * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control >> + * >> + * \var FrameContext::autoGainEnabled >> + * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control >> + * >> + * \var FrameContext::constraintMode >> + * \brief Constraint mode as set by the AeConstraintMode control >> + * >> + * \var FrameContext::exposureMode >> + * \brief Exposure mode as set by the AeExposureMode control >> + * >> + * \var FrameContext::minFrameDuration >> + * \brief Minimum frame duration as set by the FrameDurationLimits control >> + * >> + * \var FrameContext::maxFrameDuration >> + * \brief Maximum frame duration as set by the FrameDurationLimits control >> + * >> + * \var FrameContext::frameDuration >> + * \brief The actual FrameDuration used by the algorithm for the frame >> + * >> + * \var FrameContext::autoExposureModeChange >> + * \brief Indicate if autoExposureEnabled has changed from true in the previous >> + * frame to false in the current frame, and no manual exposure value has been >> + * supplied in the current frame >> + * >> + * \var FrameContext::autoGainModeChange >> + * \brief Indicate if autoGainEnabled has changed from true in the previous >> + * frame to false in the current frame, and no manual gain value has been >> + * supplied in the current frame >> + */ >> + >> } /* namespace agc */ >> >> +/** >> + * \class AgcAlgorithm >> + * \brief libIPA LSC algorithm algorithm >> + * >> + * The AgcAlgorithm class can be used to implement automatic exposure/gain >> + * control in an IPA module, conforming to the prescribed Algorithm interface. >> + * Internally AgcMeanLuminance is used, this class merely concerns itself with >> + * processing the sensor properties, establishing limits, managing controls, >> + * providing the resulting metadata, and driving the actual algorithm in >> + * process(). >> + * >> + * Users should compose the agc::Session, agc::ActiveState, and agc::FrameContext >> + * structures into their session configuration, active state, and frame contexts, >> + * respectively. Furthermore, in their implementation of the Algorithm virtual >> + * function, they should simply call the identically named member function of >> + * AgcAlgorithm. >> + * >> + * \todo DigitalGain, DigitalGainMode >> + * \todo Expand documentation >> + */ >> + >> +/** >> + * \struct AgcAlgorithm::ConfigurationParams >> + * \brief Parameters for AgcAlgorithm::configure() >> + * >> + * \var AgcAlgorithm::ConfigurationParams::sensor >> + * \brief CameraSensorHelper for the sensor >> + * >> + * \var AgcAlgorithm::ConfigurationParams::sensorInfo >> + * \brief Current configuration of the sensor >> + * >> + * \var AgcAlgorithm::ConfigurationParams::sensorControls >> + * \brief ControlInfoMap of the sensor >> + * >> + * \var AgcAlgorithm::ConfigurationParams::ctrlMap >> + * \brief ControlInfoMap::Map to update with controls >> + * >> + * \var AgcAlgorithm::ConfigurationParams::autoAllowed >> + * \brief Whether to enable auto controls >> + * >> + * If \a false, the algorithm is set up for manual exposure and gain >> + * control only, without automatic adjustments. In this mode statistics >> + * must not be provided to AgcAlgorithm::process(), and ExposureTimeMode >> + * and AnalogueGainMode will only advertise manual control. >> + */ >> + >> +/** >> + * \struct AgcAlgorithm::ProcessParams >> + * \brief Parameters for AgcAlgorithm::process() >> + * >> + * \var AgcAlgorithm::ProcessParams::traits >> + * \brief Implementation of AgcMeanLuminance::Traits >> + * >> + * \var AgcAlgorithm::ProcessParams::yHist >> + * \brief Luminance histogram of the frame >> + * >> + * \var AgcAlgorithm::ProcessParams::exposure >> + * \brief Effective exposure of the frame >> + * >> + * \var AgcAlgorithm::ProcessParams::gain >> + * \brief Effective gain of the frame >> + * >> + * \var AgcAlgorithm::ProcessParams::additionalConstraints >> + * \brief Additional AgcMeanLuminance::AgcConstraints to apply >> + * >> + * \var AgcAlgorithm::ProcessParams::lux >> + * \brief Effective lux value of the frame >> + */ >> + >> +/** >> + * \brief Load tuning data and configure >> + * \param[in] tuningData The tuning data >> + * \param[in] config The algorithm configuration >> + * >> + * This function loads the tuning data and configures the algorithm as if >> + * by a call to configure(), in order to provide the available controls > > is "as if by" intentional ? Yes. I guess my intent was that calling `configure()` is an implementation detail, but the effect is effectively the same for the user (-> the control info map is updated). > >> + * in ConfigurationParams::ctrlMap. >> + * >> + * The tuning data format is that of AgcMeanLuminance. Refer to > > is "the one" or "that" ? This is intentional as well, as far as I'm aware this is supposed to mean The tuning data format is (the tuning data format) of AgcMeanLuminance. > >> + * AgcMeanLuminance::parseTuningData() for more details. >> + * >> + * \return 0 on success, or a negative error code >> + * >> + * \sa Algorithm::init() >> + */ >> +int AgcAlgorithm::init(const ValueNode &tuningData, const ConfigurationParams &config) >> +{ >> + int ret = impl_.parseTuningData(tuningData); >> + if (ret) >> + return ret; >> + >> + /* >> + * The purpose of this `configure()` is merely to provide the >> + * available controls in `config.ctrlMap`. >> + * >> + * \todo Remove it once IPA modules have been changed to >> + * configure the algorithms during initialization. >> + */ >> + >> + agc::Session dummySession; >> + agc::ActiveState dummyState; >> + > > This was different in the previous version, right ? Ahh, right, I probably forgot to mention it. But I came to the conclusion that passing the from the ipa algorithm is a bit useless because they will be zeroed during configuration, so dummy ones could be used, removing 2 arguments. > >> + return configure(dummySession, dummyState, config); >> +} >> + >> +/** >> + * \brief Initialize the session configuration and active state >> + * \param[in] session The agc session configuration >> + * \param[in] state The agc active state >> + * \param[in] config The algorithm configuration >> + * >> + * This function initializes \a session and \a state based on the tuning >> + * data loaded by init() and the configuration in \a config. >> + * >> + * It also updates ConfigurationParams::ctrlMap with the limits of the various >> + * available agc-related controls. Users are expected to propagate these controls >> + * to the camera. >> + * >> + * \return 0 on success, or a negative error code >> + * >> + * \sa Algorithm::configure() >> + */ >> +int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, >> + const ConfigurationParams &config) >> +{ >> + session = {}; >> + session.autoAllowed = config.autoAllowed; >> + session.lineDuration = >> + config.sensorInfo.minLineLength * 1.0s / config.sensorInfo.pixelRate; >> + session.sensor.outputSize = config.sensorInfo.outputSize; >> + >> + const double lineDurationUs = session.lineDuration.get<std::micro>(); >> + >> + /* >> + * Compute exposure time limits from the V4L2_CID_EXPOSURE control >> + * limits and the line duration. >> + */ >> + >> + const ControlInfo &v4l2Exposure = config.sensorControls.find(V4L2_CID_EXPOSURE)->second; >> + int32_t minExposure = v4l2Exposure.min().get<int32_t>(); >> + int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); >> + int32_t defExposure = v4l2Exposure.def().get<int32_t>(); >> + >> + /* Compute the analogue gain limits. */ >> + const ControlInfo &v4l2Gain = config.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; >> + float minGain = config.sensor->gain(v4l2Gain.min().get<int32_t>()); >> + float maxGain = config.sensor->gain(v4l2Gain.max().get<int32_t>()); >> + float defGain = config.sensor->gain(v4l2Gain.def().get<int32_t>()); >> + >> + LOG(Agc, Debug) >> + << "Exposure: [" << minExposure << ", " << maxExposure >> + << "], gain: [" << minGain << ", " << maxGain << "]"; >> + >> + /* >> + * Compute the frame duration limits. >> + * >> + * The frame length is computed assuming a fixed line length combined >> + * with the vertical frame sizes. >> + */ >> + const ControlInfo &v4l2HBlank = config.sensorControls.find(V4L2_CID_HBLANK)->second; >> + uint32_t hblank = v4l2HBlank.def().get<int32_t>(); >> + uint32_t lineLength = config.sensorInfo.outputSize.width + hblank; >> + >> + const ControlInfo &v4l2VBlank = config.sensorControls.find(V4L2_CID_VBLANK)->second; >> + std::array<uint32_t, 3> frameHeights{ >> + v4l2VBlank.min().get<int32_t>() + config.sensorInfo.outputSize.height, >> + v4l2VBlank.max().get<int32_t>() + config.sensorInfo.outputSize.height, >> + v4l2VBlank.def().get<int32_t>() + config.sensorInfo.outputSize.height, >> + }; >> + >> + std::array<int64_t, 3> frameDurations; >> + for (unsigned int i = 0; i < frameHeights.size(); ++i) { >> + uint64_t frameSize = lineLength * frameHeights[i]; >> + frameDurations[i] = frameSize / (config.sensorInfo.pixelRate / 1000000U); >> + } >> + >> + /* >> + * When the AGC computes the new exposure values for a frame, it needs >> + * to know the limits for exposure time and analogue gain. As it depends >> + * on the sensor, update it with the controls. >> + * >> + * \todo take VBLANK into account for maximum exposure time >> + */ >> + session.minExposureTime = minExposure * session.lineDuration; >> + session.maxExposureTime = maxExposure * session.lineDuration; >> + session.minAnalogueGain = minGain; >> + session.maxAnalogueGain = maxGain; >> + session.minFrameDuration = std::chrono::microseconds(frameDurations[0]); >> + session.maxFrameDuration = std::chrono::microseconds(frameDurations[1]); >> + >> + impl_.configure(session.lineDuration, config.sensor); >> + impl_.setLimits(session.minExposureTime, session.maxExposureTime, >> + session.minAnalogueGain, session.maxAnalogueGain, >> + {}); >> + impl_.resetFrameCount(); >> + >> + /* Configure the default exposure and gain. */ >> + state = {}; >> + state.automatic.gain = session.minAnalogueGain; >> + state.automatic.exposure = 10ms / session.lineDuration; >> + state.automatic.quantizationGain = 1; >> + state.automatic.yTarget = impl_.effectiveYTarget(0, 1); >> + state.manual.gain = state.automatic.gain; >> + state.manual.exposure = state.automatic.exposure; >> + state.autoExposureEnabled = session.autoAllowed; >> + state.autoGainEnabled = session.autoAllowed; >> + state.exposureValue = 0; >> + state.constraintMode = >> + static_cast<controls::AeConstraintModeEnum>(impl_.constraintModes().begin()->first); >> + state.exposureMode = >> + static_cast<controls::AeExposureModeEnum>(impl_.exposureModeHelpers().begin()->first); >> + state.minFrameDuration = session.minFrameDuration; >> + state.maxFrameDuration = session.maxFrameDuration; >> + >> + /* \todo Move this to the `Camera` class. */ >> + config.ctrlMap[&controls::AeEnable] = ControlInfo{ >> + false, session.autoAllowed, session.autoAllowed >> + }; >> + config.ctrlMap[&controls::AnalogueGain] = ControlInfo{ >> + minGain, maxGain, defGain >> + }; >> + config.ctrlMap[&controls::ExposureTime] = ControlInfo{ >> + static_cast<int32_t>(minExposure * lineDurationUs), >> + static_cast<int32_t>(maxExposure * lineDurationUs), >> + static_cast<int32_t>(defExposure * lineDurationUs), >> + }; >> + config.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ >> + frameDurations[0], frameDurations[1], >> + Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, >> + }; >> + config.ctrlMap[&controls::ExposureTimeMode] = ControlInfo{ >> + {{ controls::ExposureTimeModeAuto, controls::ExposureTimeModeManual }}, >> + controls::ExposureTimeModeAuto, >> + }; >> + config.ctrlMap[&controls::AnalogueGainMode] = ControlInfo{ >> + {{ controls::AnalogueGainModeAuto, controls::AnalogueGainModeManual }}, >> + controls::AnalogueGainModeAuto, >> + }; >> + config.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); >> + config.ctrlMap.merge(impl_.controls()); >> + >> + return 0; >> +} >> + >> +/** >> + * \brief Queue a request >> + * \param[in] session The agc session configuration >> + * \param[in] state The agc active state >> + * \param[in] frameContext The agc frame context >> + * \param[in] controls The list of controls associated with a Request >> + * >> + * This functions processes the agc-related controls in \a controls for the frame >> + * denoted by \a frameContext, and updates \a state and \a frameContext accordingly. >> + * > > Reflow to 80 cols Done. > >> + * \sa Algorithm::queueRequest() >> + */ >> +void AgcAlgorithm::queueRequest(const agc::Session &session, agc::ActiveState &state, >> + agc::FrameContext &frameContext, const ControlList &controls) >> +{ >> + if (session.autoAllowed) { >> + const auto &aeEnable = controls.get(controls::ExposureTimeMode); >> + if (aeEnable && >> + (*aeEnable == controls::ExposureTimeModeAuto) != state.autoExposureEnabled) { >> + state.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); >> + >> + LOG(Agc, Debug) >> + << (state.autoExposureEnabled ? "Enabling" : "Disabling") >> + << " AGC (exposure)"; >> + >> + /* >> + * If we go from auto -> manual with no manual control >> + * set, use the last computed value, which we don't >> + * know until prepare() so save this information. >> + * >> + * \todo Check the previous frame at prepare() time >> + * instead of saving a flag here >> + */ >> + if (!state.autoExposureEnabled && !controls.get(controls::ExposureTime)) >> + frameContext.autoExposureModeChange = true; >> + } >> + >> + const auto &agEnable = controls.get(controls::AnalogueGainMode); >> + if (agEnable && >> + (*agEnable == controls::AnalogueGainModeAuto) != state.autoGainEnabled) { >> + state.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); >> + >> + LOG(Agc, Debug) >> + << (state.autoGainEnabled ? "Enabling" : "Disabling") >> + << " AGC (gain)"; >> + >> + /* >> + * If we go from auto -> manual with no manual control >> + * set, use the last computed value, which we don't >> + * know until prepare() so save this information. >> + */ >> + if (!state.autoGainEnabled && !controls.get(controls::AnalogueGain)) >> + frameContext.autoGainModeChange = true; >> + } >> + } >> + >> + const auto &exposure = controls.get(controls::ExposureTime); >> + if (exposure && !state.autoExposureEnabled) { >> + state.manual.exposure = *exposure * 1.0us / session.lineDuration; >> + >> + LOG(Agc, Debug) << "Set exposure to " << state.manual.exposure; >> + } >> + >> + const auto &gain = controls.get(controls::AnalogueGain); >> + if (gain && !state.autoGainEnabled) { >> + state.manual.gain = *gain; >> + >> + LOG(Agc, Debug) << "Set gain to " << state.manual.gain; >> + } >> + >> + frameContext.autoExposureEnabled = state.autoExposureEnabled; >> + frameContext.autoGainEnabled = state.autoGainEnabled; >> + >> + if (!frameContext.autoExposureEnabled) >> + frameContext.exposure = state.manual.exposure; >> + if (!frameContext.autoGainEnabled) >> + frameContext.gain = state.manual.gain; >> + >> + if (!frameContext.autoExposureEnabled && !frameContext.autoGainEnabled) >> + frameContext.quantizationGain = 1.0; >> + >> + const auto &exposureMode = controls.get(controls::AeExposureMode); >> + if (exposureMode) >> + state.exposureMode = >> + static_cast<controls::AeExposureModeEnum>(*exposureMode); >> + frameContext.exposureMode = state.exposureMode; >> + >> + const auto &constraintMode = controls.get(controls::AeConstraintMode); >> + if (constraintMode) >> + state.constraintMode = >> + static_cast<controls::AeConstraintModeEnum>(*constraintMode); >> + frameContext.constraintMode = state.constraintMode; >> + >> + const auto &exposureValue = controls.get(controls::ExposureValue); >> + if (exposureValue) >> + state.exposureValue = *exposureValue; >> + frameContext.exposureValue = state.exposureValue; >> + >> + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); >> + if (frameDurationLimits) { >> + /* Limit the control value to the limits in ControlInfo */ >> + state.minFrameDuration = std::clamp<utils::Duration>( >> + std::chrono::microseconds((*frameDurationLimits).front()), >> + session.minFrameDuration, session.maxFrameDuration); >> + >> + state.maxFrameDuration = std::clamp<utils::Duration>( >> + std::chrono::microseconds((*frameDurationLimits).back()), >> + session.minFrameDuration, session.maxFrameDuration); >> + } >> + frameContext.minFrameDuration = state.minFrameDuration; >> + frameContext.maxFrameDuration = state.maxFrameDuration; >> +} >> + >> +/** >> + * \brief Prepare a frame >> + * \param[in] state The agc active state >> + * \param[in] frameContext The agc frame context >> + * >> + * This function prepares the parameters for the frame denoted by \a frameContext. >> + * After a call to this function, the values of \ref agc::FrameContext::exposure >> + * "frameContext.exposure" and \ref agc::FrameContext::gain "frameContext.gain" >> + * will be finalized and may be used by the caller (see agc::prepareControls()). >> + * >> + * \todo Finalize \ref agc::FrameContext::vblank "frameContext.vblank" as well >> + * >> + * \sa Algorithm::prepare() > > Here and in the below function, reflow to 80 cols. Done. > > With these minors fixed > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com> > > Thanks > j > >> + */ >> +void AgcAlgorithm::prepare(agc::ActiveState &state, agc::FrameContext &frameContext) >> +{ >> + uint32_t activeAutoExposure = state.automatic.exposure; >> + double activeAutoGain = state.automatic.gain; >> + double activeAutoQGain = state.automatic.quantizationGain; >> + >> + /* Populate exposure and gain in auto mode */ >> + if (frameContext.autoExposureEnabled) { >> + frameContext.exposure = activeAutoExposure; >> + frameContext.quantizationGain = activeAutoQGain; >> + } >> + if (frameContext.autoGainEnabled) { >> + frameContext.gain = activeAutoGain; >> + frameContext.quantizationGain = activeAutoQGain; >> + } >> + >> + /* >> + * Populate manual exposure and gain from the active auto values when >> + * transitioning from auto to manual >> + */ >> + if (!frameContext.autoExposureEnabled && frameContext.autoExposureModeChange) { >> + state.manual.exposure = activeAutoExposure; >> + frameContext.exposure = activeAutoExposure; >> + } >> + if (!frameContext.autoGainEnabled && frameContext.autoGainModeChange) { >> + state.manual.gain = activeAutoGain; >> + frameContext.gain = activeAutoGain; >> + frameContext.quantizationGain = activeAutoQGain; >> + } >> + >> + frameContext.yTarget = state.automatic.yTarget; >> +} >> + >> +/** >> + * \brief Process frame statistics >> + * \param[in] session The agc session configuration >> + * \param[in] state The agc active state >> + * \param[in] frameContext The agc frame context >> + * \param[in] params The algorithm parameters >> + * \param[in] metadata The list of metadata >> + * >> + * This function processes the statistics for the completed frame denoted by >> + * \a frameContext, runs the AGC implementation, and updates \a state appropriately. >> + * This function also populates \a metadata for the completed frame. >> + * >> + * \a params must be omitted if the session was configured without "autoAllowed", >> + * and it may be omitted even if auto control is enabled, for example, if the >> + * statistics could not be delivered due to some ephemeral error. This ensures >> + * that the algorithm state will not go out of sync, and that metadata is produced >> + * as expected. >> + * >> + * Care must be taken to convert the platform specific statistics to the format >> + * expected in \a params. See ProcessParams for the details. >> + * >> + * \sa Algorithm::process() >> + */ >> +void AgcAlgorithm::process(const agc::Session &session, agc::ActiveState &state, >> + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, >> + ControlList &metadata) >> +{ >> + if (!params) { >> + processFrameDuration(session, frameContext, frameContext.minFrameDuration); >> + fillMetadata(session, frameContext, metadata); >> + return; >> + } >> + >> + ASSERT(session.autoAllowed); >> + >> + const utils::Duration &lineDuration = session.lineDuration; >> + >> + /* >> + * Set the AGC limits using the fixed exposure time and/or gain in >> + * manual mode, or the sensor limits in auto mode. >> + */ >> + utils::Duration minExposureTime; >> + utils::Duration maxExposureTime; >> + double minAnalogueGain; >> + double maxAnalogueGain; >> + >> + /* \todo This uses the configuration from an already completed frame. */ >> + >> + if (frameContext.autoExposureEnabled) { >> + minExposureTime = session.minExposureTime; >> + maxExposureTime = std::clamp(frameContext.maxFrameDuration, >> + session.minExposureTime, >> + session.maxExposureTime); >> + } else { >> + minExposureTime = lineDuration * frameContext.exposure; >> + maxExposureTime = minExposureTime; >> + } >> + >> + if (frameContext.autoGainEnabled) { >> + minAnalogueGain = session.minAnalogueGain; >> + maxAnalogueGain = session.maxAnalogueGain; >> + } else { >> + minAnalogueGain = frameContext.gain; >> + maxAnalogueGain = frameContext.gain; >> + } >> + >> + /* >> + * The Agc algorithm needs to know the effective exposure value that was >> + * applied to the sensor when the statistics were collected. >> + */ >> + utils::Duration effectiveExposureValue = >> + lineDuration * params->exposure * params->gain; >> + >> + impl_.setLimits(minExposureTime, maxExposureTime, >> + minAnalogueGain, maxAnalogueGain, >> + std::move(params->additionalConstraints)); >> + >> + const auto &newEv = impl_.calculateNewEv({ >> + .traits = params->traits, >> + .yHist = params->yHist, >> + .effectiveExposureValue = effectiveExposureValue, >> + .constraintModeIndex = frameContext.constraintMode, >> + .exposureModeIndex = frameContext.exposureMode, >> + .lux = params->lux, >> + .exposureCompensation = std::pow(2.0, frameContext.exposureValue), >> + }); >> + >> + /* Update the estimated exposure and gain. */ >> + state.automatic.exposure = newEv.exposureTime / lineDuration; >> + state.automatic.gain = newEv.analogueGain; >> + state.automatic.quantizationGain = newEv.quantizationGain; >> + state.automatic.yTarget = newEv.yTarget; >> + >> + LOG(Agc, Debug) >> + << "Divided up exposure time, analogue gain, quantization gain" >> + << " and digital gain are " << newEv.exposureTime >> + << ", " << state.automatic.gain << ", " << state.automatic.quantizationGain >> + << " and " << newEv.digitalGain; >> + >> + /* >> + * Expand the target frame duration so that we do not run faster than >> + * the minimum frame duration when we have short exposures. >> + */ >> + processFrameDuration(session, frameContext, >> + std::max(frameContext.minFrameDuration, newEv.exposureTime)); >> + >> + fillMetadata(session, frameContext, metadata); >> +} >> + >> +/** >> + * \brief Process frame duration and compute vblank >> + * \param[in] session The session parameters >> + * \param[in] frameContext The current frame context >> + * \param[in] frameDuration The target frame duration >> + * >> + * Compute and populate vblank from the target frame duration. >> + */ >> +void AgcAlgorithm::processFrameDuration(const agc::Session &session, >> + agc::FrameContext &frameContext, >> + utils::Duration frameDuration) >> +{ >> + const utils::Duration &lineDuration = session.lineDuration; >> + >> + frameContext.vblank = >> + (frameDuration / lineDuration) - session.sensor.outputSize.height; >> + >> + /* Update frame duration accounting for line length quantization. */ >> + frameContext.frameDuration = >> + (session.sensor.outputSize.height + frameContext.vblank) * lineDuration; >> +} >> + >> +void AgcAlgorithm::fillMetadata(const agc::Session &session, >> + const agc::FrameContext &frameContext, >> + ControlList &metadata) >> +{ >> + >> + metadata.set(controls::AnalogueGain, frameContext.gain); >> + metadata.set(controls::ExposureTime, >> + utils::Duration(session.lineDuration * frameContext.exposure).get<std::micro>()); >> + metadata.set(controls::FrameDuration, frameContext.frameDuration.get<std::micro>()); >> + metadata.set(controls::ExposureTimeMode, frameContext.autoExposureEnabled >> + ? controls::ExposureTimeModeAuto >> + : controls::ExposureTimeModeManual); >> + metadata.set(controls::AnalogueGainMode, frameContext.autoGainEnabled >> + ? controls::AnalogueGainModeAuto >> + : controls::AnalogueGainModeManual); >> + >> + metadata.set(controls::AeExposureMode, frameContext.exposureMode); >> + metadata.set(controls::AeConstraintMode, frameContext.constraintMode); >> + metadata.set(controls::ExposureValue, frameContext.exposureValue); >> +} >> + >> } /* namespace ipa */ >> >> } /* namespace libcamera */ >> diff --git a/src/ipa/libipa/agc.h b/src/ipa/libipa/agc.h >> index 5a67464086..06600f0208 100644 >> --- a/src/ipa/libipa/agc.h >> +++ b/src/ipa/libipa/agc.h >> @@ -7,21 +7,87 @@ >> >> #pragma once >> >> +#include <optional> >> #include <stdint.h> >> #include <utility> >> +#include <vector> >> >> #include <linux/v4l2-controls.h> >> >> +#include <libcamera/base/utils.h> >> + >> +#include <libcamera/control_ids.h> >> #include <libcamera/controls.h> >> +#include <libcamera/geometry.h> >> >> +#include "agc_mean_luminance.h" >> #include "camera_sensor_helper.h" >> >> namespace libcamera { >> >> +struct IPACameraSensorInfo; >> + >> namespace ipa { >> >> +class Histogram; >> + >> namespace agc { >> >> +struct Session { >> + utils::Duration minExposureTime; >> + utils::Duration maxExposureTime; >> + double minAnalogueGain; >> + double maxAnalogueGain; >> + utils::Duration minFrameDuration; >> + utils::Duration maxFrameDuration; >> + utils::Duration lineDuration; >> + >> + struct { >> + Size outputSize; >> + } sensor; >> + >> + bool autoAllowed; >> +}; >> + >> +struct ActiveState { >> + struct { >> + uint32_t exposure; >> + double gain; >> + } manual; >> + struct { >> + uint32_t exposure; >> + double gain; >> + double quantizationGain; >> + double yTarget; >> + } automatic; >> + >> + bool autoExposureEnabled; >> + bool autoGainEnabled; >> + double exposureValue; >> + controls::AeConstraintModeEnum constraintMode; >> + controls::AeExposureModeEnum exposureMode; >> + utils::Duration minFrameDuration; >> + utils::Duration maxFrameDuration; >> +}; >> + >> +struct FrameContext { >> + uint32_t exposure; >> + double gain; >> + double quantizationGain; >> + double exposureValue; >> + double yTarget; >> + uint32_t vblank; >> + bool autoExposureEnabled; >> + bool autoGainEnabled; >> + controls::AeConstraintModeEnum constraintMode; >> + controls::AeExposureModeEnum exposureMode; >> + utils::Duration minFrameDuration; >> + utils::Duration maxFrameDuration; >> + utils::Duration frameDuration; >> + bool autoExposureModeChange; >> + bool autoGainModeChange; >> +}; >> + >> [[nodiscard]] >> inline std::pair<uint32_t, double> >> extractControls(const ControlList &controls, const CameraSensorHelper *sensor) >> @@ -48,6 +114,51 @@ prepareControls(ControlList &controls, const CameraSensorHelper *sensor, >> >> } /* namespace agc */ >> >> +class AgcAlgorithm >> +{ >> +public: >> + struct ConfigurationParams { >> + const CameraSensorHelper *sensor; >> + const IPACameraSensorInfo &sensorInfo; >> + const ControlInfoMap &sensorControls; >> + ControlInfoMap::Map &ctrlMap; >> + bool autoAllowed = true; >> + }; >> + >> + struct ProcessParams { >> + const AgcMeanLuminance::Traits &traits; >> + const Histogram &yHist; >> + uint32_t exposure; >> + double gain; >> + std::vector<AgcMeanLuminance::AgcConstraint> &&additionalConstraints = {}; >> + double lux = 0; >> + }; >> + >> + int init(const ValueNode &tuningData, const ConfigurationParams &config); >> + >> + int configure(agc::Session &session, agc::ActiveState &state, >> + const ConfigurationParams &config); >> + >> + void queueRequest(const agc::Session &session, agc::ActiveState &state, >> + agc::FrameContext &frameContext, const ControlList &controls); >> + >> + void prepare(agc::ActiveState &state, agc::FrameContext &frameContext); >> + >> + void process(const agc::Session &session, agc::ActiveState &state, >> + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, >> + ControlList &metadata); >> + >> +private: >> + void processFrameDuration(const agc::Session &session, >> + agc::FrameContext &frameContext, >> + utils::Duration frameDuration); >> + void fillMetadata(const agc::Session &session, >> + const agc::FrameContext &frameContext, >> + ControlList &metadata); >> + >> + AgcMeanLuminance impl_; >> +}; >> + >> } /* namespace ipa */ >> >> } /* namespace libcamera */ >> diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp >> index 7230232d24..bb0345ca87 100644 >> --- a/src/ipa/rkisp1/algorithms/agc.cpp >> +++ b/src/ipa/rkisp1/algorithms/agc.cpp >> @@ -8,10 +8,7 @@ >> #include "agc.h" >> >> #include <algorithm> >> -#include <array> >> -#include <chrono> >> #include <cmath> >> -#include <tuple> >> #include <vector> >> >> #include <libcamera/base/log.h> >> @@ -36,89 +33,6 @@ namespace ipa::rkisp1::algorithms { >> >> LOG_DEFINE_CATEGORY(RkISP1Agc) >> >> -namespace { >> - >> -void reconfigure(IPAContext &context) >> -{ >> - context.configuration.sensor.lineDuration = >> - context.sensorInfo.minLineLength * 1.0s / context.sensorInfo.pixelRate; >> - >> - double lineDurationUs = context.configuration.sensor.lineDuration.get<std::micro>(); >> - >> - /* >> - * Compute exposure time limits from the V4L2_CID_EXPOSURE control >> - * limits and the line duration. >> - */ >> - >> - const ControlInfo &v4l2Exposure = context.sensorControls.find(V4L2_CID_EXPOSURE)->second; >> - int32_t minExposure = v4l2Exposure.min().get<int32_t>(); >> - int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); >> - int32_t defExposure = v4l2Exposure.def().get<int32_t>(); >> - context.ctrlMap[&controls::ExposureTime] = ControlInfo{ >> - static_cast<int32_t>(minExposure * lineDurationUs), >> - static_cast<int32_t>(maxExposure * lineDurationUs), >> - static_cast<int32_t>(defExposure * lineDurationUs), >> - }; >> - >> - /* Compute the analogue gain limits. */ >> - const ControlInfo &v4l2Gain = context.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; >> - float minGain = context.camHelper->gain(v4l2Gain.min().get<int32_t>()); >> - float maxGain = context.camHelper->gain(v4l2Gain.max().get<int32_t>()); >> - float defGain = context.camHelper->gain(v4l2Gain.def().get<int32_t>()); >> - context.ctrlMap[&controls::AnalogueGain] = ControlInfo{ >> - minGain, >> - maxGain, >> - defGain, >> - }; >> - >> - LOG(RkISP1Agc, Debug) >> - << "Exposure: [" << minExposure << ", " << maxExposure >> - << "], gain: [" << minGain << ", " << maxGain << "]"; >> - >> - /* >> - * Compute the frame duration limits. >> - * >> - * The frame length is computed assuming a fixed line length combined >> - * with the vertical frame sizes. >> - */ >> - const ControlInfo &v4l2HBlank = context.sensorControls.find(V4L2_CID_HBLANK)->second; >> - uint32_t hblank = v4l2HBlank.def().get<int32_t>(); >> - uint32_t lineLength = context.sensorInfo.outputSize.width + hblank; >> - >> - const ControlInfo &v4l2VBlank = context.sensorControls.find(V4L2_CID_VBLANK)->second; >> - std::array<uint32_t, 3> frameHeights{ >> - v4l2VBlank.min().get<int32_t>() + context.sensorInfo.outputSize.height, >> - v4l2VBlank.max().get<int32_t>() + context.sensorInfo.outputSize.height, >> - v4l2VBlank.def().get<int32_t>() + context.sensorInfo.outputSize.height, >> - }; >> - >> - std::array<int64_t, 3> frameDurations; >> - for (unsigned int i = 0; i < frameHeights.size(); ++i) { >> - uint64_t frameSize = lineLength * frameHeights[i]; >> - frameDurations[i] = frameSize / (context.sensorInfo.pixelRate / 1000000U); >> - } >> - >> - context.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ >> - frameDurations[0], >> - frameDurations[1], >> - Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, >> - }; >> - >> - /* >> - * When the AGC computes the new exposure values for a frame, it needs >> - * to know the limits for exposure time and analogue gain. As it depends >> - * on the sensor, update it with the controls. >> - * >> - * \todo take VBLANK into account for maximum exposure time >> - */ >> - context.configuration.sensor.minExposureTime = minExposure * context.configuration.sensor.lineDuration; >> - context.configuration.sensor.maxExposureTime = maxExposure * context.configuration.sensor.lineDuration; >> - context.configuration.sensor.minAnalogueGain = minGain; >> - context.configuration.sensor.maxAnalogueGain = maxGain; >> -} >> - >> -} /* namespace */ >> - >> /** >> * \class Agc >> * \brief A mean-based auto-exposure algorithm >> @@ -222,7 +136,12 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) >> { >> int ret; >> >> - ret = agc_.parseTuningData(tuningData); >> + ret = agc_.init(tuningData, { >> + .sensor = context.camHelper.get(), >> + .sensorInfo = context.sensorInfo, >> + .sensorControls = context.sensorControls, >> + .ctrlMap = context.ctrlMap, >> + }); >> if (ret) >> return ret; >> >> @@ -231,21 +150,6 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) >> if (ret) >> return ret; >> >> - context.ctrlMap[&controls::ExposureTimeMode] = >> - ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), >> - ControlValue(controls::ExposureTimeModeManual) } }, >> - ControlValue(controls::ExposureTimeModeAuto)); >> - context.ctrlMap[&controls::AnalogueGainMode] = >> - ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), >> - ControlValue(controls::AnalogueGainModeManual) } }, >> - ControlValue(controls::AnalogueGainModeAuto)); >> - /* \todo Move this to the Camera class */ >> - context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true, true); >> - context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); >> - context.ctrlMap.merge(agc_.controls()); >> - >> - reconfigure(context); >> - >> return 0; >> } >> >> @@ -258,47 +162,24 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) >> */ >> int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) >> { >> - reconfigure(context); >> - >> - /* Configure the default exposure and gain. */ >> - context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; >> - context.activeState.agc.automatic.exposure = >> - 10ms / context.configuration.sensor.lineDuration; >> - context.activeState.agc.automatic.quantizationGain = 1.0; >> - context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; >> - context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; >> - context.activeState.agc.autoExposureEnabled = !context.configuration.raw; >> - context.activeState.agc.autoGainEnabled = !context.configuration.raw; >> - context.activeState.agc.exposureValue = 0.0; >> - >> - context.activeState.agc.constraintMode = >> - static_cast<controls::AeConstraintModeEnum>(agc_.constraintModes().begin()->first); >> - context.activeState.agc.exposureMode = >> - static_cast<controls::AeExposureModeEnum>(agc_.exposureModeHelpers().begin()->first); >> + int ret = agc_.configure(context.configuration.agc, context.activeState.agc, { >> + .sensor = context.camHelper.get(), >> + .sensorInfo = context.sensorInfo, >> + .sensorControls = context.sensorControls, >> + .ctrlMap = context.ctrlMap, >> + .autoAllowed = !context.configuration.raw, >> + }); >> + if (ret) >> + return ret; >> + >> context.activeState.agc.meteringMode = >> static_cast<controls::AeMeteringModeEnum>(meteringModes_.begin()->first); >> >> - /* Limit the frame duration to match current initialisation */ >> - ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; >> - context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get<int64_t>()); >> - context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get<int64_t>()); >> - >> context.configuration.agc.measureWindow.h_offs = 0; >> context.configuration.agc.measureWindow.v_offs = 0; >> context.configuration.agc.measureWindow.h_size = configInfo.outputSize.width; >> context.configuration.agc.measureWindow.v_size = configInfo.outputSize.height; >> >> - agc_.configure(context.configuration.sensor.lineDuration, context.camHelper.get()); >> - >> - agc_.setLimits(context.configuration.sensor.minExposureTime, >> - context.configuration.sensor.maxExposureTime, >> - context.configuration.sensor.minAnalogueGain, >> - context.configuration.sensor.maxAnalogueGain, {}); >> - >> - context.activeState.agc.automatic.yTarget = agc_.effectiveYTarget(0, 1); >> - >> - agc_.resetFrameCount(); >> - >> return 0; >> } >> >> @@ -312,73 +193,7 @@ void Agc::queueRequest(IPAContext &context, >> { >> auto &agc = context.activeState.agc; >> >> - if (!context.configuration.raw) { >> - const auto &aeEnable = controls.get(controls::ExposureTimeMode); >> - if (aeEnable && >> - (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { >> - agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); >> - >> - LOG(RkISP1Agc, Debug) >> - << (agc.autoExposureEnabled ? "Enabling" : "Disabling") >> - << " AGC (exposure)"; >> - >> - /* >> - * If we go from auto -> manual with no manual control >> - * set, use the last computed value, which we don't >> - * know until prepare() so save this information. >> - * >> - * \todo Check the previous frame at prepare() time >> - * instead of saving a flag here >> - */ >> - if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) >> - frameContext.agc.autoExposureModeChange = true; >> - } >> - >> - const auto &agEnable = controls.get(controls::AnalogueGainMode); >> - if (agEnable && >> - (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { >> - agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); >> - >> - LOG(RkISP1Agc, Debug) >> - << (agc.autoGainEnabled ? "Enabling" : "Disabling") >> - << " AGC (gain)"; >> - /* >> - * If we go from auto -> manual with no manual control >> - * set, use the last computed value, which we don't >> - * know until prepare() so save this information. >> - */ >> - if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) >> - frameContext.agc.autoGainModeChange = true; >> - } >> - } >> - >> - const auto &exposure = controls.get(controls::ExposureTime); >> - if (exposure && !agc.autoExposureEnabled) { >> - agc.manual.exposure = *exposure * 1.0us >> - / context.configuration.sensor.lineDuration; >> - >> - LOG(RkISP1Agc, Debug) >> - << "Set exposure to " << agc.manual.exposure; >> - } >> - >> - const auto &gain = controls.get(controls::AnalogueGain); >> - if (gain && !agc.autoGainEnabled) { >> - agc.manual.gain = *gain; >> - >> - LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; >> - } >> - >> - frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; >> - frameContext.agc.autoGainEnabled = agc.autoGainEnabled; >> - >> - if (!frameContext.agc.autoExposureEnabled) >> - frameContext.agc.exposure = agc.manual.exposure; >> - if (!frameContext.agc.autoGainEnabled) >> - frameContext.agc.gain = agc.manual.gain; >> - >> - if (!frameContext.agc.autoExposureEnabled && >> - !frameContext.agc.autoGainEnabled) >> - frameContext.agc.quantizationGain = 1.0; >> + agc_.queueRequest(context.configuration.agc, agc, frameContext.agc, controls); >> >> const auto &meteringMode = controls.get(controls::AeMeteringMode); >> if (meteringMode) { >> @@ -387,42 +202,6 @@ void Agc::queueRequest(IPAContext &context, >> static_cast<controls::AeMeteringModeEnum>(*meteringMode); >> } >> frameContext.agc.meteringMode = agc.meteringMode; >> - >> - const auto &exposureMode = controls.get(controls::AeExposureMode); >> - if (exposureMode) >> - agc.exposureMode = >> - static_cast<controls::AeExposureModeEnum>(*exposureMode); >> - frameContext.agc.exposureMode = agc.exposureMode; >> - >> - const auto &constraintMode = controls.get(controls::AeConstraintMode); >> - if (constraintMode) >> - agc.constraintMode = >> - static_cast<controls::AeConstraintModeEnum>(*constraintMode); >> - frameContext.agc.constraintMode = agc.constraintMode; >> - >> - const auto &exposureValue = controls.get(controls::ExposureValue); >> - if (exposureValue) >> - agc.exposureValue = *exposureValue; >> - frameContext.agc.exposureValue = agc.exposureValue; >> - >> - const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); >> - if (frameDurationLimits) { >> - /* Limit the control value to the limits in ControlInfo */ >> - ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; >> - int64_t minFrameDuration = >> - std::clamp((*frameDurationLimits).front(), >> - limits.min().get<int64_t>(), >> - limits.max().get<int64_t>()); >> - int64_t maxFrameDuration = >> - std::clamp((*frameDurationLimits).back(), >> - limits.min().get<int64_t>(), >> - limits.max().get<int64_t>()); >> - >> - agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); >> - agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); >> - } >> - frameContext.agc.minFrameDuration = agc.minFrameDuration; >> - frameContext.agc.maxFrameDuration = agc.maxFrameDuration; >> } >> >> /** >> @@ -431,41 +210,13 @@ void Agc::queueRequest(IPAContext &context, >> void Agc::prepare(IPAContext &context, const uint32_t frame, >> IPAFrameContext &frameContext, RkISP1Params *params) >> { >> - uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; >> - double activeAutoGain = context.activeState.agc.automatic.gain; >> - double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; >> - >> - /* Populate exposure and gain in auto mode */ >> - if (frameContext.agc.autoExposureEnabled) { >> - frameContext.agc.exposure = activeAutoExposure; >> - frameContext.agc.quantizationGain = activeAutoQGain; >> - } >> - if (frameContext.agc.autoGainEnabled) { >> - frameContext.agc.gain = activeAutoGain; >> - frameContext.agc.quantizationGain = activeAutoQGain; >> - } >> - >> - /* >> - * Populate manual exposure and gain from the active auto values when >> - * transitioning from auto to manual >> - */ >> - if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { >> - context.activeState.agc.manual.exposure = activeAutoExposure; >> - frameContext.agc.exposure = activeAutoExposure; >> - } >> - if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { >> - context.activeState.agc.manual.gain = activeAutoGain; >> - frameContext.agc.gain = activeAutoGain; >> - frameContext.agc.quantizationGain = activeAutoQGain; >> - } >> + agc_.prepare(context.activeState.agc, frameContext.agc); >> >> if (context.configuration.compress.supported) { >> frameContext.compress.enable = true; >> frameContext.compress.gain = frameContext.agc.quantizationGain; >> } >> >> - frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; >> - >> if (frame > 0 && !frameContext.agc.updateMetering) >> return; >> >> @@ -521,50 +272,6 @@ void Agc::prepare(IPAContext &context, const uint32_t frame, >> static_cast<rkisp1_cif_isp_histogram_mode>(hstConfig->mode)); >> } >> >> -void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, >> - ControlList &metadata) >> -{ >> - utils::Duration exposureTime = context.configuration.sensor.lineDuration >> - * frameContext.sensor.exposure; >> - metadata.set(controls::AnalogueGain, frameContext.sensor.gain); >> - metadata.set(controls::ExposureTime, exposureTime.get<std::micro>()); >> - metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get<std::micro>()); >> - metadata.set(controls::ExposureTimeMode, >> - frameContext.agc.autoExposureEnabled >> - ? controls::ExposureTimeModeAuto >> - : controls::ExposureTimeModeManual); >> - metadata.set(controls::AnalogueGainMode, >> - frameContext.agc.autoGainEnabled >> - ? controls::AnalogueGainModeAuto >> - : controls::AnalogueGainModeManual); >> - >> - metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); >> - metadata.set(controls::AeExposureMode, frameContext.agc.exposureMode); >> - metadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode); >> - metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); >> -} >> - >> -/** >> - * \brief Process frame duration and compute vblank >> - * \param[in] context The shared IPA context >> - * \param[in] frameContext The current frame context >> - * \param[in] frameDuration The target frame duration >> - * >> - * Compute and populate vblank from the target frame duration. >> - */ >> -void Agc::processFrameDuration(IPAContext &context, >> - IPAFrameContext &frameContext, >> - utils::Duration frameDuration) >> -{ >> - IPACameraSensorInfo &sensorInfo = context.sensorInfo; >> - utils::Duration lineDuration = context.configuration.sensor.lineDuration; >> - >> - frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; >> - >> - /* Update frame duration accounting for line length quantization. */ >> - frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; >> -} >> - >> namespace { >> >> class AgcTraits final : public AgcMeanLuminance::Traits >> @@ -638,21 +345,6 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, >> IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats, >> ControlList &metadata) >> { >> - if (!stats) { >> - processFrameDuration(context, frameContext, >> - frameContext.agc.minFrameDuration); >> - fillMetadata(context, frameContext, metadata); >> - return; >> - } >> - >> - if (!(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP)) { >> - fillMetadata(context, frameContext, metadata); >> - LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; >> - return; >> - } >> - >> - const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; >> - >> /* >> * \todo Verify that the exposure and gain applied by the sensor for >> * this frame match what has been requested. This isn't a hard >> @@ -661,95 +353,46 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, >> * we receive), but is important in manual mode. >> */ >> >> - const rkisp1_cif_isp_stat *params = &stats->params; >> + const rkisp1_cif_isp_stat *params = nullptr; >> >> - /* >> - * Set the AGC limits using the fixed exposure time and/or gain in >> - * manual mode, or the sensor limits in auto mode. >> - */ >> - utils::Duration minExposureTime; >> - utils::Duration maxExposureTime; >> - double minAnalogueGain; >> - double maxAnalogueGain; >> - >> - if (frameContext.agc.autoExposureEnabled) { >> - minExposureTime = context.configuration.sensor.minExposureTime; >> - maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, >> - context.configuration.sensor.minExposureTime, >> - context.configuration.sensor.maxExposureTime); >> - } else { >> - minExposureTime = context.configuration.sensor.lineDuration >> - * frameContext.agc.exposure; >> - maxExposureTime = minExposureTime; >> + if (stats) { >> + if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) >> + params = &stats->params; >> + else >> + LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; >> } >> >> - if (frameContext.agc.autoGainEnabled) { >> - minAnalogueGain = context.configuration.sensor.minAnalogueGain; >> - maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; >> + if (params) { >> + std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; >> + if (context.activeState.wdr.mode != controls::WdrOff) >> + additionalConstraints.push_back(context.activeState.wdr.constraint); >> + >> + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {{ >> + .traits = AgcTraits{ >> + { params->ae.exp_mean, context.hw.numAeCells }, >> + meteringModes_.at(frameContext.agc.meteringMode), >> + }, >> + .yHist = { >> + /* The lower 4 bits are fractional and meant to be discarded. */ >> + { params->hist.hist_bins, context.hw.numHistogramBins }, >> + [](uint32_t x) { return x >> 4; }, >> + }, >> + .exposure = frameContext.sensor.exposure, >> + /* >> + * Include the quantization gain if it was applied. Do not use >> + * compress.gain because it will include gains that shall not be >> + * reported to the user when HDR is implemented. >> + */ >> + .gain = frameContext.sensor.gain >> + * (frameContext.compress.enable ? frameContext.agc.quantizationGain : 1), >> + .additionalConstraints = std::move(additionalConstraints), >> + .lux = frameContext.lux.lux, >> + }}, metadata); >> } else { >> - minAnalogueGain = frameContext.agc.gain; >> - maxAnalogueGain = frameContext.agc.gain; >> + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {}, metadata); >> } >> >> - std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; >> - if (context.activeState.wdr.mode != controls::WdrOff) >> - additionalConstraints.push_back(context.activeState.wdr.constraint); >> - >> - agc_.setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, >> - std::move(additionalConstraints)); >> - >> - /* >> - * The Agc algorithm needs to know the effective exposure value that was >> - * applied to the sensor when the statistics were collected. >> - */ >> - utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; >> - double analogueGain = frameContext.sensor.gain; >> - utils::Duration effectiveExposureValue = exposureTime * analogueGain; >> - >> - /* >> - * Include the quantization gain if it was applied. Do not use >> - * compress.gain because it will include gains that shall not be >> - * reported to the user when HDR is implemented. >> - */ >> - if (frameContext.compress.enable) >> - effectiveExposureValue *= frameContext.agc.quantizationGain; >> - >> - /* The lower 4 bits are fractional and meant to be discarded. */ >> - Histogram hist({ params->hist.hist_bins, context.hw.numHistogramBins }, >> - [](uint32_t x) { return x >> 4; }); >> - >> - const auto &newEv = agc_.calculateNewEv({ >> - .traits = AgcTraits{ >> - { params->ae.exp_mean, context.hw.numAeCells }, >> - meteringModes_.at(frameContext.agc.meteringMode), >> - }, >> - .yHist = hist, >> - .effectiveExposureValue = effectiveExposureValue, >> - .constraintModeIndex = frameContext.agc.constraintMode, >> - .exposureModeIndex = frameContext.agc.exposureMode, >> - .lux = frameContext.lux.lux, >> - .exposureCompensation = pow(2.0, frameContext.agc.exposureValue), >> - }); >> - >> - LOG(RkISP1Agc, Debug) >> - << "Divided up exposure time, analogue gain, quantization gain" >> - << " and digital gain are " << newEv.exposureTime << ", " << newEv.analogueGain >> - << ", " << newEv.quantizationGain << " and " << newEv.digitalGain; >> - >> - IPAActiveState &activeState = context.activeState; >> - /* Update the estimated exposure and gain. */ >> - activeState.agc.automatic.exposure = newEv.exposureTime / lineDuration; >> - activeState.agc.automatic.gain = newEv.analogueGain; >> - activeState.agc.automatic.quantizationGain = newEv.quantizationGain; >> - activeState.agc.automatic.yTarget = newEv.yTarget; >> - /* >> - * Expand the target frame duration so that we do not run faster than >> - * the minimum frame duration when we have short exposures. >> - */ >> - processFrameDuration(context, frameContext, >> - std::max(frameContext.agc.minFrameDuration, newEv.exposureTime)); >> - >> - fillMetadata(context, frameContext, metadata); >> + metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); >> } >> >> REGISTER_IPA_ALGORITHM(Agc, "Agc") >> diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h >> index 0527ca0d5f..3a4d7bc546 100644 >> --- a/src/ipa/rkisp1/algorithms/agc.h >> +++ b/src/ipa/rkisp1/algorithms/agc.h >> @@ -14,7 +14,7 @@ >> >> #include <libcamera/geometry.h> >> >> -#include "libipa/agc_mean_luminance.h" >> +#include "libipa/agc.h" >> >> #include "algorithm.h" >> >> @@ -47,14 +47,8 @@ private: >> uint8_t computeHistogramPredivider(const Size &size, >> enum rkisp1_cif_isp_histogram_mode mode); >> >> - void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, >> - ControlList &metadata); >> - void processFrameDuration(IPAContext &context, >> - IPAFrameContext &frameContext, >> - utils::Duration frameDuration); >> - >> std::map<int32_t, std::vector<uint8_t>> meteringModes_; >> - AgcMeanLuminance agc_; >> + AgcAlgorithm agc_; >> }; >> >> } /* namespace ipa::rkisp1::algorithms */ >> diff --git a/src/ipa/rkisp1/algorithms/lux.cpp b/src/ipa/rkisp1/algorithms/lux.cpp >> index 86e46c492f..ce6928a55d 100644 >> --- a/src/ipa/rkisp1/algorithms/lux.cpp >> +++ b/src/ipa/rkisp1/algorithms/lux.cpp >> @@ -74,7 +74,7 @@ void Lux::process(IPAContext &context, >> if (!stats) >> return; >> >> - utils::Duration exposureTime = context.configuration.sensor.lineDuration * >> + utils::Duration exposureTime = context.configuration.agc.lineDuration * >> frameContext.sensor.exposure; >> double gain = frameContext.sensor.gain; >> >> diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp >> index 1f94afda6b..47691674ad 100644 >> --- a/src/ipa/rkisp1/ipa_context.cpp >> +++ b/src/ipa/rkisp1/ipa_context.cpp >> @@ -86,21 +86,6 @@ namespace libcamera::ipa::rkisp1 { >> * \var IPASessionConfiguration::sensor >> * \brief Sensor-specific configuration of the IPA >> * >> - * \var IPASessionConfiguration::sensor.minExposureTime >> - * \brief Minimum exposure time supported with the sensor >> - * >> - * \var IPASessionConfiguration::sensor.maxExposureTime >> - * \brief Maximum exposure time supported with the sensor >> - * >> - * \var IPASessionConfiguration::sensor.minAnalogueGain >> - * \brief Minimum analogue gain supported with the sensor >> - * >> - * \var IPASessionConfiguration::sensor.maxAnalogueGain >> - * \brief Maximum analogue gain supported with the sensor >> - * >> - * \var IPASessionConfiguration::sensor.lineDuration >> - * \brief Line duration in microseconds >> - * >> * \var IPASessionConfiguration::sensor.size >> * \brief Sensor output resolution >> */ >> @@ -147,49 +132,8 @@ namespace libcamera::ipa::rkisp1 { >> * \var IPAActiveState::agc >> * \brief State for the Automatic Gain Control algorithm >> * >> - * The \a automatic variables track the latest values computed by algorithm >> - * based on the latest processed statistics. All other variables track the >> - * consolidated controls requested in queued requests. >> - * >> - * \struct IPAActiveState::agc.manual >> - * \brief Manual exposure time and analog gain (set through requests) >> - * >> - * \var IPAActiveState::agc.manual.exposure >> - * \brief Manual exposure time expressed as a number of lines as set by the >> - * ExposureTime control >> - * >> - * \var IPAActiveState::agc.manual.gain >> - * \brief Manual analogue gain as set by the AnalogueGain control >> - * >> - * \struct IPAActiveState::agc.automatic >> - * \brief Automatic exposure time and analog gain (computed by the algorithm) >> - * >> - * \var IPAActiveState::agc.automatic.exposure >> - * \brief Automatic exposure time expressed as a number of lines >> - * >> - * \var IPAActiveState::agc.automatic.gain >> - * \brief Automatic analogue gain multiplier >> - * >> - * \var IPAActiveState::agc.autoExposureEnabled >> - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control >> - * >> - * \var IPAActiveState::agc.autoGainEnabled >> - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control >> - * >> - * \var IPAActiveState::agc.constraintMode >> - * \brief Constraint mode as set by the AeConstraintMode control >> - * >> - * \var IPAActiveState::agc.exposureMode >> - * \brief Exposure mode as set by the AeExposureMode control >> - * >> * \var IPAActiveState::agc.meteringMode >> * \brief Metering mode as set by the AeMeteringMode control >> - * >> - * \var IPAActiveState::agc.minFrameDuration >> - * \brief Minimum frame duration as set by the FrameDurationLimits control >> - * >> - * \var IPAActiveState::agc.maxFrameDuration >> - * \brief Maximum frame duration as set by the FrameDurationLimits control >> */ >> >> /** >> @@ -314,53 +258,11 @@ namespace libcamera::ipa::rkisp1 { >> * the vertical blanking period is determined to maintain a consistent frame >> * rate matched to the FrameDurationLimits as set by the user. >> * >> - * \var IPAFrameContext::agc.exposure >> - * \brief Exposure time expressed as a number of lines computed by the algorithm >> - * >> - * \var IPAFrameContext::agc.gain >> - * \brief Analogue gain multiplier computed by the algorithm >> - * >> - * The gain should be adapted to the sensor specific gain code before applying. >> - * >> - * \var IPAFrameContext::agc.vblank >> - * \brief Vertical blanking parameter computed by the algorithm >> - * >> - * \var IPAFrameContext::agc.autoExposureEnabled >> - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control >> - * >> - * \var IPAFrameContext::agc.autoGainEnabled >> - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control >> - * >> - * \var IPAFrameContext::agc.constraintMode >> - * \brief Constraint mode as set by the AeConstraintMode control >> - * >> - * \var IPAFrameContext::agc.exposureMode >> - * \brief Exposure mode as set by the AeExposureMode control >> - * >> * \var IPAFrameContext::agc.meteringMode >> * \brief Metering mode as set by the AeMeteringMode control >> * >> - * \var IPAFrameContext::agc.minFrameDuration >> - * \brief Minimum frame duration as set by the FrameDurationLimits control >> - * >> - * \var IPAFrameContext::agc.maxFrameDuration >> - * \brief Maximum frame duration as set by the FrameDurationLimits control >> - * >> - * \var IPAFrameContext::agc.frameDuration >> - * \brief The actual FrameDuration used by the algorithm for the frame >> - * >> * \var IPAFrameContext::agc.updateMetering >> * \brief Indicate if new ISP AGC metering parameters need to be applied >> - * >> - * \var IPAFrameContext::agc.autoExposureModeChange >> - * \brief Indicate if autoExposureEnabled has changed from true in the previous >> - * frame to false in the current frame, and no manual exposure value has been >> - * supplied in the current frame. >> - * >> - * \var IPAFrameContext::agc.autoGainModeChange >> - * \brief Indicate if autoGainEnabled has changed from true in the previous >> - * frame to false in the current frame, and no manual gain value has been >> - * supplied in the current frame. >> */ >> >> /** >> diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h >> index cd213dd991..cc07bb9462 100644 >> --- a/src/ipa/rkisp1/ipa_context.h >> +++ b/src/ipa/rkisp1/ipa_context.h >> @@ -24,7 +24,7 @@ >> #include "libcamera/internal/matrix.h" >> #include "libcamera/internal/vector.h" >> >> -#include "libipa/agc_mean_luminance.h" >> +#include "libipa/agc.h" >> #include "libipa/awb.h" >> #include "libipa/camera_sensor_helper.h" >> #include "libipa/ccm.h" >> @@ -57,7 +57,7 @@ struct RKISP1AwbSession { >> }; >> >> struct IPASessionConfiguration { >> - struct { >> + struct Agc : agc::Session { >> struct rkisp1_cif_isp_window measureWindow; >> } agc; >> >> @@ -68,12 +68,6 @@ struct IPASessionConfiguration { >> } compress; >> >> struct { >> - utils::Duration minExposureTime; >> - utils::Duration maxExposureTime; >> - double minAnalogueGain; >> - double maxAnalogueGain; >> - >> - utils::Duration lineDuration; >> Size size; >> } sensor; >> >> @@ -82,26 +76,8 @@ struct IPASessionConfiguration { >> }; >> >> struct IPAActiveState { >> - struct { >> - struct { >> - uint32_t exposure; >> - double gain; >> - } manual; >> - struct { >> - uint32_t exposure; >> - double gain; >> - double quantizationGain; >> - double yTarget; >> - } automatic; >> - >> - bool autoExposureEnabled; >> - bool autoGainEnabled; >> - double exposureValue; >> - controls::AeConstraintModeEnum constraintMode; >> - controls::AeExposureModeEnum exposureMode; >> + struct Agc : agc::ActiveState { >> controls::AeMeteringModeEnum meteringMode; >> - utils::Duration minFrameDuration; >> - utils::Duration maxFrameDuration; >> } agc; >> >> ipa::awb::ActiveState awb; >> @@ -145,24 +121,9 @@ struct IPAActiveState { >> }; >> >> struct IPAFrameContext : public FrameContext { >> - struct { >> - uint32_t exposure; >> - double gain; >> - double exposureValue; >> - double quantizationGain; >> - uint32_t vblank; >> - double yTarget; >> - bool autoExposureEnabled; >> - bool autoGainEnabled; >> - controls::AeConstraintModeEnum constraintMode; >> - controls::AeExposureModeEnum exposureMode; >> + struct Agc : agc::FrameContext { >> controls::AeMeteringModeEnum meteringMode; >> - utils::Duration minFrameDuration; >> - utils::Duration maxFrameDuration; >> - utils::Duration frameDuration; >> bool updateMetering; >> - bool autoExposureModeChange; >> - bool autoGainModeChange; >> } agc; >> >> ipa::awb::FrameContext awb; >> -- >> 2.55.0 >>
diff --git a/src/ipa/libipa/agc.cpp b/src/ipa/libipa/agc.cpp index 864b73d5f1..e311bfd0d6 100644 --- a/src/ipa/libipa/agc.cpp +++ b/src/ipa/libipa/agc.cpp @@ -1,21 +1,41 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* - * Copyright (C) 2026 Ideas On Board + * Copyright (C) 2021-2026 Ideas On Board * - * Auto exposure/gain algorithm for implementing the IPA-specific AGC algorithms + * libIPA Agc algorithm */ #include "agc.h" +#include <algorithm> +#include <array> +#include <chrono> +#include <cmath> +#include <optional> +#include <ratio> + +#include <linux/v4l2-controls.h> + +#include <libcamera/base/log.h> +#include <libcamera/base/span.h> + +#include <libcamera/control_ids.h> +#include <libcamera/controls.h> + +#include <libcamera/ipa/core_ipa_interface.h> + /** * \file agc.h * \brief libipa AGC algorithm */ - namespace libcamera { namespace ipa { +using namespace std::chrono_literals; + +LOG_DEFINE_CATEGORY(Agc) + namespace agc { /** @@ -44,8 +64,711 @@ namespace agc { * otherwise the gain value will be used directly. */ +/** + * \struct Session + * \brief Session configuration for AgcAlgorithm + * + * \var Session::minExposureTime + * \brief Minimum exposure time for the streaming session + * + * \var Session::maxExposureTime + * \brief Maximum exposure time for the streaming session + * + * \var Session::minAnalogueGain + * \brief Minimum analogue gain for the streaming session + * + * \var Session::maxAnalogueGain + * \brief Maximum analogue gain for the streaming session + * + * \var Session::minFrameDuration + * \brief Minimum frame duration for the streaming session + * + * \var Session::maxFrameDuration + * \brief Maximum frame duration for the streaming session + * + * \var Session::lineDuration + * \brief Line duration for the streaming session + * + * \var Session::sensor + * \brief Details of the sensor configuration + * + * \var Session::sensor.outputSize + * \brief Configured output size of the sensor + * + * \var Session::autoAllowed + * \copybrief AgcAlgorithm::ConfigurationParams::autoAllowed + * \sa AgcAlgorithm::ConfigurationParams::autoAllowed + */ + +/** + * \struct ActiveState + * \brief Active state for AgcAlgorithm + * + * The \a automatic variables track the latest values computed by algorithm + * based on the latest processed statistics. All other variables track the + * consolidated controls requested in queued requests. + * + * \var ActiveState::manual + * \brief Manual exposure time and analog gain (set through requests) + * + * \var ActiveState::manual.exposure + * \brief Manual exposure time expressed as a number of lines as set by the + * ExposureTime control + * + * \var ActiveState::manual.gain + * \brief Manual analogue gain as set by the AnalogueGain control + * + * \var ActiveState::automatic + * \brief Automatic exposure time and analog gain (computed by the algorithm) + * + * \var ActiveState::automatic.exposure + * \brief Automatic exposure time expressed as a number of lines + * + * \var ActiveState::automatic.gain + * \brief Automatic analogue gain multiplier + * + * \var ActiveState::automatic.quantizationGain + * \brief Automatic quantization gain multiplier + * + * \var ActiveState::automatic.yTarget + * \brief Automatically determined luminance target + * + * \var ActiveState::autoExposureEnabled + * \brief Whether automatic exposure control is enabled by the ExposureTimeMode control + * + * \var ActiveState::autoGainEnabled + * \brief Whether automatic gain control is enabled by the AnalogueGainMode control + * + * \var ActiveState::exposureValue + * \brief Exposure value as set by the ExposureValue control + * + * \var ActiveState::constraintMode + * \brief Constraint mode as set by the AeConstraintMode control + * + * \var ActiveState::exposureMode + * \brief Exposure mode as set by the AeExposureMode control + * + * \var ActiveState::minFrameDuration + * \brief Minimum frame duration as set by the FrameDurationLimits control + * + * \var ActiveState::maxFrameDuration + * \brief Maximum frame duration as set by the FrameDurationLimits control + */ + +/** + * \struct FrameContext + * \brief Per-frame context for AgcAlgorithm + * + * \var FrameContext::exposure + * \brief Exposure time expressed as a number of lines computed by the algorithm + * + * \var FrameContext::gain + * \brief Analogue gain multiplier computed by the algorithm + * + * The gain should be translated to the sensor specific gain code before applying. + * + * \var FrameContext::quantizationGain + * \brief Quantization gain multiplier computed by the algorithm + * + * \var FrameContext::exposureValue + * \brief Exposure value as set by the ExposureValue control + * + * \var FrameContext::yTarget + * \brief Luminance target computed by the algorithm + * + * \var FrameContext::vblank + * \brief Vertical blanking parameter computed by the algorithm + * + * \var FrameContext::autoExposureEnabled + * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control + * + * \var FrameContext::autoGainEnabled + * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control + * + * \var FrameContext::constraintMode + * \brief Constraint mode as set by the AeConstraintMode control + * + * \var FrameContext::exposureMode + * \brief Exposure mode as set by the AeExposureMode control + * + * \var FrameContext::minFrameDuration + * \brief Minimum frame duration as set by the FrameDurationLimits control + * + * \var FrameContext::maxFrameDuration + * \brief Maximum frame duration as set by the FrameDurationLimits control + * + * \var FrameContext::frameDuration + * \brief The actual FrameDuration used by the algorithm for the frame + * + * \var FrameContext::autoExposureModeChange + * \brief Indicate if autoExposureEnabled has changed from true in the previous + * frame to false in the current frame, and no manual exposure value has been + * supplied in the current frame + * + * \var FrameContext::autoGainModeChange + * \brief Indicate if autoGainEnabled has changed from true in the previous + * frame to false in the current frame, and no manual gain value has been + * supplied in the current frame + */ + } /* namespace agc */ +/** + * \class AgcAlgorithm + * \brief libIPA LSC algorithm algorithm + * + * The AgcAlgorithm class can be used to implement automatic exposure/gain + * control in an IPA module, conforming to the prescribed Algorithm interface. + * Internally AgcMeanLuminance is used, this class merely concerns itself with + * processing the sensor properties, establishing limits, managing controls, + * providing the resulting metadata, and driving the actual algorithm in + * process(). + * + * Users should compose the agc::Session, agc::ActiveState, and agc::FrameContext + * structures into their session configuration, active state, and frame contexts, + * respectively. Furthermore, in their implementation of the Algorithm virtual + * function, they should simply call the identically named member function of + * AgcAlgorithm. + * + * \todo DigitalGain, DigitalGainMode + * \todo Expand documentation + */ + +/** + * \struct AgcAlgorithm::ConfigurationParams + * \brief Parameters for AgcAlgorithm::configure() + * + * \var AgcAlgorithm::ConfigurationParams::sensor + * \brief CameraSensorHelper for the sensor + * + * \var AgcAlgorithm::ConfigurationParams::sensorInfo + * \brief Current configuration of the sensor + * + * \var AgcAlgorithm::ConfigurationParams::sensorControls + * \brief ControlInfoMap of the sensor + * + * \var AgcAlgorithm::ConfigurationParams::ctrlMap + * \brief ControlInfoMap::Map to update with controls + * + * \var AgcAlgorithm::ConfigurationParams::autoAllowed + * \brief Whether to enable auto controls + * + * If \a false, the algorithm is set up for manual exposure and gain + * control only, without automatic adjustments. In this mode statistics + * must not be provided to AgcAlgorithm::process(), and ExposureTimeMode + * and AnalogueGainMode will only advertise manual control. + */ + +/** + * \struct AgcAlgorithm::ProcessParams + * \brief Parameters for AgcAlgorithm::process() + * + * \var AgcAlgorithm::ProcessParams::traits + * \brief Implementation of AgcMeanLuminance::Traits + * + * \var AgcAlgorithm::ProcessParams::yHist + * \brief Luminance histogram of the frame + * + * \var AgcAlgorithm::ProcessParams::exposure + * \brief Effective exposure of the frame + * + * \var AgcAlgorithm::ProcessParams::gain + * \brief Effective gain of the frame + * + * \var AgcAlgorithm::ProcessParams::additionalConstraints + * \brief Additional AgcMeanLuminance::AgcConstraints to apply + * + * \var AgcAlgorithm::ProcessParams::lux + * \brief Effective lux value of the frame + */ + +/** + * \brief Load tuning data and configure + * \param[in] tuningData The tuning data + * \param[in] config The algorithm configuration + * + * This function loads the tuning data and configures the algorithm as if + * by a call to configure(), in order to provide the available controls + * in ConfigurationParams::ctrlMap. + * + * The tuning data format is that of AgcMeanLuminance. Refer to + * AgcMeanLuminance::parseTuningData() for more details. + * + * \return 0 on success, or a negative error code + * + * \sa Algorithm::init() + */ +int AgcAlgorithm::init(const ValueNode &tuningData, const ConfigurationParams &config) +{ + int ret = impl_.parseTuningData(tuningData); + if (ret) + return ret; + + /* + * The purpose of this `configure()` is merely to provide the + * available controls in `config.ctrlMap`. + * + * \todo Remove it once IPA modules have been changed to + * configure the algorithms during initialization. + */ + + agc::Session dummySession; + agc::ActiveState dummyState; + + return configure(dummySession, dummyState, config); +} + +/** + * \brief Initialize the session configuration and active state + * \param[in] session The agc session configuration + * \param[in] state The agc active state + * \param[in] config The algorithm configuration + * + * This function initializes \a session and \a state based on the tuning + * data loaded by init() and the configuration in \a config. + * + * It also updates ConfigurationParams::ctrlMap with the limits of the various + * available agc-related controls. Users are expected to propagate these controls + * to the camera. + * + * \return 0 on success, or a negative error code + * + * \sa Algorithm::configure() + */ +int AgcAlgorithm::configure(agc::Session &session, agc::ActiveState &state, + const ConfigurationParams &config) +{ + session = {}; + session.autoAllowed = config.autoAllowed; + session.lineDuration = + config.sensorInfo.minLineLength * 1.0s / config.sensorInfo.pixelRate; + session.sensor.outputSize = config.sensorInfo.outputSize; + + const double lineDurationUs = session.lineDuration.get<std::micro>(); + + /* + * Compute exposure time limits from the V4L2_CID_EXPOSURE control + * limits and the line duration. + */ + + const ControlInfo &v4l2Exposure = config.sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get<int32_t>(); + int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); + int32_t defExposure = v4l2Exposure.def().get<int32_t>(); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = config.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = config.sensor->gain(v4l2Gain.min().get<int32_t>()); + float maxGain = config.sensor->gain(v4l2Gain.max().get<int32_t>()); + float defGain = config.sensor->gain(v4l2Gain.def().get<int32_t>()); + + LOG(Agc, Debug) + << "Exposure: [" << minExposure << ", " << maxExposure + << "], gain: [" << minGain << ", " << maxGain << "]"; + + /* + * Compute the frame duration limits. + * + * The frame length is computed assuming a fixed line length combined + * with the vertical frame sizes. + */ + const ControlInfo &v4l2HBlank = config.sensorControls.find(V4L2_CID_HBLANK)->second; + uint32_t hblank = v4l2HBlank.def().get<int32_t>(); + uint32_t lineLength = config.sensorInfo.outputSize.width + hblank; + + const ControlInfo &v4l2VBlank = config.sensorControls.find(V4L2_CID_VBLANK)->second; + std::array<uint32_t, 3> frameHeights{ + v4l2VBlank.min().get<int32_t>() + config.sensorInfo.outputSize.height, + v4l2VBlank.max().get<int32_t>() + config.sensorInfo.outputSize.height, + v4l2VBlank.def().get<int32_t>() + config.sensorInfo.outputSize.height, + }; + + std::array<int64_t, 3> frameDurations; + for (unsigned int i = 0; i < frameHeights.size(); ++i) { + uint64_t frameSize = lineLength * frameHeights[i]; + frameDurations[i] = frameSize / (config.sensorInfo.pixelRate / 1000000U); + } + + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for exposure time and analogue gain. As it depends + * on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum exposure time + */ + session.minExposureTime = minExposure * session.lineDuration; + session.maxExposureTime = maxExposure * session.lineDuration; + session.minAnalogueGain = minGain; + session.maxAnalogueGain = maxGain; + session.minFrameDuration = std::chrono::microseconds(frameDurations[0]); + session.maxFrameDuration = std::chrono::microseconds(frameDurations[1]); + + impl_.configure(session.lineDuration, config.sensor); + impl_.setLimits(session.minExposureTime, session.maxExposureTime, + session.minAnalogueGain, session.maxAnalogueGain, + {}); + impl_.resetFrameCount(); + + /* Configure the default exposure and gain. */ + state = {}; + state.automatic.gain = session.minAnalogueGain; + state.automatic.exposure = 10ms / session.lineDuration; + state.automatic.quantizationGain = 1; + state.automatic.yTarget = impl_.effectiveYTarget(0, 1); + state.manual.gain = state.automatic.gain; + state.manual.exposure = state.automatic.exposure; + state.autoExposureEnabled = session.autoAllowed; + state.autoGainEnabled = session.autoAllowed; + state.exposureValue = 0; + state.constraintMode = + static_cast<controls::AeConstraintModeEnum>(impl_.constraintModes().begin()->first); + state.exposureMode = + static_cast<controls::AeExposureModeEnum>(impl_.exposureModeHelpers().begin()->first); + state.minFrameDuration = session.minFrameDuration; + state.maxFrameDuration = session.maxFrameDuration; + + /* \todo Move this to the `Camera` class. */ + config.ctrlMap[&controls::AeEnable] = ControlInfo{ + false, session.autoAllowed, session.autoAllowed + }; + config.ctrlMap[&controls::AnalogueGain] = ControlInfo{ + minGain, maxGain, defGain + }; + config.ctrlMap[&controls::ExposureTime] = ControlInfo{ + static_cast<int32_t>(minExposure * lineDurationUs), + static_cast<int32_t>(maxExposure * lineDurationUs), + static_cast<int32_t>(defExposure * lineDurationUs), + }; + config.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ + frameDurations[0], frameDurations[1], + Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, + }; + config.ctrlMap[&controls::ExposureTimeMode] = ControlInfo{ + {{ controls::ExposureTimeModeAuto, controls::ExposureTimeModeManual }}, + controls::ExposureTimeModeAuto, + }; + config.ctrlMap[&controls::AnalogueGainMode] = ControlInfo{ + {{ controls::AnalogueGainModeAuto, controls::AnalogueGainModeManual }}, + controls::AnalogueGainModeAuto, + }; + config.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + config.ctrlMap.merge(impl_.controls()); + + return 0; +} + +/** + * \brief Queue a request + * \param[in] session The agc session configuration + * \param[in] state The agc active state + * \param[in] frameContext The agc frame context + * \param[in] controls The list of controls associated with a Request + * + * This functions processes the agc-related controls in \a controls for the frame + * denoted by \a frameContext, and updates \a state and \a frameContext accordingly. + * + * \sa Algorithm::queueRequest() + */ +void AgcAlgorithm::queueRequest(const agc::Session &session, agc::ActiveState &state, + agc::FrameContext &frameContext, const ControlList &controls) +{ + if (session.autoAllowed) { + const auto &aeEnable = controls.get(controls::ExposureTimeMode); + if (aeEnable && + (*aeEnable == controls::ExposureTimeModeAuto) != state.autoExposureEnabled) { + state.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); + + LOG(Agc, Debug) + << (state.autoExposureEnabled ? "Enabling" : "Disabling") + << " AGC (exposure)"; + + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + * + * \todo Check the previous frame at prepare() time + * instead of saving a flag here + */ + if (!state.autoExposureEnabled && !controls.get(controls::ExposureTime)) + frameContext.autoExposureModeChange = true; + } + + const auto &agEnable = controls.get(controls::AnalogueGainMode); + if (agEnable && + (*agEnable == controls::AnalogueGainModeAuto) != state.autoGainEnabled) { + state.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); + + LOG(Agc, Debug) + << (state.autoGainEnabled ? "Enabling" : "Disabling") + << " AGC (gain)"; + + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + */ + if (!state.autoGainEnabled && !controls.get(controls::AnalogueGain)) + frameContext.autoGainModeChange = true; + } + } + + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure && !state.autoExposureEnabled) { + state.manual.exposure = *exposure * 1.0us / session.lineDuration; + + LOG(Agc, Debug) << "Set exposure to " << state.manual.exposure; + } + + const auto &gain = controls.get(controls::AnalogueGain); + if (gain && !state.autoGainEnabled) { + state.manual.gain = *gain; + + LOG(Agc, Debug) << "Set gain to " << state.manual.gain; + } + + frameContext.autoExposureEnabled = state.autoExposureEnabled; + frameContext.autoGainEnabled = state.autoGainEnabled; + + if (!frameContext.autoExposureEnabled) + frameContext.exposure = state.manual.exposure; + if (!frameContext.autoGainEnabled) + frameContext.gain = state.manual.gain; + + if (!frameContext.autoExposureEnabled && !frameContext.autoGainEnabled) + frameContext.quantizationGain = 1.0; + + const auto &exposureMode = controls.get(controls::AeExposureMode); + if (exposureMode) + state.exposureMode = + static_cast<controls::AeExposureModeEnum>(*exposureMode); + frameContext.exposureMode = state.exposureMode; + + const auto &constraintMode = controls.get(controls::AeConstraintMode); + if (constraintMode) + state.constraintMode = + static_cast<controls::AeConstraintModeEnum>(*constraintMode); + frameContext.constraintMode = state.constraintMode; + + const auto &exposureValue = controls.get(controls::ExposureValue); + if (exposureValue) + state.exposureValue = *exposureValue; + frameContext.exposureValue = state.exposureValue; + + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); + if (frameDurationLimits) { + /* Limit the control value to the limits in ControlInfo */ + state.minFrameDuration = std::clamp<utils::Duration>( + std::chrono::microseconds((*frameDurationLimits).front()), + session.minFrameDuration, session.maxFrameDuration); + + state.maxFrameDuration = std::clamp<utils::Duration>( + std::chrono::microseconds((*frameDurationLimits).back()), + session.minFrameDuration, session.maxFrameDuration); + } + frameContext.minFrameDuration = state.minFrameDuration; + frameContext.maxFrameDuration = state.maxFrameDuration; +} + +/** + * \brief Prepare a frame + * \param[in] state The agc active state + * \param[in] frameContext The agc frame context + * + * This function prepares the parameters for the frame denoted by \a frameContext. + * After a call to this function, the values of \ref agc::FrameContext::exposure + * "frameContext.exposure" and \ref agc::FrameContext::gain "frameContext.gain" + * will be finalized and may be used by the caller (see agc::prepareControls()). + * + * \todo Finalize \ref agc::FrameContext::vblank "frameContext.vblank" as well + * + * \sa Algorithm::prepare() + */ +void AgcAlgorithm::prepare(agc::ActiveState &state, agc::FrameContext &frameContext) +{ + uint32_t activeAutoExposure = state.automatic.exposure; + double activeAutoGain = state.automatic.gain; + double activeAutoQGain = state.automatic.quantizationGain; + + /* Populate exposure and gain in auto mode */ + if (frameContext.autoExposureEnabled) { + frameContext.exposure = activeAutoExposure; + frameContext.quantizationGain = activeAutoQGain; + } + if (frameContext.autoGainEnabled) { + frameContext.gain = activeAutoGain; + frameContext.quantizationGain = activeAutoQGain; + } + + /* + * Populate manual exposure and gain from the active auto values when + * transitioning from auto to manual + */ + if (!frameContext.autoExposureEnabled && frameContext.autoExposureModeChange) { + state.manual.exposure = activeAutoExposure; + frameContext.exposure = activeAutoExposure; + } + if (!frameContext.autoGainEnabled && frameContext.autoGainModeChange) { + state.manual.gain = activeAutoGain; + frameContext.gain = activeAutoGain; + frameContext.quantizationGain = activeAutoQGain; + } + + frameContext.yTarget = state.automatic.yTarget; +} + +/** + * \brief Process frame statistics + * \param[in] session The agc session configuration + * \param[in] state The agc active state + * \param[in] frameContext The agc frame context + * \param[in] params The algorithm parameters + * \param[in] metadata The list of metadata + * + * This function processes the statistics for the completed frame denoted by + * \a frameContext, runs the AGC implementation, and updates \a state appropriately. + * This function also populates \a metadata for the completed frame. + * + * \a params must be omitted if the session was configured without "autoAllowed", + * and it may be omitted even if auto control is enabled, for example, if the + * statistics could not be delivered due to some ephemeral error. This ensures + * that the algorithm state will not go out of sync, and that metadata is produced + * as expected. + * + * Care must be taken to convert the platform specific statistics to the format + * expected in \a params. See ProcessParams for the details. + * + * \sa Algorithm::process() + */ +void AgcAlgorithm::process(const agc::Session &session, agc::ActiveState &state, + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, + ControlList &metadata) +{ + if (!params) { + processFrameDuration(session, frameContext, frameContext.minFrameDuration); + fillMetadata(session, frameContext, metadata); + return; + } + + ASSERT(session.autoAllowed); + + const utils::Duration &lineDuration = session.lineDuration; + + /* + * Set the AGC limits using the fixed exposure time and/or gain in + * manual mode, or the sensor limits in auto mode. + */ + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + /* \todo This uses the configuration from an already completed frame. */ + + if (frameContext.autoExposureEnabled) { + minExposureTime = session.minExposureTime; + maxExposureTime = std::clamp(frameContext.maxFrameDuration, + session.minExposureTime, + session.maxExposureTime); + } else { + minExposureTime = lineDuration * frameContext.exposure; + maxExposureTime = minExposureTime; + } + + if (frameContext.autoGainEnabled) { + minAnalogueGain = session.minAnalogueGain; + maxAnalogueGain = session.maxAnalogueGain; + } else { + minAnalogueGain = frameContext.gain; + maxAnalogueGain = frameContext.gain; + } + + /* + * The Agc algorithm needs to know the effective exposure value that was + * applied to the sensor when the statistics were collected. + */ + utils::Duration effectiveExposureValue = + lineDuration * params->exposure * params->gain; + + impl_.setLimits(minExposureTime, maxExposureTime, + minAnalogueGain, maxAnalogueGain, + std::move(params->additionalConstraints)); + + const auto &newEv = impl_.calculateNewEv({ + .traits = params->traits, + .yHist = params->yHist, + .effectiveExposureValue = effectiveExposureValue, + .constraintModeIndex = frameContext.constraintMode, + .exposureModeIndex = frameContext.exposureMode, + .lux = params->lux, + .exposureCompensation = std::pow(2.0, frameContext.exposureValue), + }); + + /* Update the estimated exposure and gain. */ + state.automatic.exposure = newEv.exposureTime / lineDuration; + state.automatic.gain = newEv.analogueGain; + state.automatic.quantizationGain = newEv.quantizationGain; + state.automatic.yTarget = newEv.yTarget; + + LOG(Agc, Debug) + << "Divided up exposure time, analogue gain, quantization gain" + << " and digital gain are " << newEv.exposureTime + << ", " << state.automatic.gain << ", " << state.automatic.quantizationGain + << " and " << newEv.digitalGain; + + /* + * Expand the target frame duration so that we do not run faster than + * the minimum frame duration when we have short exposures. + */ + processFrameDuration(session, frameContext, + std::max(frameContext.minFrameDuration, newEv.exposureTime)); + + fillMetadata(session, frameContext, metadata); +} + +/** + * \brief Process frame duration and compute vblank + * \param[in] session The session parameters + * \param[in] frameContext The current frame context + * \param[in] frameDuration The target frame duration + * + * Compute and populate vblank from the target frame duration. + */ +void AgcAlgorithm::processFrameDuration(const agc::Session &session, + agc::FrameContext &frameContext, + utils::Duration frameDuration) +{ + const utils::Duration &lineDuration = session.lineDuration; + + frameContext.vblank = + (frameDuration / lineDuration) - session.sensor.outputSize.height; + + /* Update frame duration accounting for line length quantization. */ + frameContext.frameDuration = + (session.sensor.outputSize.height + frameContext.vblank) * lineDuration; +} + +void AgcAlgorithm::fillMetadata(const agc::Session &session, + const agc::FrameContext &frameContext, + ControlList &metadata) +{ + + metadata.set(controls::AnalogueGain, frameContext.gain); + metadata.set(controls::ExposureTime, + utils::Duration(session.lineDuration * frameContext.exposure).get<std::micro>()); + metadata.set(controls::FrameDuration, frameContext.frameDuration.get<std::micro>()); + metadata.set(controls::ExposureTimeMode, frameContext.autoExposureEnabled + ? controls::ExposureTimeModeAuto + : controls::ExposureTimeModeManual); + metadata.set(controls::AnalogueGainMode, frameContext.autoGainEnabled + ? controls::AnalogueGainModeAuto + : controls::AnalogueGainModeManual); + + metadata.set(controls::AeExposureMode, frameContext.exposureMode); + metadata.set(controls::AeConstraintMode, frameContext.constraintMode); + metadata.set(controls::ExposureValue, frameContext.exposureValue); +} + } /* namespace ipa */ } /* namespace libcamera */ diff --git a/src/ipa/libipa/agc.h b/src/ipa/libipa/agc.h index 5a67464086..06600f0208 100644 --- a/src/ipa/libipa/agc.h +++ b/src/ipa/libipa/agc.h @@ -7,21 +7,87 @@ #pragma once +#include <optional> #include <stdint.h> #include <utility> +#include <vector> #include <linux/v4l2-controls.h> +#include <libcamera/base/utils.h> + +#include <libcamera/control_ids.h> #include <libcamera/controls.h> +#include <libcamera/geometry.h> +#include "agc_mean_luminance.h" #include "camera_sensor_helper.h" namespace libcamera { +struct IPACameraSensorInfo; + namespace ipa { +class Histogram; + namespace agc { +struct Session { + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + utils::Duration lineDuration; + + struct { + Size outputSize; + } sensor; + + bool autoAllowed; +}; + +struct ActiveState { + struct { + uint32_t exposure; + double gain; + } manual; + struct { + uint32_t exposure; + double gain; + double quantizationGain; + double yTarget; + } automatic; + + bool autoExposureEnabled; + bool autoGainEnabled; + double exposureValue; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; +}; + +struct FrameContext { + uint32_t exposure; + double gain; + double quantizationGain; + double exposureValue; + double yTarget; + uint32_t vblank; + bool autoExposureEnabled; + bool autoGainEnabled; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + utils::Duration frameDuration; + bool autoExposureModeChange; + bool autoGainModeChange; +}; + [[nodiscard]] inline std::pair<uint32_t, double> extractControls(const ControlList &controls, const CameraSensorHelper *sensor) @@ -48,6 +114,51 @@ prepareControls(ControlList &controls, const CameraSensorHelper *sensor, } /* namespace agc */ +class AgcAlgorithm +{ +public: + struct ConfigurationParams { + const CameraSensorHelper *sensor; + const IPACameraSensorInfo &sensorInfo; + const ControlInfoMap &sensorControls; + ControlInfoMap::Map &ctrlMap; + bool autoAllowed = true; + }; + + struct ProcessParams { + const AgcMeanLuminance::Traits &traits; + const Histogram &yHist; + uint32_t exposure; + double gain; + std::vector<AgcMeanLuminance::AgcConstraint> &&additionalConstraints = {}; + double lux = 0; + }; + + int init(const ValueNode &tuningData, const ConfigurationParams &config); + + int configure(agc::Session &session, agc::ActiveState &state, + const ConfigurationParams &config); + + void queueRequest(const agc::Session &session, agc::ActiveState &state, + agc::FrameContext &frameContext, const ControlList &controls); + + void prepare(agc::ActiveState &state, agc::FrameContext &frameContext); + + void process(const agc::Session &session, agc::ActiveState &state, + agc::FrameContext &frameContext, std::optional<ProcessParams> &¶ms, + ControlList &metadata); + +private: + void processFrameDuration(const agc::Session &session, + agc::FrameContext &frameContext, + utils::Duration frameDuration); + void fillMetadata(const agc::Session &session, + const agc::FrameContext &frameContext, + ControlList &metadata); + + AgcMeanLuminance impl_; +}; + } /* namespace ipa */ } /* namespace libcamera */ diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 7230232d24..bb0345ca87 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -8,10 +8,7 @@ #include "agc.h" #include <algorithm> -#include <array> -#include <chrono> #include <cmath> -#include <tuple> #include <vector> #include <libcamera/base/log.h> @@ -36,89 +33,6 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Agc) -namespace { - -void reconfigure(IPAContext &context) -{ - context.configuration.sensor.lineDuration = - context.sensorInfo.minLineLength * 1.0s / context.sensorInfo.pixelRate; - - double lineDurationUs = context.configuration.sensor.lineDuration.get<std::micro>(); - - /* - * Compute exposure time limits from the V4L2_CID_EXPOSURE control - * limits and the line duration. - */ - - const ControlInfo &v4l2Exposure = context.sensorControls.find(V4L2_CID_EXPOSURE)->second; - int32_t minExposure = v4l2Exposure.min().get<int32_t>(); - int32_t maxExposure = v4l2Exposure.max().get<int32_t>(); - int32_t defExposure = v4l2Exposure.def().get<int32_t>(); - context.ctrlMap[&controls::ExposureTime] = ControlInfo{ - static_cast<int32_t>(minExposure * lineDurationUs), - static_cast<int32_t>(maxExposure * lineDurationUs), - static_cast<int32_t>(defExposure * lineDurationUs), - }; - - /* Compute the analogue gain limits. */ - const ControlInfo &v4l2Gain = context.sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; - float minGain = context.camHelper->gain(v4l2Gain.min().get<int32_t>()); - float maxGain = context.camHelper->gain(v4l2Gain.max().get<int32_t>()); - float defGain = context.camHelper->gain(v4l2Gain.def().get<int32_t>()); - context.ctrlMap[&controls::AnalogueGain] = ControlInfo{ - minGain, - maxGain, - defGain, - }; - - LOG(RkISP1Agc, Debug) - << "Exposure: [" << minExposure << ", " << maxExposure - << "], gain: [" << minGain << ", " << maxGain << "]"; - - /* - * Compute the frame duration limits. - * - * The frame length is computed assuming a fixed line length combined - * with the vertical frame sizes. - */ - const ControlInfo &v4l2HBlank = context.sensorControls.find(V4L2_CID_HBLANK)->second; - uint32_t hblank = v4l2HBlank.def().get<int32_t>(); - uint32_t lineLength = context.sensorInfo.outputSize.width + hblank; - - const ControlInfo &v4l2VBlank = context.sensorControls.find(V4L2_CID_VBLANK)->second; - std::array<uint32_t, 3> frameHeights{ - v4l2VBlank.min().get<int32_t>() + context.sensorInfo.outputSize.height, - v4l2VBlank.max().get<int32_t>() + context.sensorInfo.outputSize.height, - v4l2VBlank.def().get<int32_t>() + context.sensorInfo.outputSize.height, - }; - - std::array<int64_t, 3> frameDurations; - for (unsigned int i = 0; i < frameHeights.size(); ++i) { - uint64_t frameSize = lineLength * frameHeights[i]; - frameDurations[i] = frameSize / (context.sensorInfo.pixelRate / 1000000U); - } - - context.ctrlMap[&controls::FrameDurationLimits] = ControlInfo{ - frameDurations[0], - frameDurations[1], - Span<const int64_t, 2>{ { frameDurations[2], frameDurations[2] } }, - }; - - /* - * When the AGC computes the new exposure values for a frame, it needs - * to know the limits for exposure time and analogue gain. As it depends - * on the sensor, update it with the controls. - * - * \todo take VBLANK into account for maximum exposure time - */ - context.configuration.sensor.minExposureTime = minExposure * context.configuration.sensor.lineDuration; - context.configuration.sensor.maxExposureTime = maxExposure * context.configuration.sensor.lineDuration; - context.configuration.sensor.minAnalogueGain = minGain; - context.configuration.sensor.maxAnalogueGain = maxGain; -} - -} /* namespace */ - /** * \class Agc * \brief A mean-based auto-exposure algorithm @@ -222,7 +136,12 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) { int ret; - ret = agc_.parseTuningData(tuningData); + ret = agc_.init(tuningData, { + .sensor = context.camHelper.get(), + .sensorInfo = context.sensorInfo, + .sensorControls = context.sensorControls, + .ctrlMap = context.ctrlMap, + }); if (ret) return ret; @@ -231,21 +150,6 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) if (ret) return ret; - context.ctrlMap[&controls::ExposureTimeMode] = - ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), - ControlValue(controls::ExposureTimeModeManual) } }, - ControlValue(controls::ExposureTimeModeAuto)); - context.ctrlMap[&controls::AnalogueGainMode] = - ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), - ControlValue(controls::AnalogueGainModeManual) } }, - ControlValue(controls::AnalogueGainModeAuto)); - /* \todo Move this to the Camera class */ - context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true, true); - context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); - context.ctrlMap.merge(agc_.controls()); - - reconfigure(context); - return 0; } @@ -258,47 +162,24 @@ int Agc::init(IPAContext &context, const ValueNode &tuningData) */ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - reconfigure(context); - - /* Configure the default exposure and gain. */ - context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; - context.activeState.agc.automatic.exposure = - 10ms / context.configuration.sensor.lineDuration; - context.activeState.agc.automatic.quantizationGain = 1.0; - context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; - context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; - context.activeState.agc.autoExposureEnabled = !context.configuration.raw; - context.activeState.agc.autoGainEnabled = !context.configuration.raw; - context.activeState.agc.exposureValue = 0.0; - - context.activeState.agc.constraintMode = - static_cast<controls::AeConstraintModeEnum>(agc_.constraintModes().begin()->first); - context.activeState.agc.exposureMode = - static_cast<controls::AeExposureModeEnum>(agc_.exposureModeHelpers().begin()->first); + int ret = agc_.configure(context.configuration.agc, context.activeState.agc, { + .sensor = context.camHelper.get(), + .sensorInfo = context.sensorInfo, + .sensorControls = context.sensorControls, + .ctrlMap = context.ctrlMap, + .autoAllowed = !context.configuration.raw, + }); + if (ret) + return ret; + context.activeState.agc.meteringMode = static_cast<controls::AeMeteringModeEnum>(meteringModes_.begin()->first); - /* Limit the frame duration to match current initialisation */ - ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; - context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get<int64_t>()); - context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get<int64_t>()); - context.configuration.agc.measureWindow.h_offs = 0; context.configuration.agc.measureWindow.v_offs = 0; context.configuration.agc.measureWindow.h_size = configInfo.outputSize.width; context.configuration.agc.measureWindow.v_size = configInfo.outputSize.height; - agc_.configure(context.configuration.sensor.lineDuration, context.camHelper.get()); - - agc_.setLimits(context.configuration.sensor.minExposureTime, - context.configuration.sensor.maxExposureTime, - context.configuration.sensor.minAnalogueGain, - context.configuration.sensor.maxAnalogueGain, {}); - - context.activeState.agc.automatic.yTarget = agc_.effectiveYTarget(0, 1); - - agc_.resetFrameCount(); - return 0; } @@ -312,73 +193,7 @@ void Agc::queueRequest(IPAContext &context, { auto &agc = context.activeState.agc; - if (!context.configuration.raw) { - const auto &aeEnable = controls.get(controls::ExposureTimeMode); - if (aeEnable && - (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { - agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); - - LOG(RkISP1Agc, Debug) - << (agc.autoExposureEnabled ? "Enabling" : "Disabling") - << " AGC (exposure)"; - - /* - * If we go from auto -> manual with no manual control - * set, use the last computed value, which we don't - * know until prepare() so save this information. - * - * \todo Check the previous frame at prepare() time - * instead of saving a flag here - */ - if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) - frameContext.agc.autoExposureModeChange = true; - } - - const auto &agEnable = controls.get(controls::AnalogueGainMode); - if (agEnable && - (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { - agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); - - LOG(RkISP1Agc, Debug) - << (agc.autoGainEnabled ? "Enabling" : "Disabling") - << " AGC (gain)"; - /* - * If we go from auto -> manual with no manual control - * set, use the last computed value, which we don't - * know until prepare() so save this information. - */ - if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) - frameContext.agc.autoGainModeChange = true; - } - } - - const auto &exposure = controls.get(controls::ExposureTime); - if (exposure && !agc.autoExposureEnabled) { - agc.manual.exposure = *exposure * 1.0us - / context.configuration.sensor.lineDuration; - - LOG(RkISP1Agc, Debug) - << "Set exposure to " << agc.manual.exposure; - } - - const auto &gain = controls.get(controls::AnalogueGain); - if (gain && !agc.autoGainEnabled) { - agc.manual.gain = *gain; - - LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; - } - - frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; - frameContext.agc.autoGainEnabled = agc.autoGainEnabled; - - if (!frameContext.agc.autoExposureEnabled) - frameContext.agc.exposure = agc.manual.exposure; - if (!frameContext.agc.autoGainEnabled) - frameContext.agc.gain = agc.manual.gain; - - if (!frameContext.agc.autoExposureEnabled && - !frameContext.agc.autoGainEnabled) - frameContext.agc.quantizationGain = 1.0; + agc_.queueRequest(context.configuration.agc, agc, frameContext.agc, controls); const auto &meteringMode = controls.get(controls::AeMeteringMode); if (meteringMode) { @@ -387,42 +202,6 @@ void Agc::queueRequest(IPAContext &context, static_cast<controls::AeMeteringModeEnum>(*meteringMode); } frameContext.agc.meteringMode = agc.meteringMode; - - const auto &exposureMode = controls.get(controls::AeExposureMode); - if (exposureMode) - agc.exposureMode = - static_cast<controls::AeExposureModeEnum>(*exposureMode); - frameContext.agc.exposureMode = agc.exposureMode; - - const auto &constraintMode = controls.get(controls::AeConstraintMode); - if (constraintMode) - agc.constraintMode = - static_cast<controls::AeConstraintModeEnum>(*constraintMode); - frameContext.agc.constraintMode = agc.constraintMode; - - const auto &exposureValue = controls.get(controls::ExposureValue); - if (exposureValue) - agc.exposureValue = *exposureValue; - frameContext.agc.exposureValue = agc.exposureValue; - - const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); - if (frameDurationLimits) { - /* Limit the control value to the limits in ControlInfo */ - ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; - int64_t minFrameDuration = - std::clamp((*frameDurationLimits).front(), - limits.min().get<int64_t>(), - limits.max().get<int64_t>()); - int64_t maxFrameDuration = - std::clamp((*frameDurationLimits).back(), - limits.min().get<int64_t>(), - limits.max().get<int64_t>()); - - agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); - agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); - } - frameContext.agc.minFrameDuration = agc.minFrameDuration; - frameContext.agc.maxFrameDuration = agc.maxFrameDuration; } /** @@ -431,41 +210,13 @@ void Agc::queueRequest(IPAContext &context, void Agc::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) { - uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; - double activeAutoGain = context.activeState.agc.automatic.gain; - double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; - - /* Populate exposure and gain in auto mode */ - if (frameContext.agc.autoExposureEnabled) { - frameContext.agc.exposure = activeAutoExposure; - frameContext.agc.quantizationGain = activeAutoQGain; - } - if (frameContext.agc.autoGainEnabled) { - frameContext.agc.gain = activeAutoGain; - frameContext.agc.quantizationGain = activeAutoQGain; - } - - /* - * Populate manual exposure and gain from the active auto values when - * transitioning from auto to manual - */ - if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { - context.activeState.agc.manual.exposure = activeAutoExposure; - frameContext.agc.exposure = activeAutoExposure; - } - if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { - context.activeState.agc.manual.gain = activeAutoGain; - frameContext.agc.gain = activeAutoGain; - frameContext.agc.quantizationGain = activeAutoQGain; - } + agc_.prepare(context.activeState.agc, frameContext.agc); if (context.configuration.compress.supported) { frameContext.compress.enable = true; frameContext.compress.gain = frameContext.agc.quantizationGain; } - frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; - if (frame > 0 && !frameContext.agc.updateMetering) return; @@ -521,50 +272,6 @@ void Agc::prepare(IPAContext &context, const uint32_t frame, static_cast<rkisp1_cif_isp_histogram_mode>(hstConfig->mode)); } -void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, - ControlList &metadata) -{ - utils::Duration exposureTime = context.configuration.sensor.lineDuration - * frameContext.sensor.exposure; - metadata.set(controls::AnalogueGain, frameContext.sensor.gain); - metadata.set(controls::ExposureTime, exposureTime.get<std::micro>()); - metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get<std::micro>()); - metadata.set(controls::ExposureTimeMode, - frameContext.agc.autoExposureEnabled - ? controls::ExposureTimeModeAuto - : controls::ExposureTimeModeManual); - metadata.set(controls::AnalogueGainMode, - frameContext.agc.autoGainEnabled - ? controls::AnalogueGainModeAuto - : controls::AnalogueGainModeManual); - - metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); - metadata.set(controls::AeExposureMode, frameContext.agc.exposureMode); - metadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode); - metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); -} - -/** - * \brief Process frame duration and compute vblank - * \param[in] context The shared IPA context - * \param[in] frameContext The current frame context - * \param[in] frameDuration The target frame duration - * - * Compute and populate vblank from the target frame duration. - */ -void Agc::processFrameDuration(IPAContext &context, - IPAFrameContext &frameContext, - utils::Duration frameDuration) -{ - IPACameraSensorInfo &sensorInfo = context.sensorInfo; - utils::Duration lineDuration = context.configuration.sensor.lineDuration; - - frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; - - /* Update frame duration accounting for line length quantization. */ - frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; -} - namespace { class AgcTraits final : public AgcMeanLuminance::Traits @@ -638,21 +345,6 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats, ControlList &metadata) { - if (!stats) { - processFrameDuration(context, frameContext, - frameContext.agc.minFrameDuration); - fillMetadata(context, frameContext, metadata); - return; - } - - if (!(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP)) { - fillMetadata(context, frameContext, metadata); - LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; - return; - } - - const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; - /* * \todo Verify that the exposure and gain applied by the sensor for * this frame match what has been requested. This isn't a hard @@ -661,95 +353,46 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * we receive), but is important in manual mode. */ - const rkisp1_cif_isp_stat *params = &stats->params; + const rkisp1_cif_isp_stat *params = nullptr; - /* - * Set the AGC limits using the fixed exposure time and/or gain in - * manual mode, or the sensor limits in auto mode. - */ - utils::Duration minExposureTime; - utils::Duration maxExposureTime; - double minAnalogueGain; - double maxAnalogueGain; - - if (frameContext.agc.autoExposureEnabled) { - minExposureTime = context.configuration.sensor.minExposureTime; - maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, - context.configuration.sensor.minExposureTime, - context.configuration.sensor.maxExposureTime); - } else { - minExposureTime = context.configuration.sensor.lineDuration - * frameContext.agc.exposure; - maxExposureTime = minExposureTime; + if (stats) { + if (stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP) + params = &stats->params; + else + LOG(RkISP1Agc, Error) << "AUTOEXP data is missing in statistics"; } - if (frameContext.agc.autoGainEnabled) { - minAnalogueGain = context.configuration.sensor.minAnalogueGain; - maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; + if (params) { + std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; + if (context.activeState.wdr.mode != controls::WdrOff) + additionalConstraints.push_back(context.activeState.wdr.constraint); + + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {{ + .traits = AgcTraits{ + { params->ae.exp_mean, context.hw.numAeCells }, + meteringModes_.at(frameContext.agc.meteringMode), + }, + .yHist = { + /* The lower 4 bits are fractional and meant to be discarded. */ + { params->hist.hist_bins, context.hw.numHistogramBins }, + [](uint32_t x) { return x >> 4; }, + }, + .exposure = frameContext.sensor.exposure, + /* + * Include the quantization gain if it was applied. Do not use + * compress.gain because it will include gains that shall not be + * reported to the user when HDR is implemented. + */ + .gain = frameContext.sensor.gain + * (frameContext.compress.enable ? frameContext.agc.quantizationGain : 1), + .additionalConstraints = std::move(additionalConstraints), + .lux = frameContext.lux.lux, + }}, metadata); } else { - minAnalogueGain = frameContext.agc.gain; - maxAnalogueGain = frameContext.agc.gain; + agc_.process(context.configuration.agc, context.activeState.agc, frameContext.agc, {}, metadata); } - std::vector<AgcMeanLuminance::AgcConstraint> additionalConstraints; - if (context.activeState.wdr.mode != controls::WdrOff) - additionalConstraints.push_back(context.activeState.wdr.constraint); - - agc_.setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, - std::move(additionalConstraints)); - - /* - * The Agc algorithm needs to know the effective exposure value that was - * applied to the sensor when the statistics were collected. - */ - utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; - double analogueGain = frameContext.sensor.gain; - utils::Duration effectiveExposureValue = exposureTime * analogueGain; - - /* - * Include the quantization gain if it was applied. Do not use - * compress.gain because it will include gains that shall not be - * reported to the user when HDR is implemented. - */ - if (frameContext.compress.enable) - effectiveExposureValue *= frameContext.agc.quantizationGain; - - /* The lower 4 bits are fractional and meant to be discarded. */ - Histogram hist({ params->hist.hist_bins, context.hw.numHistogramBins }, - [](uint32_t x) { return x >> 4; }); - - const auto &newEv = agc_.calculateNewEv({ - .traits = AgcTraits{ - { params->ae.exp_mean, context.hw.numAeCells }, - meteringModes_.at(frameContext.agc.meteringMode), - }, - .yHist = hist, - .effectiveExposureValue = effectiveExposureValue, - .constraintModeIndex = frameContext.agc.constraintMode, - .exposureModeIndex = frameContext.agc.exposureMode, - .lux = frameContext.lux.lux, - .exposureCompensation = pow(2.0, frameContext.agc.exposureValue), - }); - - LOG(RkISP1Agc, Debug) - << "Divided up exposure time, analogue gain, quantization gain" - << " and digital gain are " << newEv.exposureTime << ", " << newEv.analogueGain - << ", " << newEv.quantizationGain << " and " << newEv.digitalGain; - - IPAActiveState &activeState = context.activeState; - /* Update the estimated exposure and gain. */ - activeState.agc.automatic.exposure = newEv.exposureTime / lineDuration; - activeState.agc.automatic.gain = newEv.analogueGain; - activeState.agc.automatic.quantizationGain = newEv.quantizationGain; - activeState.agc.automatic.yTarget = newEv.yTarget; - /* - * Expand the target frame duration so that we do not run faster than - * the minimum frame duration when we have short exposures. - */ - processFrameDuration(context, frameContext, - std::max(frameContext.agc.minFrameDuration, newEv.exposureTime)); - - fillMetadata(context, frameContext, metadata); + metadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode); } REGISTER_IPA_ALGORITHM(Agc, "Agc") diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 0527ca0d5f..3a4d7bc546 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -14,7 +14,7 @@ #include <libcamera/geometry.h> -#include "libipa/agc_mean_luminance.h" +#include "libipa/agc.h" #include "algorithm.h" @@ -47,14 +47,8 @@ private: uint8_t computeHistogramPredivider(const Size &size, enum rkisp1_cif_isp_histogram_mode mode); - void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, - ControlList &metadata); - void processFrameDuration(IPAContext &context, - IPAFrameContext &frameContext, - utils::Duration frameDuration); - std::map<int32_t, std::vector<uint8_t>> meteringModes_; - AgcMeanLuminance agc_; + AgcAlgorithm agc_; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/lux.cpp b/src/ipa/rkisp1/algorithms/lux.cpp index 86e46c492f..ce6928a55d 100644 --- a/src/ipa/rkisp1/algorithms/lux.cpp +++ b/src/ipa/rkisp1/algorithms/lux.cpp @@ -74,7 +74,7 @@ void Lux::process(IPAContext &context, if (!stats) return; - utils::Duration exposureTime = context.configuration.sensor.lineDuration * + utils::Duration exposureTime = context.configuration.agc.lineDuration * frameContext.sensor.exposure; double gain = frameContext.sensor.gain; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 1f94afda6b..47691674ad 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -86,21 +86,6 @@ namespace libcamera::ipa::rkisp1 { * \var IPASessionConfiguration::sensor * \brief Sensor-specific configuration of the IPA * - * \var IPASessionConfiguration::sensor.minExposureTime - * \brief Minimum exposure time supported with the sensor - * - * \var IPASessionConfiguration::sensor.maxExposureTime - * \brief Maximum exposure time supported with the sensor - * - * \var IPASessionConfiguration::sensor.minAnalogueGain - * \brief Minimum analogue gain supported with the sensor - * - * \var IPASessionConfiguration::sensor.maxAnalogueGain - * \brief Maximum analogue gain supported with the sensor - * - * \var IPASessionConfiguration::sensor.lineDuration - * \brief Line duration in microseconds - * * \var IPASessionConfiguration::sensor.size * \brief Sensor output resolution */ @@ -147,49 +132,8 @@ namespace libcamera::ipa::rkisp1 { * \var IPAActiveState::agc * \brief State for the Automatic Gain Control algorithm * - * The \a automatic variables track the latest values computed by algorithm - * based on the latest processed statistics. All other variables track the - * consolidated controls requested in queued requests. - * - * \struct IPAActiveState::agc.manual - * \brief Manual exposure time and analog gain (set through requests) - * - * \var IPAActiveState::agc.manual.exposure - * \brief Manual exposure time expressed as a number of lines as set by the - * ExposureTime control - * - * \var IPAActiveState::agc.manual.gain - * \brief Manual analogue gain as set by the AnalogueGain control - * - * \struct IPAActiveState::agc.automatic - * \brief Automatic exposure time and analog gain (computed by the algorithm) - * - * \var IPAActiveState::agc.automatic.exposure - * \brief Automatic exposure time expressed as a number of lines - * - * \var IPAActiveState::agc.automatic.gain - * \brief Automatic analogue gain multiplier - * - * \var IPAActiveState::agc.autoExposureEnabled - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control - * - * \var IPAActiveState::agc.autoGainEnabled - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control - * - * \var IPAActiveState::agc.constraintMode - * \brief Constraint mode as set by the AeConstraintMode control - * - * \var IPAActiveState::agc.exposureMode - * \brief Exposure mode as set by the AeExposureMode control - * * \var IPAActiveState::agc.meteringMode * \brief Metering mode as set by the AeMeteringMode control - * - * \var IPAActiveState::agc.minFrameDuration - * \brief Minimum frame duration as set by the FrameDurationLimits control - * - * \var IPAActiveState::agc.maxFrameDuration - * \brief Maximum frame duration as set by the FrameDurationLimits control */ /** @@ -314,53 +258,11 @@ namespace libcamera::ipa::rkisp1 { * the vertical blanking period is determined to maintain a consistent frame * rate matched to the FrameDurationLimits as set by the user. * - * \var IPAFrameContext::agc.exposure - * \brief Exposure time expressed as a number of lines computed by the algorithm - * - * \var IPAFrameContext::agc.gain - * \brief Analogue gain multiplier computed by the algorithm - * - * The gain should be adapted to the sensor specific gain code before applying. - * - * \var IPAFrameContext::agc.vblank - * \brief Vertical blanking parameter computed by the algorithm - * - * \var IPAFrameContext::agc.autoExposureEnabled - * \brief Manual/automatic AGC state (exposure) as set by the ExposureTimeMode control - * - * \var IPAFrameContext::agc.autoGainEnabled - * \brief Manual/automatic AGC state (gain) as set by the AnalogueGainMode control - * - * \var IPAFrameContext::agc.constraintMode - * \brief Constraint mode as set by the AeConstraintMode control - * - * \var IPAFrameContext::agc.exposureMode - * \brief Exposure mode as set by the AeExposureMode control - * * \var IPAFrameContext::agc.meteringMode * \brief Metering mode as set by the AeMeteringMode control * - * \var IPAFrameContext::agc.minFrameDuration - * \brief Minimum frame duration as set by the FrameDurationLimits control - * - * \var IPAFrameContext::agc.maxFrameDuration - * \brief Maximum frame duration as set by the FrameDurationLimits control - * - * \var IPAFrameContext::agc.frameDuration - * \brief The actual FrameDuration used by the algorithm for the frame - * * \var IPAFrameContext::agc.updateMetering * \brief Indicate if new ISP AGC metering parameters need to be applied - * - * \var IPAFrameContext::agc.autoExposureModeChange - * \brief Indicate if autoExposureEnabled has changed from true in the previous - * frame to false in the current frame, and no manual exposure value has been - * supplied in the current frame. - * - * \var IPAFrameContext::agc.autoGainModeChange - * \brief Indicate if autoGainEnabled has changed from true in the previous - * frame to false in the current frame, and no manual gain value has been - * supplied in the current frame. */ /** diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index cd213dd991..cc07bb9462 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -24,7 +24,7 @@ #include "libcamera/internal/matrix.h" #include "libcamera/internal/vector.h" -#include "libipa/agc_mean_luminance.h" +#include "libipa/agc.h" #include "libipa/awb.h" #include "libipa/camera_sensor_helper.h" #include "libipa/ccm.h" @@ -57,7 +57,7 @@ struct RKISP1AwbSession { }; struct IPASessionConfiguration { - struct { + struct Agc : agc::Session { struct rkisp1_cif_isp_window measureWindow; } agc; @@ -68,12 +68,6 @@ struct IPASessionConfiguration { } compress; struct { - utils::Duration minExposureTime; - utils::Duration maxExposureTime; - double minAnalogueGain; - double maxAnalogueGain; - - utils::Duration lineDuration; Size size; } sensor; @@ -82,26 +76,8 @@ struct IPASessionConfiguration { }; struct IPAActiveState { - struct { - struct { - uint32_t exposure; - double gain; - } manual; - struct { - uint32_t exposure; - double gain; - double quantizationGain; - double yTarget; - } automatic; - - bool autoExposureEnabled; - bool autoGainEnabled; - double exposureValue; - controls::AeConstraintModeEnum constraintMode; - controls::AeExposureModeEnum exposureMode; + struct Agc : agc::ActiveState { controls::AeMeteringModeEnum meteringMode; - utils::Duration minFrameDuration; - utils::Duration maxFrameDuration; } agc; ipa::awb::ActiveState awb; @@ -145,24 +121,9 @@ struct IPAActiveState { }; struct IPAFrameContext : public FrameContext { - struct { - uint32_t exposure; - double gain; - double exposureValue; - double quantizationGain; - uint32_t vblank; - double yTarget; - bool autoExposureEnabled; - bool autoGainEnabled; - controls::AeConstraintModeEnum constraintMode; - controls::AeExposureModeEnum exposureMode; + struct Agc : agc::FrameContext { controls::AeMeteringModeEnum meteringMode; - utils::Duration minFrameDuration; - utils::Duration maxFrameDuration; - utils::Duration frameDuration; bool updateMetering; - bool autoExposureModeChange; - bool autoGainModeChange; } agc; ipa::awb::FrameContext awb;
Add a class that implements the `Algorithm` interface using `AgcMeanLuminance` based on the rkisp1 `Agc` algorithm, with the following main adjustments: * the parameters for `process()` have been made optional to handle the cases where statistics are not available; * the "raw" capture check has been replaced with the "autoAllowed" session parameter; * the controls are only provided after `configure()`. Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com> --- src/ipa/libipa/agc.cpp | 729 +++++++++++++++++++++++++++++- src/ipa/libipa/agc.h | 111 +++++ src/ipa/rkisp1/algorithms/agc.cpp | 461 +++---------------- src/ipa/rkisp1/algorithms/agc.h | 10 +- src/ipa/rkisp1/algorithms/lux.cpp | 2 +- src/ipa/rkisp1/ipa_context.cpp | 98 ---- src/ipa/rkisp1/ipa_context.h | 47 +- 7 files changed, 896 insertions(+), 562 deletions(-)