From patchwork Wed Apr 17 13:15:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 19895 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7478EC3213 for ; Wed, 17 Apr 2024 13:16:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 80C166336C; Wed, 17 Apr 2024 15:16:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bU0TrXU0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2967961B54 for ; Wed, 17 Apr 2024 15:15:56 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2097A63F; Wed, 17 Apr 2024 15:15:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1713359709; bh=aXniyL4nNy5kE3pchIQ3LFPkBE/O75lGYCEQDj60lqA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bU0TrXU0QX6BHVkRCwFuG2RrCqIGWN4c2ugxlwb0aKtat70G3OsKTxWBA11TiqZ7b IMmx4Di4rWIPJc96sQlT618hz1kkEnbH8+ZN4RY3wlPelW4HNwsLoXJvjuC2UKSsI/ M9UHC6Kw5knPM4VxIozecQ+gboQPwJc7jwpLQw88= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , Daniel Scally Subject: [PATCH v2 3/8] ipa: libipa: Add ExposureModeHelper Date: Wed, 17 Apr 2024 14:15:31 +0100 Message-Id: <20240417131536.484129-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240417131536.484129-1-dan.scally@ideasonboard.com> References: <20240417131536.484129-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Paul Elder Add a helper for managing exposure modes and splitting exposure times into shutter and gain values. Signed-off-by: Paul Elder Signed-off-by: Daniel Scally Reviewed-by: Paul Elder --- Changes in v2: - Expanded the documentation - Dropped the overloads for fixed shutter / gain - the same functionality is instead done by setting min and max shutter and gain to the same value - Changed ::init() to consume a vector of pairs instead of two separate vectors - Reworked splitExposure() src/ipa/libipa/exposure_mode_helper.cpp | 257 ++++++++++++++++++++++++ src/ipa/libipa/exposure_mode_helper.h | 53 +++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 312 insertions(+) create mode 100644 src/ipa/libipa/exposure_mode_helper.cpp create mode 100644 src/ipa/libipa/exposure_mode_helper.h diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp new file mode 100644 index 00000000..6463de18 --- /dev/null +++ b/src/ipa/libipa/exposure_mode_helper.cpp @@ -0,0 +1,257 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Paul Elder + * + * exposure_mode_helper.cpp - Helper class that performs computations relating to exposure + */ +#include "exposure_mode_helper.h" + +#include + +#include + +/** + * \file exposure_mode_helper.h + * \brief Helper class that performs computations relating to exposure + * + * AEGC algorithms have a need to split exposure between shutter and gain. + * Multiple implementations do so based on paired sets of shutter and gain + * limits; provide a helper to avoid duplicating the code. + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(ExposureModeHelper) + +namespace ipa { + +/** + * \class ExposureModeHelper + * \brief Class for splitting exposure into shutter and gain + * + * The ExposureModeHelper class provides a standard interface through which an + * AEGC algorithm can divide exposure between shutter and gain. It is configured + * with a set of shutter and gain pairs and works by initially fixing gain at + * 1.0 and increasing shutter up to the shutter value from the first pair in the + * set in an attempt to meet the required exposure value. + * + * If the required exposure is not achievable by the first shutter value alone + * it ramps gain up to the value from the first pair in the set. If the required + * exposure is still not met it then allows shutter to ramp up to the shutter + * value from the second pair in the set, and continues in this vein until + * either the required exposure value is met, or else the hardware's shutter or + * gain limits are reached. + * + * This method allows users to strike a balance between a well-exposed image and + * an acceptable frame-rate, as opposed to simply maximising shutter followed by + * gain. The same helpers can be used to perform the latter operation if needed + * by passing an empty set of pairs to the initialisation function. + * + * The gain values may exceed a camera sensor's analogue gain limits if either + * it or the IPA is also capable of digital gain. The configure() function must + * be called with the hardware's limits to inform the helper of those + * constraints. Any gain that is needed will be applied as analogue gain first + * until the hardware's limit is reached, following which digital gain will be + * used. + */ + +/** + * \brief Construct an ExposureModeHelper instance + */ +ExposureModeHelper::ExposureModeHelper() + : minShutter_(0), maxShutter_(0), minGain_(0), maxGain_(0) +{ +} + +/** + * \brief Initialize an ExposureModeHelper instance + * \param[in] stages The vector of paired shutter time and gain limits + * + * This function is expected to be called a single time once the algorithm that + * is using these helpers has built the necessary list of shutter and gain pairs + * to use (archetypically by parsing them from a tuning file during the + * algorithm's .init() call). + * + * The input steps are shutter and _total_ gain pairs; the gain encompasses both + * analogue and digital gain. + * + * The vector of stages may be empty. In that case, the helper will simply use + * the runtime limits set through setShutterGainLimits() instead. + */ +void ExposureModeHelper::init(const std::vector> stages) +{ + /* We only need to check shutters_, as gains_ is filled alongside it */ + if (!shutters_.empty()) { + LOG(ExposureModeHelper, Warning) + << "Initialization attempted multiple times"; + return; + } + + for (auto stage : stages) { + shutters_.push_back(stage.first); + gains_.push_back(stage.second); + } +} + +/** + * \brief Set the shutter and gain limits + * \param[in] minShutter The minimum shutter time supported + * \param[in] maxShutter The maximum shutter time supported + * \param[in] minGain The minimum analogue gain supported + * \param[in] maxGain The maximum analogue gain supported + * + * This function configures the shutter and analogue gain limits that need to be + * adhered to as the helper divides up exposure. Note that these function *must* + * be called whenever those limits change and before splitExposure() is used. + * + * If the algorithm using the helpers needs to indicate that either shutter, + * analogue gain or both should be fixed it can do so by setting both the minima + * and maxima to the same value. + */ +void ExposureModeHelper::setShutterGainLimits(utils::Duration minShutter, + utils::Duration maxShutter, + double minGain, + double maxGain) +{ + minShutter_ = minShutter; + maxShutter_ = maxShutter; + minGain_ = minGain; + maxGain_ = maxGain; +} + +utils::Duration ExposureModeHelper::clampShutter(utils::Duration shutter) const +{ + return std::clamp(shutter, minShutter_, maxShutter_); +} + +double ExposureModeHelper::clampGain(double gain) const +{ + return std::clamp(gain, minGain_, maxGain_); +} + +/** + * \brief Split exposure time into shutter and gain + * \param[in] exposure Exposure time + * + * This function divides a given exposure value into shutter time, analogue and + * digital gain by iterating through sets of shutter and gain limits. At each + * stage the current stage's shutter limit is multiplied by the previous stage's + * gain limit (or 1.0 initially) to see if the combination of the two can meet + * the required exposure value. If they cannot then splitExpothe current stage's shutter + * limit is multiplied by the same stage's gain limit to see if that combination + * can meet the required exposure value. If they cannot then the function moves + * to consider the next stage. + * + * When a combination of shutter and gain _stage_ limits are found that are + * sufficient to meet the required exposure value, the function attempts to + * reduce shutter time as much as possible whilst fixing gain and still meeting + * the exposure value. If a _runtime_ limit prevents shutter time from being + * lowered enough to meet the exposure value with gain fixed at the stage limit, + * gain is also lowered to compensate. + * + * Once the shutter time and gain values are ascertained, gain is assigned as + * analogue gain as much as possible, with digital gain only in use if the + * maximum analogue gain runtime limit is unable to accomodate the exposure + * value. + * + * If no combination of shutter and gain limits is found that meets the required + * exposure value, the helper falls-back to simply maximising the shutter time + * first, followed by analogue gain, followed by digital gain. + * + * @return Tuple of shutter time, analogue gain, and digital gain + */ +std::tuple +ExposureModeHelper::splitExposure(utils::Duration exposure) const +{ + ASSERT(maxShutter_); + ASSERT(maxGain_); + + bool gainFixed = minGain_ == maxGain_; + bool shutterFixed = minShutter_ == maxShutter_; + + /* + * There's no point entering the loop if we cannot change either gain + * nor shutter anyway. + */ + if (shutterFixed && gainFixed) + return { minShutter_, minGain_, exposure / (minShutter_ * minGain_) }; + + utils::Duration shutter; + double stageGain; + double gain; + + for (unsigned int stage = 0; stage < gains_.size(); stage++) { + double lastStageGain = stage == 0 ? 1.0 : clampGain(gains_[stage - 1]); + utils::Duration stageShutter = clampShutter(shutters_[stage]); + stageGain = clampGain(gains_[stage]); + + /* + * We perform the clamping on both shutter and gain in case the + * helper has had limits set that prevent those values being + * lowered beyond a certain minimum...this can happen at runtime + * for various reasons and so would not be known when the stage + * limits are initialised. + */ + + if (stageShutter * lastStageGain >= exposure) { + shutter = clampShutter(exposure / clampGain(lastStageGain)); + gain = clampGain(exposure / shutter); + + return { shutter, gain, exposure / (shutter * gain) }; + } + + if (stageShutter * stageGain >= exposure) { + shutter = clampShutter(exposure / clampGain(stageGain)); + gain = clampGain(exposure / shutter); + + return { shutter, gain, exposure / (shutter * gain) }; + } + } + + /* + * From here on all we can do is max out the shutter, followed by the + * analogue gain. If we still haven't achieved the target we send the + * rest of the exposure time to digital gain. If we were given no stages + * to use then set stageGain to 1.0 so that shutter is maxed before gain + * touched at all. + */ + if (gains_.empty()) + stageGain = 1.0; + + shutter = clampShutter(exposure / clampGain(stageGain)); + gain = clampGain(exposure / shutter); + + return { shutter, gain, exposure / (shutter * gain) }; +} + +/** + * \fn ExposureModeHelper::minShutter() + * \brief Retrieve the configured minimum shutter time limit set through + * setShutterGainLimits() + * \return The minShutter_ value + */ + +/** + * \fn ExposureModeHelper::maxShutter() + * \brief Retrieve the configured maximum shutter time set through + * setShutterGainLimits() + * \return The maxShutter_ value + */ + +/** + * \fn ExposureModeHelper::minGain() + * \brief Retrieve the configured minimum gain set through + * setShutterGainLimits() + * \return The minGain_ value + */ + +/** + * \fn ExposureModeHelper::maxGain() + * \brief Retrieve the configured maximum gain set through + * setShutterGainLimits() + * \return The maxGain_ value + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h new file mode 100644 index 00000000..1f672135 --- /dev/null +++ b/src/ipa/libipa/exposure_mode_helper.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Paul Elder + * + * exposure_mode_helper.h - Helper class that performs computations relating to exposure + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +namespace ipa { + +class ExposureModeHelper +{ +public: + ExposureModeHelper(); + ~ExposureModeHelper() = default; + + void init(const std::vector> stages); + void setShutterGainLimits(utils::Duration minShutter, + utils::Duration maxShutter, + double minGain, double maxGain); + + std::tuple + splitExposure(utils::Duration exposure) const; + + utils::Duration minShutter() const { return minShutter_; } + utils::Duration maxShutter() const { return maxShutter_; } + double minGain() const { return minGain_; } + double maxGain() const { return maxGain_; } + +private: + utils::Duration clampShutter(utils::Duration shutter) const; + double clampGain(double gain) const; + + std::vector shutters_; + std::vector gains_; + + utils::Duration minShutter_; + utils::Duration maxShutter_; + double minGain_; + double maxGain_; +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 016b8e0e..37fbd177 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -3,6 +3,7 @@ libipa_headers = files([ 'algorithm.h', 'camera_sensor_helper.h', + 'exposure_mode_helper.h', 'fc_queue.h', 'histogram.h', 'module.h', @@ -11,6 +12,7 @@ libipa_headers = files([ libipa_sources = files([ 'algorithm.cpp', 'camera_sensor_helper.cpp', + 'exposure_mode_helper.cpp', 'fc_queue.cpp', 'histogram.cpp', 'module.cpp',