[{"id":29337,"web_url":"https://patchwork.libcamera.org/comment/29337/","msgid":"","date":"2024-04-24T17:35:33","subject":"Re: [PATCH v3 3/8] ipa: libipa: Add ExposureModeHelper","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Dan\n\nOn Wed, Apr 24, 2024 at 01:49:12PM +0100, Daniel Scally wrote:\n> From: Paul Elder \n>\n> Add a helper for managing exposure modes and splitting exposure times\n> into shutter and gain values.\n>\n> Reviewed-by: Paul Elder \n> Signed-off-by: Paul Elder \n> Signed-off-by: Daniel Scally \n> ---\n> Changes in v3:\n>\n> \t- Referred to \"shutter time\" instead of \"shutter\"\n> \t- Removed ::init() and swapped its functionality to the constructor\n> \t- Replaced the parameter to the constructor (ex-::init()) with a Span\n> \t instead of a vector.\n> \t- Lots of documentation updates and function renaming\n> Changes in v2:\n>\n> \t- Expanded the documentation\n> \t- Dropped the overloads for fixed shutter / gain - the same\n> \t functionality is instead done by setting min and max shutter and gain\n> \t to the same value\n> \t- Changed ::init() to consume a vector of pairs instead of two separate\n> \t vectors\n> \t- Reworked splitExposure()\n>\n> src/ipa/libipa/exposure_mode_helper.cpp | 246 ++++++++++++++++++++++++\n> src/ipa/libipa/exposure_mode_helper.h | 53 +++++\n> src/ipa/libipa/meson.build | 2 +\n> 3 files changed, 301 insertions(+)\n> create mode 100644 src/ipa/libipa/exposure_mode_helper.cpp\n> create mode 100644 src/ipa/libipa/exposure_mode_helper.h\n>\n> diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp\n> new file mode 100644\n> index 00000000..47a300f3\n> --- /dev/null\n> +++ b/src/ipa/libipa/exposure_mode_helper.cpp\n> @@ -0,0 +1,246 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Paul Elder \n> + *\n> + * exposure_mode_helper.cpp - Helper class that performs computations relating to exposure\n> + */\n> +#include \"exposure_mode_helper.h\"\n> +\n> +#include \n> +\n> +#include \n> +\n> +/**\n> + * \\file exposure_mode_helper.h\n> + * \\brief Helper class that performs computations relating to exposure\n> + *\n> + * AEGC algorithms have a need to split exposure between shutter time, analogue\n> + * and digital gain. Multiple implementations do so based on paired stages of\n> + * shutter time and gain limits; provide a helper to avoid duplicating the code.\n> + */\n> +\n> +namespace libcamera {\n> +\n> +using namespace std::literals::chrono_literals;\n> +\n> +LOG_DEFINE_CATEGORY(ExposureModeHelper)\n> +\n> +namespace ipa {\n> +\n> +/**\n> + * \\class ExposureModeHelper\n> + * \\brief Class for splitting exposure into shutter time and total gain\n> + *\n> + * The ExposureModeHelper class provides a standard interface through which an\n> + * AEGC algorithm can divide exposure between shutter time and gain. It is\n> + * configured with a set of shutter time and gain pairs and works by initially\n> + * fixing gain at 1.0 and increasing shutter time up to the shutter time value\n> + * from the first pair in the set in an attempt to meet the required exposure\n> + * value.\n> + *\n> + * If the required exposure is not achievable by the first shutter time value\n> + * alone it ramps gain up to the value from the first pair in the set. If the\n> + * required exposure is still not met it then allows shutter time to ramp up to\n> + * the shutter time value from the second pair in the set, and continues in this\n> + * vein until either the required exposure time is met, or else the hardware's\n> + * shutter time or gain limits are reached.\n> + *\n> + * This method allows users to strike a balance between a well-exposed image and\n> + * an acceptable frame-rate, as opposed to simply maximising shutter time\n> + * followed by gain. The same helpers can be used to perform the latter\n> + * operation if needed by passing an empty set of pairs to the initialisation\n> + * function.\n> + *\n> + * The gain values may exceed a camera sensor's analogue gain limits if either\n> + * it or the IPA is also capable of digital gain. The configure() function must\n> + * be called with the hardware's limits to inform the helper of those\n> + * constraints. Any gain that is needed will be applied as analogue gain first\n> + * until the hardware's limit is reached, following which digital gain will be\n> + * used.\n> + */\n> +\n> +/**\n> + * \\brief Construct an ExposureModeHelper instance\n> + * \\param[in] stages The vector of paired shutter time and gain limits\n> + *\n> + * The input stages are shutter time and _total_ gain pairs; the gain\n> + * encompasses both analogue and digital gain.\n> + *\n> + * The vector of stages may be empty. In that case, the helper will simply use\n> + * the runtime limits set through setShutterGainLimits() instead.\n> + */\n> +ExposureModeHelper::ExposureModeHelper(const Span> stages)\n> +{\n> +\tminShutter_ = 0us;\n> +\tmaxShutter_ = 0us;\n> +\tminGain_ = 0;\n> +\tmaxGain_ = 0;\n> +\n> +\tfor (const auto &[s, g] : stages) {\n> +\t\tshutters_.push_back(s);\n> +\t\tgains_.push_back(g);\n> +\t}\n> +}\n> +\n> +/**\n> + * \\brief Set the shutter time and gain limits\n> + * \\param[in] minShutter The minimum shutter time supported\n> + * \\param[in] maxShutter The maximum shutter time supported\n> + * \\param[in] minGain The minimum analogue gain supported\n> + * \\param[in] maxGain The maximum analogue gain supported\n> + *\n> + * This function configures the shutter time and analogue gain limits that need\n> + * to be adhered to as the helper divides up exposure. Note that this function\n> + * *must* be called whenever those limits change and before splitExposure() is\n> + * used.\n> + *\n> + * If the algorithm using the helpers needs to indicate that either shutter time\n> + * or analogue gain or both should be fixed it can do so by setting both the\n> + * minima and maxima to the same value.\n> + */\n> +void ExposureModeHelper::setLimits(utils::Duration minShutter,\n> +\t\t\t\t utils::Duration maxShutter,\n> +\t\t\t\t double minGain, double maxGain)\n> +{\n> +\tminShutter_ = minShutter;\n> +\tmaxShutter_ = maxShutter;\n> +\tminGain_ = minGain;\n> +\tmaxGain_ = maxGain;\n> +}\n> +\n> +utils::Duration ExposureModeHelper::clampShutter(utils::Duration shutter) const\n> +{\n> +\treturn std::clamp(shutter, minShutter_, maxShutter_);\n> +}\n> +\n> +double ExposureModeHelper::clampGain(double gain) const\n> +{\n> +\treturn std::clamp(gain, minGain_, maxGain_);\n> +}\n> +\n> +/**\n> + * \\brief Split exposure time into shutter time and gain\n\nOk, I'll accept that what you pass in as exposure value (in\nagc_mean_value.cpp) is here an exposure time\n\n> + * \\param[in] exposure Exposure time\n> + *\n> + * This function divides a given exposure time into shutter time, analogue and\n> + * digital gain by iterating through stages of shutter time and gain limits. At\n> + * each stage the current stage's shutter time limit is multiplied by the\n> + * previous stage's gain limit (or 1.0 initially) to see if the combination of\n> + * the two can meet the required exposure time. If they cannot then the current\n> + * stage's shutter time limit is multiplied by the same stage's gain limit to\n> + * see if that combination can meet the required exposure time. If they cannot\n> + * then the function moves to consider the next stage.\n> + *\n> + * When a combination of shutter time and gain _stage_ limits are found that are\n> + * sufficient to meet the required exposure time, the function attempts to\n> + * reduce shutter time as much as possible whilst fixing gain and still meeting\n> + * the exposure time. If a _runtime_ limit prevents shutter time from being\n> + * lowered enough to meet the exposure time with gain fixed at the stage limit,\n> + * gain is also lowered to compensate.\n> + *\n> + * Once the shutter time and gain values are ascertained, gain is assigned as\n> + * analogue gain as much as possible, with digital gain only in use if the\n> + * maximum analogue gain runtime limit is unable to accomodate the exposure\n\naccommodate ?\n\nReviewed-by: Jacopo Mondi \n\nThanks\n j\n\n> + * value.\n> + *\n> + * If no combination of shutter time and gain limits is found that meets the\n> + * required exposure time, the helper falls-back to simply maximising the\n> + * shutter time first, followed by analogue gain, followed by digital gain.\n> + *\n> + * \\return Tuple of shutter time, analogue gain, and digital gain\n> + */\n> +std::tuple\n> +ExposureModeHelper::splitExposure(utils::Duration exposure) const\n> +{\n> +\tASSERT(maxShutter_);\n> +\tASSERT(maxGain_);\n> +\n> +\tbool gainFixed = minGain_ == maxGain_;\n> +\tbool shutterFixed = minShutter_ == maxShutter_;\n> +\n> +\t/*\n> +\t * There's no point entering the loop if we cannot change either gain\n> +\t * nor shutter anyway.\n> +\t */\n> +\tif (shutterFixed && gainFixed)\n> +\t\treturn { minShutter_, minGain_, exposure / (minShutter_ * minGain_) };\n> +\n> +\tutils::Duration shutter;\n> +\tdouble stageGain;\n> +\tdouble gain;\n> +\n> +\tfor (unsigned int stage = 0; stage < gains_.size(); stage++) {\n> +\t\tdouble lastStageGain = stage == 0 ? 1.0 : clampGain(gains_[stage - 1]);\n> +\t\tutils::Duration stageShutter = clampShutter(shutters_[stage]);\n> +\t\tstageGain = clampGain(gains_[stage]);\n> +\n> +\t\t/*\n> +\t\t * We perform the clamping on both shutter and gain in case the\n> +\t\t * helper has had limits set that prevent those values being\n> +\t\t * lowered beyond a certain minimum...this can happen at runtime\n> +\t\t * for various reasons and so would not be known when the stage\n> +\t\t * limits are initialised.\n> +\t\t */\n> +\n> +\t\tif (stageShutter * lastStageGain >= exposure) {\n> +\t\t\tshutter = clampShutter(exposure / clampGain(lastStageGain));\n> +\t\t\tgain = clampGain(exposure / shutter);\n> +\n> +\t\t\treturn { shutter, gain, exposure / (shutter * gain) };\n> +\t\t}\n> +\n> +\t\tif (stageShutter * stageGain >= exposure) {\n> +\t\t\tshutter = clampShutter(exposure / clampGain(stageGain));\n> +\t\t\tgain = clampGain(exposure / shutter);\n> +\n> +\t\t\treturn { shutter, gain, exposure / (shutter * gain) };\n> +\t\t}\n> +\t}\n> +\n> +\t/*\n> +\t * From here on all we can do is max out the shutter time, followed by\n> +\t * the analogue gain. If we still haven't achieved the target we send\n> +\t * the rest of the exposure time to digital gain. If we were given no\n> +\t * stages to use then set stageGain to 1.0 so that shutter time is maxed\n> +\t * before gain touched at all.\n> +\t */\n> +\tif (gains_.empty())\n> +\t\tstageGain = 1.0;\n> +\n> +\tshutter = clampShutter(exposure / clampGain(stageGain));\n> +\tgain = clampGain(exposure / shutter);\n> +\n> +\treturn { shutter, gain, exposure / (shutter * gain) };\n> +}\n> +\n> +/**\n> + * \\fn ExposureModeHelper::minShutter()\n> + * \\brief Retrieve the configured minimum shutter time limit set through\n> + * setShutterGainLimits()\n> + * \\return The minShutter_ value\n> + */\n> +\n> +/**\n> + * \\fn ExposureModeHelper::maxShutter()\n> + * \\brief Retrieve the configured maximum shutter time set through\n> + * setShutterGainLimits()\n> + * \\return The maxShutter_ value\n> + */\n> +\n> +/**\n> + * \\fn ExposureModeHelper::minGain()\n> + * \\brief Retrieve the configured minimum gain set through\n> + * setShutterGainLimits()\n> + * \\return The minGain_ value\n> + */\n> +\n> +/**\n> + * \\fn ExposureModeHelper::maxGain()\n> + * \\brief Retrieve the configured maximum gain set through\n> + * setShutterGainLimits()\n> + * \\return The maxGain_ value\n> + */\n> +\n> +} /* namespace ipa */\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h\n> new file mode 100644\n> index 00000000..0ffc164e\n> --- /dev/null\n> +++ b/src/ipa/libipa/exposure_mode_helper.h\n> @@ -0,0 +1,53 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Paul Elder \n> + *\n> + * exposure_mode_helper.h - Helper class that performs computations relating to exposure\n> + */\n> +\n> +#pragma once\n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +\n> +namespace libcamera {\n> +\n> +namespace ipa {\n> +\n> +class ExposureModeHelper\n> +{\n> +public:\n> +\tExposureModeHelper(const Span> stages);\n> +\t~ExposureModeHelper() = default;\n> +\n> +\tvoid setLimits(utils::Duration minShutter, utils::Duration maxShutter,\n> +\t\t double minGain, double maxGain);\n> +\n> +\tstd::tuple\n> +\tsplitExposure(utils::Duration exposure) const;\n> +\n> +\tutils::Duration minShutter() const { return minShutter_; }\n> +\tutils::Duration maxShutter() const { return maxShutter_; }\n> +\tdouble minGain() const { return minGain_; }\n> +\tdouble maxGain() const { return maxGain_; }\n> +\n> +private:\n> +\tutils::Duration clampShutter(utils::Duration shutter) const;\n> +\tdouble clampGain(double gain) const;\n> +\n> +\tstd::vector shutters_;\n> +\tstd::vector gains_;\n> +\n> +\tutils::Duration minShutter_;\n> +\tutils::Duration maxShutter_;\n> +\tdouble minGain_;\n> +\tdouble maxGain_;\n> +};\n> +\n> +} /* namespace ipa */\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build\n> index 016b8e0e..37fbd177 100644\n> --- a/src/ipa/libipa/meson.build\n> +++ b/src/ipa/libipa/meson.build\n> @@ -3,6 +3,7 @@\n> libipa_headers = files([\n> 'algorithm.h',\n> 'camera_sensor_helper.h',\n> + 'exposure_mode_helper.h',\n> 'fc_queue.h',\n> 'histogram.h',\n> 'module.h',\n> @@ -11,6 +12,7 @@ libipa_headers = files([\n> libipa_sources = files([\n> 'algorithm.cpp',\n> 'camera_sensor_helper.cpp',\n> + 'exposure_mode_helper.cpp',\n> 'fc_queue.cpp',\n> 'histogram.cpp',\n> 'module.cpp',\n> --\n> 2.34.1\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 81156BE08B\n\tfor ;\n\tWed, 24 Apr 2024 17:35:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 56ECB633FB;\n\tWed, 24 Apr 2024 19:35:39 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 47650633F2\n\tfor ;\n\tWed, 24 Apr 2024 19:35:37 +0200 (CEST)","from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it\n\t[93.61.96.190])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 114B882A;\n\tWed, 24 Apr 2024 19:34:45 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"KkJCZ4Xt\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1713980085;\n\tbh=uxzgdx8EZEQgmOY+JLec9Ea0N3Quyrwc6CsBF6+TnUg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=KkJCZ4Xt8i2DdoB05gsEwxDOM6dYP/YanfuIzdZX/saT4s15qwSDBg2Jwn02iWtXL\n\t7C8cl+Oh3/Jae7UhNfwt1lwAeO83hBjq9FJA6o/WjDi0atIWAju0ne69kOtbTuOqqx\n\tVQrfC1sTEgrTnDrTEJnRNsFWHRIplQOav770jUDU=","Date":"Wed, 24 Apr 2024 19:35:33 +0200","From":"Jacopo Mondi ","To":"Daniel Scally ","Cc":"libcamera-devel@lists.libcamera.org, \n\tPaul Elder ","Subject":"Re: [PATCH v3 3/8] ipa: libipa: Add ExposureModeHelper","Message-ID":"","References":"<20240424124917.1250837-1-dan.scally@ideasonboard.com>\n\t<20240424124917.1250837-4-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20240424124917.1250837-4-dan.scally@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]