{"id":19895,"url":"https://patchwork.libcamera.org/api/patches/19895/?format=json","web_url":"https://patchwork.libcamera.org/patch/19895/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240417131536.484129-4-dan.scally@ideasonboard.com>","date":"2024-04-17T13:15:31","name":"[v2,3/8] ipa: libipa: Add ExposureModeHelper","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"ae07766d70a25f1593731f25d2b966f0eba80779","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/people/156/?format=json","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19895/mbox/","series":[{"id":4260,"url":"https://patchwork.libcamera.org/api/series/4260/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4260","date":"2024-04-17T13:15:28","name":"Centralise Agc into libipa","version":2,"mbox":"https://patchwork.libcamera.org/series/4260/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19895/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19895/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7478EC3213\n\tfor ;\n\tWed, 17 Apr 2024 13:16:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 80C166336C;\n\tWed, 17 Apr 2024 15:16:01 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2967961B54\n\tfor ;\n\tWed, 17 Apr 2024 15:15:56 +0200 (CEST)","from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2097A63F;\n\tWed, 17 Apr 2024 15:15:09 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"bU0TrXU0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1713359709;\n\tbh=aXniyL4nNy5kE3pchIQ3LFPkBE/O75lGYCEQDj60lqA=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=bU0TrXU0QX6BHVkRCwFuG2RrCqIGWN4c2ugxlwb0aKtat70G3OsKTxWBA11TiqZ7b\n\tIMmx4Di4rWIPJc96sQlT618hz1kkEnbH8+ZN4RY3wlPelW4HNwsLoXJvjuC2UKSsI/\n\tM9UHC6Kw5knPM4VxIozecQ+gboQPwJc7jwpLQw88=","From":"Daniel Scally ","To":"libcamera-devel@lists.libcamera.org","Cc":"Paul Elder ,\n\tDaniel Scally ","Subject":"[PATCH v2 3/8] ipa: libipa: Add ExposureModeHelper","Date":"Wed, 17 Apr 2024 14:15:31 +0100","Message-Id":"<20240417131536.484129-4-dan.scally@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20240417131536.484129-1-dan.scally@ideasonboard.com>","References":"<20240417131536.484129-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Paul Elder \n\nAdd a helper for managing exposure modes and splitting exposure times\ninto shutter and gain values.\n\nSigned-off-by: Paul Elder \nSigned-off-by: Daniel Scally \n---\nChanges in v2:\n\n\t- Expanded the documentation\n\t- Dropped the overloads for fixed shutter / gain - the same\n\t functionality is instead done by setting min and max shutter and gain\n\t to the same value\n\t- Changed ::init() to consume a vector of pairs instead of two separate\n\t vectors\n\t- Reworked splitExposure()\n\n src/ipa/libipa/exposure_mode_helper.cpp | 257 ++++++++++++++++++++++++\n src/ipa/libipa/exposure_mode_helper.h | 53 +++++\n src/ipa/libipa/meson.build | 2 +\n 3 files changed, 312 insertions(+)\n create mode 100644 src/ipa/libipa/exposure_mode_helper.cpp\n create mode 100644 src/ipa/libipa/exposure_mode_helper.h","diff":"diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp\nnew file mode 100644\nindex 00000000..6463de18\n--- /dev/null\n+++ b/src/ipa/libipa/exposure_mode_helper.cpp\n@@ -0,0 +1,257 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Paul Elder \n+ *\n+ * exposure_mode_helper.cpp - Helper class that performs computations relating to exposure\n+ */\n+#include \"exposure_mode_helper.h\"\n+\n+#include \n+\n+#include \n+\n+/**\n+ * \\file exposure_mode_helper.h\n+ * \\brief Helper class that performs computations relating to exposure\n+ *\n+ * AEGC algorithms have a need to split exposure between shutter and gain.\n+ * Multiple implementations do so based on paired sets of shutter and gain\n+ * limits; provide a helper to avoid duplicating the code.\n+ */\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(ExposureModeHelper)\n+\n+namespace ipa {\n+\n+/**\n+ * \\class ExposureModeHelper\n+ * \\brief Class for splitting exposure into shutter and gain\n+ *\n+ * The ExposureModeHelper class provides a standard interface through which an\n+ * AEGC algorithm can divide exposure between shutter and gain. It is configured\n+ * with a set of shutter and gain pairs and works by initially fixing gain at\n+ * 1.0 and increasing shutter up to the shutter value from the first pair in the\n+ * set in an attempt to meet the required exposure value.\n+ *\n+ * If the required exposure is not achievable by the first shutter value alone\n+ * it ramps gain up to the value from the first pair in the set. If the required\n+ * exposure is still not met it then allows shutter to ramp up to the shutter\n+ * value from the second pair in the set, and continues in this vein until\n+ * either the required exposure value is met, or else the hardware's shutter or\n+ * gain limits are reached.\n+ *\n+ * This method allows users to strike a balance between a well-exposed image and\n+ * an acceptable frame-rate, as opposed to simply maximising shutter followed by\n+ * gain. The same helpers can be used to perform the latter operation if needed\n+ * by passing an empty set of pairs to the initialisation function.\n+ *\n+ * The gain values may exceed a camera sensor's analogue gain limits if either\n+ * it or the IPA is also capable of digital gain. The configure() function must\n+ * be called with the hardware's limits to inform the helper of those\n+ * constraints. Any gain that is needed will be applied as analogue gain first\n+ * until the hardware's limit is reached, following which digital gain will be\n+ * used.\n+ */\n+\n+/**\n+ * \\brief Construct an ExposureModeHelper instance\n+ */\n+ExposureModeHelper::ExposureModeHelper()\n+\t: minShutter_(0), maxShutter_(0), minGain_(0), maxGain_(0)\n+{\n+}\n+\n+/**\n+ * \\brief Initialize an ExposureModeHelper instance\n+ * \\param[in] stages The vector of paired shutter time and gain limits\n+ *\n+ * This function is expected to be called a single time once the algorithm that\n+ * is using these helpers has built the necessary list of shutter and gain pairs\n+ * to use (archetypically by parsing them from a tuning file during the\n+ * algorithm's .init() call).\n+ *\n+ * The input steps are shutter and _total_ gain pairs; the gain encompasses both\n+ * analogue and digital gain.\n+ *\n+ * The vector of stages may be empty. In that case, the helper will simply use\n+ * the runtime limits set through setShutterGainLimits() instead.\n+ */\n+void ExposureModeHelper::init(const std::vector> stages)\n+{\n+\t/* We only need to check shutters_, as gains_ is filled alongside it */\n+\tif (!shutters_.empty()) {\n+\t\tLOG(ExposureModeHelper, Warning)\n+\t\t\t<< \"Initialization attempted multiple times\";\n+\t\treturn;\n+\t}\n+\n+\tfor (auto stage : stages) {\n+\t\tshutters_.push_back(stage.first);\n+\t\tgains_.push_back(stage.second);\n+\t}\n+}\n+\n+/**\n+ * \\brief Set the shutter and gain limits\n+ * \\param[in] minShutter The minimum shutter time supported\n+ * \\param[in] maxShutter The maximum shutter time supported\n+ * \\param[in] minGain The minimum analogue gain supported\n+ * \\param[in] maxGain The maximum analogue gain supported\n+ *\n+ * This function configures the shutter and analogue gain limits that need to be\n+ * adhered to as the helper divides up exposure. Note that these function *must*\n+ * be called whenever those limits change and before splitExposure() is used.\n+ *\n+ * If the algorithm using the helpers needs to indicate that either shutter,\n+ * analogue gain or both should be fixed it can do so by setting both the minima\n+ * and maxima to the same value.\n+ */\n+void ExposureModeHelper::setShutterGainLimits(utils::Duration minShutter,\n+\t\t\t\t\t utils::Duration maxShutter,\n+\t\t\t\t\t double minGain,\n+\t\t\t\t\t double maxGain)\n+{\n+\tminShutter_ = minShutter;\n+\tmaxShutter_ = maxShutter;\n+\tminGain_ = minGain;\n+\tmaxGain_ = maxGain;\n+}\n+\n+utils::Duration ExposureModeHelper::clampShutter(utils::Duration shutter) const\n+{\n+\treturn std::clamp(shutter, minShutter_, maxShutter_);\n+}\n+\n+double ExposureModeHelper::clampGain(double gain) const\n+{\n+\treturn std::clamp(gain, minGain_, maxGain_);\n+}\n+\n+/**\n+ * \\brief Split exposure time into shutter and gain\n+ * \\param[in] exposure Exposure time\n+ *\n+ * This function divides a given exposure value into shutter time, analogue and\n+ * digital gain by iterating through sets of shutter and gain limits. At each\n+ * stage the current stage's shutter limit is multiplied by the previous stage's\n+ * gain limit (or 1.0 initially) to see if the combination of the two can meet\n+ * the required exposure value. If they cannot then splitExpothe current stage's shutter\n+ * limit is multiplied by the same stage's gain limit to see if that combination\n+ * can meet the required exposure value. If they cannot then the function moves\n+ * to consider the next stage.\n+ *\n+ * When a combination of shutter and gain _stage_ limits are found that are\n+ * sufficient to meet the required exposure value, the function attempts to\n+ * reduce shutter time as much as possible whilst fixing gain and still meeting\n+ * the exposure value. If a _runtime_ limit prevents shutter time from being\n+ * lowered enough to meet the exposure value with gain fixed at the stage limit,\n+ * gain is also lowered to compensate.\n+ *\n+ * Once the shutter time and gain values are ascertained, gain is assigned as\n+ * analogue gain as much as possible, with digital gain only in use if the\n+ * maximum analogue gain runtime limit is unable to accomodate the exposure\n+ * value.\n+ *\n+ * If no combination of shutter and gain limits is found that meets the required\n+ * exposure value, the helper falls-back to simply maximising the shutter time\n+ * first, followed by analogue gain, followed by digital gain.\n+ *\n+ * @return Tuple of shutter time, analogue gain, and digital gain\n+ */\n+std::tuple\n+ExposureModeHelper::splitExposure(utils::Duration exposure) const\n+{\n+\tASSERT(maxShutter_);\n+\tASSERT(maxGain_);\n+\n+\tbool gainFixed = minGain_ == maxGain_;\n+\tbool shutterFixed = minShutter_ == maxShutter_;\n+\n+\t/*\n+\t * There's no point entering the loop if we cannot change either gain\n+\t * nor shutter anyway.\n+\t */\n+\tif (shutterFixed && gainFixed)\n+\t\treturn { minShutter_, minGain_, exposure / (minShutter_ * minGain_) };\n+\n+\tutils::Duration shutter;\n+\tdouble stageGain;\n+\tdouble gain;\n+\n+\tfor (unsigned int stage = 0; stage < gains_.size(); stage++) {\n+\t\tdouble lastStageGain = stage == 0 ? 1.0 : clampGain(gains_[stage - 1]);\n+\t\tutils::Duration stageShutter = clampShutter(shutters_[stage]);\n+\t\tstageGain = clampGain(gains_[stage]);\n+\n+\t\t/*\n+\t\t * We perform the clamping on both shutter and gain in case the\n+\t\t * helper has had limits set that prevent those values being\n+\t\t * lowered beyond a certain minimum...this can happen at runtime\n+\t\t * for various reasons and so would not be known when the stage\n+\t\t * limits are initialised.\n+\t\t */\n+\n+\t\tif (stageShutter * lastStageGain >= exposure) {\n+\t\t\tshutter = clampShutter(exposure / clampGain(lastStageGain));\n+\t\t\tgain = clampGain(exposure / shutter);\n+\n+\t\t\treturn { shutter, gain, exposure / (shutter * gain) };\n+\t\t}\n+\n+\t\tif (stageShutter * stageGain >= exposure) {\n+\t\t\tshutter = clampShutter(exposure / clampGain(stageGain));\n+\t\t\tgain = clampGain(exposure / shutter);\n+\n+\t\t\treturn { shutter, gain, exposure / (shutter * gain) };\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * From here on all we can do is max out the shutter, followed by the\n+\t * analogue gain. If we still haven't achieved the target we send the\n+\t * rest of the exposure time to digital gain. If we were given no stages\n+\t * to use then set stageGain to 1.0 so that shutter is maxed before gain\n+\t * touched at all.\n+\t */\n+\tif (gains_.empty())\n+\t\tstageGain = 1.0;\n+\n+\tshutter = clampShutter(exposure / clampGain(stageGain));\n+\tgain = clampGain(exposure / shutter);\n+\n+\treturn { shutter, gain, exposure / (shutter * gain) };\n+}\n+\n+/**\n+ * \\fn ExposureModeHelper::minShutter()\n+ * \\brief Retrieve the configured minimum shutter time limit set through\n+ * setShutterGainLimits()\n+ * \\return The minShutter_ value\n+ */\n+\n+/**\n+ * \\fn ExposureModeHelper::maxShutter()\n+ * \\brief Retrieve the configured maximum shutter time set through\n+ * setShutterGainLimits()\n+ * \\return The maxShutter_ value\n+ */\n+\n+/**\n+ * \\fn ExposureModeHelper::minGain()\n+ * \\brief Retrieve the configured minimum gain set through\n+ * setShutterGainLimits()\n+ * \\return The minGain_ value\n+ */\n+\n+/**\n+ * \\fn ExposureModeHelper::maxGain()\n+ * \\brief Retrieve the configured maximum gain set through\n+ * setShutterGainLimits()\n+ * \\return The maxGain_ value\n+ */\n+\n+} /* namespace ipa */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h\nnew file mode 100644\nindex 00000000..1f672135\n--- /dev/null\n+++ b/src/ipa/libipa/exposure_mode_helper.h\n@@ -0,0 +1,53 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Paul Elder \n+ *\n+ * exposure_mode_helper.h - Helper class that performs computations relating to exposure\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+\n+#include \n+\n+namespace libcamera {\n+\n+namespace ipa {\n+\n+class ExposureModeHelper\n+{\n+public:\n+\tExposureModeHelper();\n+\t~ExposureModeHelper() = default;\n+\n+\tvoid init(const std::vector> stages);\n+\tvoid setShutterGainLimits(utils::Duration minShutter,\n+\t\t\t\t utils::Duration maxShutter,\n+\t\t\t\t double minGain, double maxGain);\n+\n+\tstd::tuple\n+\tsplitExposure(utils::Duration exposure) const;\n+\n+\tutils::Duration minShutter() const { return minShutter_; }\n+\tutils::Duration maxShutter() const { return maxShutter_; }\n+\tdouble minGain() const { return minGain_; }\n+\tdouble maxGain() const { return maxGain_; }\n+\n+private:\n+\tutils::Duration clampShutter(utils::Duration shutter) const;\n+\tdouble clampGain(double gain) const;\n+\n+\tstd::vector shutters_;\n+\tstd::vector gains_;\n+\n+\tutils::Duration minShutter_;\n+\tutils::Duration maxShutter_;\n+\tdouble minGain_;\n+\tdouble maxGain_;\n+};\n+\n+} /* namespace ipa */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build\nindex 016b8e0e..37fbd177 100644\n--- a/src/ipa/libipa/meson.build\n+++ b/src/ipa/libipa/meson.build\n@@ -3,6 +3,7 @@\n libipa_headers = files([\n 'algorithm.h',\n 'camera_sensor_helper.h',\n+ 'exposure_mode_helper.h',\n 'fc_queue.h',\n 'histogram.h',\n 'module.h',\n@@ -11,6 +12,7 @@ libipa_headers = files([\n libipa_sources = files([\n 'algorithm.cpp',\n 'camera_sensor_helper.cpp',\n+ 'exposure_mode_helper.cpp',\n 'fc_queue.cpp',\n 'histogram.cpp',\n 'module.cpp',\n","prefixes":["v2","3/8"]}