{"id":19985,"url":"https://patchwork.libcamera.org/api/1.1/patches/19985/?format=json","web_url":"https://patchwork.libcamera.org/patch/19985/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240502133046.1976565-4-dan.scally@ideasonboard.com>","date":"2024-05-02T13:30:41","name":"[v4,3/8] ipa: libipa: Add ExposureModeHelper","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"2ceb05aefd53a7fcfed24622965a220262ed3127","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/1.1/people/156/?format=json","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19985/mbox/","series":[{"id":4284,"url":"https://patchwork.libcamera.org/api/1.1/series/4284/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4284","date":"2024-05-02T13:30:38","name":"Centralise Agc into libipa","version":4,"mbox":"https://patchwork.libcamera.org/series/4284/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19985/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19985/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6CA54C32C9\n\tfor ;\n\tThu, 2 May 2024 13:31:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A8AF06341E;\n\tThu, 2 May 2024 15:31:19 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1E88D63419\n\tfor ;\n\tThu, 2 May 2024 15:31:15 +0200 (CEST)","from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 88DA0552;\n\tThu, 2 May 2024 15:30:17 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"fnP0LLAD\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1714656617;\n\tbh=Ppz6DpdJAte+sbU+ERKAoA+wcjH/NPpaN/rlSrVEyoM=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=fnP0LLADeQ9jGLfMG+5RWkYKVqFgRQ8xcvlHzQgg2EdGefi0O8xrLn2qp07vJRz5K\n\tZFQXmtd89gWLB5n+b1AkIaprDFG/q+1fWTQnYu6TVloh/1fBGGOatZk9CNIH/YLTl8\n\tRhJDtlHH/jMxq7sxOJL2FLFPzP2Oblzotf9wbyEI=","From":"Daniel Scally ","To":"libcamera-devel@lists.libcamera.org","Cc":"Paul Elder ,\n\tJacopo Mondi ,\n\tDaniel Scally ","Subject":"[PATCH v4 3/8] ipa: libipa: Add ExposureModeHelper","Date":"Thu, 2 May 2024 14:30:41 +0100","Message-Id":"<20240502133046.1976565-4-dan.scally@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20240502133046.1976565-1-dan.scally@ideasonboard.com>","References":"<20240502133046.1976565-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Paul Elder \n\nAdd a helper for managing exposure modes and splitting exposure times\ninto shutter and gain values.\n\nReviewed-by: Jacopo Mondi \nReviewed-by: Paul Elder \nSigned-off-by: Paul Elder \nSigned-off-by: Daniel Scally \n---\nChanges in v4:\n\n\t- None\n\nChanges in v3:\n\n\t- Referred to \"shutter time\" instead of \"shutter\"\n\t- Removed ::init() and swapped its functionality to the constructor\n\t- Replaced the parameter to the constructor (ex-::init()) with a Span\n\t instead of a vector.\n\t- Lots of documentation updates and function renaming\nChanges in v2:\n\n\t- Expanded the documentation\n\t- Dropped the overloads for fixed shutter / gain - the same\n\t functionality is instead done by setting min and max shutter and gain\n\t to the same value\n\t- Changed ::init() to consume a vector of pairs instead of two separate\n\t vectors\n\t- Reworked splitExposure()\n\n src/ipa/libipa/exposure_mode_helper.cpp | 246 ++++++++++++++++++++++++\n src/ipa/libipa/exposure_mode_helper.h | 53 +++++\n src/ipa/libipa/meson.build | 2 +\n 3 files changed, 301 insertions(+)\n create mode 100644 src/ipa/libipa/exposure_mode_helper.cpp\n create mode 100644 src/ipa/libipa/exposure_mode_helper.h","diff":"diff --git a/src/ipa/libipa/exposure_mode_helper.cpp b/src/ipa/libipa/exposure_mode_helper.cpp\nnew file mode 100644\nindex 00000000..7135fb2e\n--- /dev/null\n+++ b/src/ipa/libipa/exposure_mode_helper.cpp\n@@ -0,0 +1,246 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Paul Elder \n+ *\n+ * exposure_mode_helper.cpp - Helper class that performs computations relating to exposure\n+ */\n+#include \"exposure_mode_helper.h\"\n+\n+#include \n+\n+#include \n+\n+/**\n+ * \\file exposure_mode_helper.h\n+ * \\brief Helper class that performs computations relating to exposure\n+ *\n+ * AEGC algorithms have a need to split exposure between shutter time, analogue\n+ * and digital gain. Multiple implementations do so based on paired stages of\n+ * shutter time and gain limits; provide a helper to avoid duplicating the code.\n+ */\n+\n+namespace libcamera {\n+\n+using namespace std::literals::chrono_literals;\n+\n+LOG_DEFINE_CATEGORY(ExposureModeHelper)\n+\n+namespace ipa {\n+\n+/**\n+ * \\class ExposureModeHelper\n+ * \\brief Class for splitting exposure into shutter time and total gain\n+ *\n+ * The ExposureModeHelper class provides a standard interface through which an\n+ * AEGC algorithm can divide exposure between shutter time and gain. It is\n+ * configured with a set of shutter time and gain pairs and works by initially\n+ * fixing gain at 1.0 and increasing shutter time up to the shutter time value\n+ * from the first pair in the set in an attempt to meet the required exposure\n+ * value.\n+ *\n+ * If the required exposure is not achievable by the first shutter time value\n+ * alone it ramps gain up to the value from the first pair in the set. If the\n+ * required exposure is still not met it then allows shutter time to ramp up to\n+ * the shutter time value from the second pair in the set, and continues in this\n+ * vein until either the required exposure time is met, or else the hardware's\n+ * shutter time or gain limits are reached.\n+ *\n+ * This method allows users to strike a balance between a well-exposed image and\n+ * an acceptable frame-rate, as opposed to simply maximising shutter time\n+ * followed by gain. The same helpers can be used to perform the latter\n+ * operation if needed by passing an empty set of pairs to the initialisation\n+ * function.\n+ *\n+ * The gain values may exceed a camera sensor's analogue gain limits if either\n+ * it or the IPA is also capable of digital gain. The configure() function must\n+ * be called with the hardware's limits to inform the helper of those\n+ * constraints. Any gain that is needed will be applied as analogue gain first\n+ * until the hardware's limit is reached, following which digital gain will be\n+ * used.\n+ */\n+\n+/**\n+ * \\brief Construct an ExposureModeHelper instance\n+ * \\param[in] stages The vector of paired shutter time and gain limits\n+ *\n+ * The input stages are shutter time and _total_ gain pairs; the gain\n+ * encompasses both analogue and digital gain.\n+ *\n+ * The vector of stages may be empty. In that case, the helper will simply use\n+ * the runtime limits set through setShutterGainLimits() instead.\n+ */\n+ExposureModeHelper::ExposureModeHelper(const Span> stages)\n+{\n+\tminShutter_ = 0us;\n+\tmaxShutter_ = 0us;\n+\tminGain_ = 0;\n+\tmaxGain_ = 0;\n+\n+\tfor (const auto &[s, g] : stages) {\n+\t\tshutters_.push_back(s);\n+\t\tgains_.push_back(g);\n+\t}\n+}\n+\n+/**\n+ * \\brief Set the shutter time and gain limits\n+ * \\param[in] minShutter The minimum shutter time supported\n+ * \\param[in] maxShutter The maximum shutter time supported\n+ * \\param[in] minGain The minimum analogue gain supported\n+ * \\param[in] maxGain The maximum analogue gain supported\n+ *\n+ * This function configures the shutter time and analogue gain limits that need\n+ * to be adhered to as the helper divides up exposure. Note that this function\n+ * *must* be called whenever those limits change and before splitExposure() is\n+ * used.\n+ *\n+ * If the algorithm using the helpers needs to indicate that either shutter time\n+ * or analogue gain or both should be fixed it can do so by setting both the\n+ * minima and maxima to the same value.\n+ */\n+void ExposureModeHelper::setLimits(utils::Duration minShutter,\n+\t\t\t\t utils::Duration maxShutter,\n+\t\t\t\t double minGain, double maxGain)\n+{\n+\tminShutter_ = minShutter;\n+\tmaxShutter_ = maxShutter;\n+\tminGain_ = minGain;\n+\tmaxGain_ = maxGain;\n+}\n+\n+utils::Duration ExposureModeHelper::clampShutter(utils::Duration shutter) const\n+{\n+\treturn std::clamp(shutter, minShutter_, maxShutter_);\n+}\n+\n+double ExposureModeHelper::clampGain(double gain) const\n+{\n+\treturn std::clamp(gain, minGain_, maxGain_);\n+}\n+\n+/**\n+ * \\brief Split exposure time into shutter time and gain\n+ * \\param[in] exposure Exposure time\n+ *\n+ * This function divides a given exposure time into shutter time, analogue and\n+ * digital gain by iterating through stages of shutter time and gain limits. At\n+ * each stage the current stage's shutter time limit is multiplied by the\n+ * previous stage's gain limit (or 1.0 initially) to see if the combination of\n+ * the two can meet the required exposure time. If they cannot then the current\n+ * stage's shutter time limit is multiplied by the same stage's gain limit to\n+ * see if that combination can meet the required exposure time. If they cannot\n+ * then the function moves to consider the next stage.\n+ *\n+ * When a combination of shutter time and gain _stage_ limits are found that are\n+ * sufficient to meet the required exposure time, the function attempts to\n+ * reduce shutter time as much as possible whilst fixing gain and still meeting\n+ * the exposure time. If a _runtime_ limit prevents shutter time from being\n+ * lowered enough to meet the exposure time with gain fixed at the stage limit,\n+ * gain is also lowered to compensate.\n+ *\n+ * Once the shutter time and gain values are ascertained, gain is assigned as\n+ * analogue gain as much as possible, with digital gain only in use if the\n+ * maximum analogue gain runtime limit is unable to accommodate the exposure\n+ * value.\n+ *\n+ * If no combination of shutter time and gain limits is found that meets the\n+ * required exposure time, the helper falls-back to simply maximising the\n+ * shutter time first, followed by analogue gain, followed by digital gain.\n+ *\n+ * \\return Tuple of shutter time, analogue gain, and digital gain\n+ */\n+std::tuple\n+ExposureModeHelper::splitExposure(utils::Duration exposure) const\n+{\n+\tASSERT(maxShutter_);\n+\tASSERT(maxGain_);\n+\n+\tbool gainFixed = minGain_ == maxGain_;\n+\tbool shutterFixed = minShutter_ == maxShutter_;\n+\n+\t/*\n+\t * There's no point entering the loop if we cannot change either gain\n+\t * nor shutter anyway.\n+\t */\n+\tif (shutterFixed && gainFixed)\n+\t\treturn { minShutter_, minGain_, exposure / (minShutter_ * minGain_) };\n+\n+\tutils::Duration shutter;\n+\tdouble stageGain;\n+\tdouble gain;\n+\n+\tfor (unsigned int stage = 0; stage < gains_.size(); stage++) {\n+\t\tdouble lastStageGain = stage == 0 ? 1.0 : clampGain(gains_[stage - 1]);\n+\t\tutils::Duration stageShutter = clampShutter(shutters_[stage]);\n+\t\tstageGain = clampGain(gains_[stage]);\n+\n+\t\t/*\n+\t\t * We perform the clamping on both shutter and gain in case the\n+\t\t * helper has had limits set that prevent those values being\n+\t\t * lowered beyond a certain minimum...this can happen at runtime\n+\t\t * for various reasons and so would not be known when the stage\n+\t\t * limits are initialised.\n+\t\t */\n+\n+\t\tif (stageShutter * lastStageGain >= exposure) {\n+\t\t\tshutter = clampShutter(exposure / clampGain(lastStageGain));\n+\t\t\tgain = clampGain(exposure / shutter);\n+\n+\t\t\treturn { shutter, gain, exposure / (shutter * gain) };\n+\t\t}\n+\n+\t\tif (stageShutter * stageGain >= exposure) {\n+\t\t\tshutter = clampShutter(exposure / clampGain(stageGain));\n+\t\t\tgain = clampGain(exposure / shutter);\n+\n+\t\t\treturn { shutter, gain, exposure / (shutter * gain) };\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * From here on all we can do is max out the shutter time, followed by\n+\t * the analogue gain. If we still haven't achieved the target we send\n+\t * the rest of the exposure time to digital gain. If we were given no\n+\t * stages to use then set stageGain to 1.0 so that shutter time is maxed\n+\t * before gain touched at all.\n+\t */\n+\tif (gains_.empty())\n+\t\tstageGain = 1.0;\n+\n+\tshutter = clampShutter(exposure / clampGain(stageGain));\n+\tgain = clampGain(exposure / shutter);\n+\n+\treturn { shutter, gain, exposure / (shutter * gain) };\n+}\n+\n+/**\n+ * \\fn ExposureModeHelper::minShutter()\n+ * \\brief Retrieve the configured minimum shutter time limit set through\n+ * setShutterGainLimits()\n+ * \\return The minShutter_ value\n+ */\n+\n+/**\n+ * \\fn ExposureModeHelper::maxShutter()\n+ * \\brief Retrieve the configured maximum shutter time set through\n+ * setShutterGainLimits()\n+ * \\return The maxShutter_ value\n+ */\n+\n+/**\n+ * \\fn ExposureModeHelper::minGain()\n+ * \\brief Retrieve the configured minimum gain set through\n+ * setShutterGainLimits()\n+ * \\return The minGain_ value\n+ */\n+\n+/**\n+ * \\fn ExposureModeHelper::maxGain()\n+ * \\brief Retrieve the configured maximum gain set through\n+ * setShutterGainLimits()\n+ * \\return The maxGain_ value\n+ */\n+\n+} /* namespace ipa */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/libipa/exposure_mode_helper.h b/src/ipa/libipa/exposure_mode_helper.h\nnew file mode 100644\nindex 00000000..0ffc164e\n--- /dev/null\n+++ b/src/ipa/libipa/exposure_mode_helper.h\n@@ -0,0 +1,53 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Paul Elder \n+ *\n+ * exposure_mode_helper.h - Helper class that performs computations relating to exposure\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+namespace libcamera {\n+\n+namespace ipa {\n+\n+class ExposureModeHelper\n+{\n+public:\n+\tExposureModeHelper(const Span> stages);\n+\t~ExposureModeHelper() = default;\n+\n+\tvoid setLimits(utils::Duration minShutter, utils::Duration maxShutter,\n+\t\t double minGain, double maxGain);\n+\n+\tstd::tuple\n+\tsplitExposure(utils::Duration exposure) const;\n+\n+\tutils::Duration minShutter() const { return minShutter_; }\n+\tutils::Duration maxShutter() const { return maxShutter_; }\n+\tdouble minGain() const { return minGain_; }\n+\tdouble maxGain() const { return maxGain_; }\n+\n+private:\n+\tutils::Duration clampShutter(utils::Duration shutter) const;\n+\tdouble clampGain(double gain) const;\n+\n+\tstd::vector shutters_;\n+\tstd::vector gains_;\n+\n+\tutils::Duration minShutter_;\n+\tutils::Duration maxShutter_;\n+\tdouble minGain_;\n+\tdouble maxGain_;\n+};\n+\n+} /* namespace ipa */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build\nindex 016b8e0e..37fbd177 100644\n--- a/src/ipa/libipa/meson.build\n+++ b/src/ipa/libipa/meson.build\n@@ -3,6 +3,7 @@\n libipa_headers = files([\n 'algorithm.h',\n 'camera_sensor_helper.h',\n+ 'exposure_mode_helper.h',\n 'fc_queue.h',\n 'histogram.h',\n 'module.h',\n@@ -11,6 +12,7 @@ libipa_headers = files([\n libipa_sources = files([\n 'algorithm.cpp',\n 'camera_sensor_helper.cpp',\n+ 'exposure_mode_helper.cpp',\n 'fc_queue.cpp',\n 'histogram.cpp',\n 'module.cpp',\n","prefixes":["v4","3/8"]}