{"id":19792,"url":"https://patchwork.libcamera.org/api/1.1/patches/19792/?format=json","web_url":"https://patchwork.libcamera.org/patch/19792/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240322131451.3092931-7-dan.scally@ideasonboard.com>","date":"2024-03-22T13:14:47","name":"[06/10] ipa: ipu3: Derive ipu3::algorithms::Agc from MeanLuminanceAgc","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"0265f60d7bbb69a8a65e454931dd0c9664002439","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/1.1/people/156/?format=json","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19792/mbox/","series":[{"id":4236,"url":"https://patchwork.libcamera.org/api/1.1/series/4236/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4236","date":"2024-03-22T13:14:41","name":"Centralise Agc into libipa","version":1,"mbox":"https://patchwork.libcamera.org/series/4236/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19792/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19792/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D48ECC32C4\n\tfor ;\n\tFri, 22 Mar 2024 13:15:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D0A5363075;\n\tFri, 22 Mar 2024 14:15:18 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D41356330D\n\tfor ;\n\tFri, 22 Mar 2024 14:15:09 +0100 (CET)","from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id DD91313AB;\n\tFri, 22 Mar 2024 14:14:40 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"KpDHTMFe\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1711113281;\n\tbh=X9v72liXQVj9a2RAt2HzvwcVJtOFyAghxcn474kziEc=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=KpDHTMFeCfaZRakA91FEXqzi868C0aDlDhBikpB/dDOH1J0jjO3ja57YlzslBMxf/\n\tE9E22N6GyP1jM1g+avpKR4MuawyX5OjqYRPsquwcIKTwANp4yJV0nXvsroddyhpPSj\n\tp1VsH8Tm0eqG9/SuPt07BDoS3bF3ysNqDVCzZGg4=","From":"Daniel Scally ","To":"libcamera-devel@lists.libcamera.org","Cc":"Daniel Scally ","Subject":"[PATCH 06/10] ipa: ipu3: Derive ipu3::algorithms::Agc from\n\tMeanLuminanceAgc","Date":"Fri, 22 Mar 2024 13:14:47 +0000","Message-Id":"<20240322131451.3092931-7-dan.scally@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20240322131451.3092931-1-dan.scally@ideasonboard.com>","References":"<20240322131451.3092931-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Now that we have a MeanLuminanceAgc class that centralises our AEGC\nalgorithm, derive the IPU3's Agc class from it and plumb in the\nnecessary framework to enable it to be used. For simplicities sake\nthis commit switches the algorithm to use the derived class, but\ndoes not remove the bespoke functions at this time.\n\nSigned-off-by: Daniel Scally \n---\n src/ipa/ipu3/algorithms/agc.cpp | 98 ++++++++++++++++++++++++++++++---\n src/ipa/ipu3/algorithms/agc.h | 6 +-\n src/ipa/ipu3/ipa_context.cpp | 3 +\n src/ipa/ipu3/ipa_context.h | 5 ++\n src/ipa/ipu3/ipu3.cpp | 2 +-\n 5 files changed, 105 insertions(+), 9 deletions(-)","diff":"diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\nindex ee9a42cf..a84534ea 100644\n--- a/src/ipa/ipu3/algorithms/agc.cpp\n+++ b/src/ipa/ipu3/algorithms/agc.cpp\n@@ -71,11 +71,41 @@ static constexpr uint32_t kNumStartupFrames = 10;\n static constexpr double kRelativeLuminanceTarget = 0.16;\n \n Agc::Agc()\n-\t: frameCount_(0), minShutterSpeed_(0s),\n-\t maxShutterSpeed_(0s), filteredExposure_(0s)\n+\t: minShutterSpeed_(0s), maxShutterSpeed_(0s), context_(nullptr)\n {\n }\n \n+/**\n+ * \\brief Initialise the AGC algorith from tuning files\n+ *\n+ * \\param[in] context The shared IPA context\n+ * \\param[in] tuningData The YamlObject containing Agc tuning data\n+ *\n+ * This function calls the base class' tuningData parsers to discover which\n+ * control values are supported.\n+ *\n+ * \\return 0 on success or errors from the base class\n+ */\n+int Agc::init(IPAContext &context, const YamlObject &tuningData)\n+{\n+\tint ret;\n+\n+\tparseRelativeLuminanceTarget(tuningData);\n+\n+\tret = parseConstraintModes(tuningData);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = parseExposureModes(tuningData);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tcontext.ctrlMap.merge(controls());\n+\tcontext_ = &context;\n+\n+\treturn 0;\n+}\n+\n /**\n * \\brief Configure the AGC given a configInfo\n * \\param[in] context The shared IPA context\n@@ -103,6 +133,20 @@ int Agc::configure(IPAContext &context,\n \tactiveState.agc.exposure = 10ms / configuration.sensor.lineDuration;\n \n \tframeCount_ = 0;\n+\n+\t/*\n+\t * \\todo We should use the first available mode rather than assume that\n+\t * the \"Normal\" modes are present in tuning data.\n+\t */\n+\tcontext.activeState.agc.constraintMode = controls::ConstraintNormal;\n+\tcontext.activeState.agc.exposureMode = controls::ExposureNormal;\n+\n+\tfor (auto &[id, helper] : exposureModeHelpers()) {\n+\t\t/* \\todo Run this again when FrameDurationLimits is passed in */\n+\t\thelper->configure(minShutterSpeed_, maxShutterSpeed_,\n+\t\t\t\t minAnalogueGain_, maxAnalogueGain_);\n+\t}\n+\n \treturn 0;\n }\n \n@@ -280,11 +324,6 @@ void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext,\n \tLOG(IPU3Agc, Debug) << \"Divided up shutter and gain are \"\n \t\t\t << shutterTime << \" and \"\n \t\t\t << stepGain;\n-\n-\tIPAActiveState &activeState = context.activeState;\n-\t/* Update the estimated exposure and gain. */\n-\tactiveState.agc.exposure = shutterTime / configuration.sensor.lineDuration;\n-\tactiveState.agc.gain = stepGain;\n }\n \n /**\n@@ -347,6 +386,26 @@ double Agc::estimateLuminance(IPAActiveState &activeState,\n \treturn ySum / (grid.height * grid.width) / 255;\n }\n \n+double Agc::estimateLuminance(double gain)\n+{\n+\tASSERT(reds_.size() == greens_.size());\n+\tASSERT(greens_.size() == blues_.size());\n+\tconst ipu3_uapi_grid_config &grid = context_->configuration.grid.bdsGrid;\n+\tdouble redSum = 0, greenSum = 0, blueSum = 0;\n+\n+\tfor (unsigned int i = 0; i < reds_.size(); i++) {\n+\t\tredSum += std::min(reds_[i] * gain, 255.0);\n+\t\tgreenSum += std::min(greens_[i] * gain, 255.0);\n+\t\tblueSum += std::min(blues_[i] * gain, 255.0);\n+\t}\n+\n+\tdouble ySum = redSum * context_->activeState.awb.gains.red * 0.299\n+\t\t+ greenSum * context_->activeState.awb.gains.green * 0.587\n+\t\t+ blueSum * context_->activeState.awb.gains.blue * 0.114;\n+\n+\treturn ySum / (grid.height * grid.width) / 255;\n+}\n+\n /**\n * \\brief Process IPU3 statistics, and run AGC operations\n * \\param[in] context The shared IPA context\n@@ -399,8 +458,33 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \tcomputeExposure(context, frameContext, yGain, iqMeanGain);\n \tframeCount_++;\n \n+\tparseStatistics(stats, context.configuration.grid.bdsGrid);\n+\n+\t/*\n+\t * The Agc algorithm needs to know the effective exposure value that was\n+\t * applied to the sensor when the statistics were collected.\n+\t */\n \tutils::Duration exposureTime = context.configuration.sensor.lineDuration\n \t\t\t\t * frameContext.sensor.exposure;\n+\tdouble analogueGain = frameContext.sensor.gain;\n+\tutils::Duration effectiveExposureValue = exposureTime * analogueGain;\n+\n+\tutils::Duration shutterTime;\n+\tdouble aGain, dGain;\n+\tstd::tie(shutterTime, aGain, dGain) =\n+\t\tcalculateNewEv(context.activeState.agc.constraintMode,\n+\t\t\t context.activeState.agc.exposureMode, hist_,\n+\t\t\t effectiveExposureValue);\n+\n+\tLOG(IPU3Agc, Debug)\n+\t\t<< \"Divided up shutter, analogue gain and digital gain are \"\n+\t\t<< shutterTime << \", \" << aGain << \" and \" << dGain;\n+\n+\tIPAActiveState &activeState = context.activeState;\n+\t/* Update the estimated exposure and gain. */\n+\tactiveState.agc.exposure = shutterTime / context.configuration.sensor.lineDuration;\n+\tactiveState.agc.gain = aGain;\n+\n \tmetadata.set(controls::AnalogueGain, frameContext.sensor.gain);\n \tmetadata.set(controls::ExposureTime, exposureTime.get());\n \ndiff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h\nindex 7ed0ef7a..8405da9d 100644\n--- a/src/ipa/ipu3/algorithms/agc.h\n+++ b/src/ipa/ipu3/algorithms/agc.h\n@@ -13,6 +13,7 @@\n \n #include \n \n+#include \"libipa/agc.h\"\n #include \"libipa/histogram.h\"\n \n #include \"algorithm.h\"\n@@ -23,12 +24,13 @@ struct IPACameraSensorInfo;\n \n namespace ipa::ipu3::algorithms {\n \n-class Agc : public Algorithm\n+class Agc : public Algorithm, public MeanLuminanceAgc\n {\n public:\n \tAgc();\n \t~Agc() = default;\n \n+\tint init(IPAContext &context, const YamlObject &tuningData) override;\n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid process(IPAContext &context, const uint32_t frame,\n \t\t IPAFrameContext &frameContext,\n@@ -45,6 +47,7 @@ private:\n \t\t\t\t const ipu3_uapi_grid_config &grid,\n \t\t\t\t const ipu3_uapi_stats_3a *stats,\n \t\t\t\t double gain);\n+\tdouble estimateLuminance(double gain) override;\n \tvoid parseStatistics(const ipu3_uapi_stats_3a *stats,\n \t\t\t const ipu3_uapi_grid_config &grid);\n \n@@ -59,6 +62,7 @@ private:\n \tutils::Duration filteredExposure_;\n \n \tuint32_t stride_;\n+\tIPAContext *context_;\n \tstd::vector reds_;\n \tstd::vector blues_;\n \tstd::vector greens_;\ndiff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp\nindex 959f314f..c4fb5642 100644\n--- a/src/ipa/ipu3/ipa_context.cpp\n+++ b/src/ipa/ipu3/ipa_context.cpp\n@@ -47,6 +47,9 @@ namespace libcamera::ipa::ipu3 {\n *\n * \\var IPAContext::activeState\n * \\brief The current state of IPA algorithms\n+ *\n+ * \\var IPAContext::ctrlMap\n+ * \\brief A ControlInfoMap::Map of controls populated by the algorithms\n */\n \n /**\ndiff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h\nindex e9a3863b..a92cb6ce 100644\n--- a/src/ipa/ipu3/ipa_context.h\n+++ b/src/ipa/ipu3/ipa_context.h\n@@ -12,6 +12,7 @@\n \n #include \n \n+#include \n #include \n \n #include \n@@ -55,6 +56,8 @@ struct IPAActiveState {\n \tstruct {\n \t\tuint32_t exposure;\n \t\tdouble gain;\n+\t\tuint32_t constraintMode;\n+\t\tuint32_t exposureMode;\n \t} agc;\n \n \tstruct {\n@@ -85,6 +88,8 @@ struct IPAContext {\n \tIPAActiveState activeState;\n \n \tFCQueue frameContexts;\n+\n+\tControlInfoMap::Map ctrlMap;\n };\n \n } /* namespace ipa::ipu3 */\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex 08ee6eb3..2fcc0c91 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -189,7 +189,7 @@ private:\n };\n \n IPAIPU3::IPAIPU3()\n-\t: context_({ {}, {}, { kMaxFrameContexts } })\n+\t: context_({ {}, {}, { kMaxFrameContexts }, {} })\n {\n }\n \n","prefixes":["06/10"]}