{"id":19795,"url":"https://patchwork.libcamera.org/api/patches/19795/?format=json","web_url":"https://patchwork.libcamera.org/patch/19795/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240322131451.3092931-10-dan.scally@ideasonboard.com>","date":"2024-03-22T13:14:50","name":"[09/10] ipa: rkisp1: Derive rkisp1::algorithms::Agc from MeanLuminanceAgc","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"12f2b70db6e181457a18d3fab5d820a7797961e8","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/people/156/?format=json","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19795/mbox/","series":[{"id":4236,"url":"https://patchwork.libcamera.org/api/series/4236/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4236","date":"2024-03-22T13:14:41","name":"Centralise Agc into libipa","version":1,"mbox":"https://patchwork.libcamera.org/series/4236/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19795/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19795/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A3073C32C6\n\tfor ;\n\tFri, 22 Mar 2024 13:15:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 51E1A63365;\n\tFri, 22 Mar 2024 14:15:23 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3791A6331B\n\tfor ;\n\tFri, 22 Mar 2024 14:15:11 +0100 (CET)","from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 42AF0842;\n\tFri, 22 Mar 2024 14:14:42 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"czVbsI9M\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1711113282;\n\tbh=MchdhWt7VO2UZ5A91x/o2dRbQJp5CMuOhmRxDyMDCNo=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=czVbsI9MRmDwhyTI40r35SArHaeespK5UCnHhzNHV2ovGOC2OXJl6SBD++1CDkILA\n\tIU5Pc6PQwdCQ6s+C7hBogLi6adE/vw5LFTqnYfKI5vh44pQh1eY3zl9cJRSAPDT5Kh\n\tkoofIjh7l5bhO3E0h7alnWzd2MyKeP4Ubj5N3Gz4=","From":"Daniel Scally ","To":"libcamera-devel@lists.libcamera.org","Cc":"Daniel Scally ","Subject":"[PATCH 09/10] ipa: rkisp1: Derive rkisp1::algorithms::Agc from\n\tMeanLuminanceAgc","Date":"Fri, 22 Mar 2024 13:14:50 +0000","Message-Id":"<20240322131451.3092931-10-dan.scally@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20240322131451.3092931-1-dan.scally@ideasonboard.com>","References":"<20240322131451.3092931-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Now that we have a MeanLuminanceAgc class that centralises our AEGC\nalgorithm, derive the RkISP1's Agc class from it and plumb in the\nnecessary framework to enable it to be used. For simplicities sake\nthis commit switches the algorithm to use the derived class, but\ndoes not remove the bespoke functions at this time.\n\nSigned-off-by: Daniel Scally \n---\n src/ipa/rkisp1/algorithms/agc.cpp | 91 +++++++++++++++++++++++++++++--\n src/ipa/rkisp1/algorithms/agc.h | 5 +-\n src/ipa/rkisp1/ipa_context.h | 5 ++\n src/ipa/rkisp1/rkisp1.cpp | 2 +-\n 4 files changed, 96 insertions(+), 7 deletions(-)","diff":"diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex d380194d..3389c471 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -64,6 +64,36 @@ Agc::Agc()\n \tsupportsRaw_ = true;\n }\n \n+/**\n+ * \\brief Initialise the AGC algorithm from tuning files\n+ *\n+ * \\param[in] context The shared IPA context\n+ * \\param[in] tuningData The YamlObject containing Agc tuning data\n+ *\n+ * This function calls the base class' tuningData parsers to discover which\n+ * control values are supported.\n+ *\n+ * \\return 0 on success or errors from the base class\n+ */\n+int Agc::init(IPAContext &context, const YamlObject &tuningData)\n+{\n+\tint ret;\n+\n+\tparseRelativeLuminanceTarget(tuningData);\n+\n+\tret = parseConstraintModes(tuningData);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = parseExposureModes(tuningData);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tcontext.ctrlMap.merge(controls());\n+\n+\treturn 0;\n+}\n+\n /**\n * \\brief Configure the AGC given a configInfo\n * \\param[in] context The shared IPA context\n@@ -81,6 +111,13 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo)\n \tcontext.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure;\n \tcontext.activeState.agc.autoEnabled = !context.configuration.raw;\n \n+\t/*\n+\t* \\todo We should use the first available mode rather than assume that\n+\t* the \"Normal\" modes are present in tuning data.\n+\t*/\n+\tcontext.activeState.agc.constraintMode = controls::ConstraintNormal;\n+\tcontext.activeState.agc.exposureMode = controls::ExposureNormal;\n+\n \t/*\n \t * Define the measurement window for AGC as a centered rectangle\n \t * covering 3/4 of the image width and height.\n@@ -95,6 +132,15 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo)\n \t * frame index\n \t */\n \tframeCount_ = 0;\n+\n+\tfor (auto &[id, helper] : exposureModeHelpers()) {\n+\t\t/* \\todo Run this again when FrameDurationLimits is passed in */\n+\t\thelper->configure(context.configuration.sensor.minShutterSpeed,\n+\t\t\t\t context.configuration.sensor.maxShutterSpeed,\n+\t\t\t\t context.configuration.sensor.minAnalogueGain,\n+\t\t\t\t context.configuration.sensor.maxAnalogueGain);\n+\t}\n+\n \treturn 0;\n }\n \n@@ -234,7 +280,6 @@ void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext,\n \t\t\t double yGain, double iqMeanGain)\n {\n \tIPASessionConfiguration &configuration = context.configuration;\n-\tIPAActiveState &activeState = context.activeState;\n \n \t/* Get the effective exposure and gain applied on the sensor. */\n \tuint32_t exposure = frameContext.sensor.exposure;\n@@ -300,10 +345,6 @@ void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext,\n \tLOG(RkISP1Agc, Debug) << \"Divided up shutter and gain are \"\n \t\t\t << shutterTime << \" and \"\n \t\t\t << stepGain;\n-\n-\t/* Update the estimated exposure and gain. */\n-\tactiveState.agc.automatic.exposure = shutterTime / configuration.sensor.lineDuration;\n-\tactiveState.agc.automatic.gain = stepGain;\n }\n \n /**\n@@ -383,6 +424,19 @@ void Agc::parseStatistics(const rkisp1_stat_buffer *stats,\n \t\t\t\t\t context.hw->numHistogramBins));\n }\n \n+double Agc::estimateLuminance(double gain)\n+{\n+\tdouble ySum = 0.0;\n+\n+\t/* Sum the averages, saturated to 255. */\n+\tfor (uint8_t expMean : expMeans_)\n+\t\tySum += std::min(expMean * gain, 255.0);\n+\n+\t/* \\todo Weight with the AWB gains */\n+\n+\treturn ySum / expMeans_.size() / 255;\n+}\n+\n /**\n * \\brief Process RkISP1 statistics, and run AGC operations\n * \\param[in] context The shared IPA context\n@@ -448,6 +502,33 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \tcomputeExposure(context, frameContext, yGain, iqMeanGain);\n \tframeCount_++;\n \n+\tparseStatistics(stats, context);\n+\n+\t/*\n+\t * The Agc algorithm needs to know the effective exposure value that was\n+\t * applied to the sensor when the statistics were collected.\n+\t */\n+\tutils::Duration exposureTime = context.configuration.sensor.lineDuration\n+\t\t\t\t * frameContext.sensor.exposure;\n+\tdouble analogueGain = frameContext.sensor.gain;\n+\tutils::Duration effectiveExposureValue = exposureTime * analogueGain;\n+\n+\tutils::Duration shutterTime;\n+\tdouble aGain, dGain;\n+\tstd::tie(shutterTime, aGain, dGain) =\n+\t\tcalculateNewEv(context.activeState.agc.constraintMode,\n+\t\t\t context.activeState.agc.exposureMode, hist_,\n+\t\t\t effectiveExposureValue);\n+\n+\tLOG(RkISP1Agc, Debug)\n+\t\t<< \"Divided up shutter, analogue gain and digital gain are \"\n+\t\t<< shutterTime << \", \" << aGain << \" and \" << dGain;\n+\n+\tIPAActiveState &activeState = context.activeState;\n+\t/* Update the estimated exposure and gain. */\n+\tactiveState.agc.automatic.exposure = shutterTime / context.configuration.sensor.lineDuration;\n+\tactiveState.agc.automatic.gain = aGain;\n+\n \tfillMetadata(context, frameContext, metadata);\n }\n \ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex b0de4898..1271741e 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -14,6 +14,7 @@\n \n #include \n \n+#include \"libipa/agc.h\"\n #include \"libipa/histogram.h\"\n \n #include \"algorithm.h\"\n@@ -22,12 +23,13 @@ namespace libcamera {\n \n namespace ipa::rkisp1::algorithms {\n \n-class Agc : public Algorithm\n+class Agc : public Algorithm, public MeanLuminanceAgc\n {\n public:\n \tAgc();\n \t~Agc() = default;\n \n+\tint init(IPAContext &context, const YamlObject &tuningData) override;\n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid queueRequest(IPAContext &context,\n \t\t\t const uint32_t frame,\n@@ -51,6 +53,7 @@ private:\n \t\t\t ControlList &metadata);\n \tvoid parseStatistics(const rkisp1_stat_buffer *stats,\n \t\t\t IPAContext &context);\n+\tdouble estimateLuminance(double gain) override;\n \n \tuint64_t frameCount_;\n \ndiff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\nindex 10d8f38c..256b75eb 100644\n--- a/src/ipa/rkisp1/ipa_context.h\n+++ b/src/ipa/rkisp1/ipa_context.h\n@@ -12,6 +12,7 @@\n \n #include \n \n+#include \n #include \n \n #include \n@@ -67,6 +68,8 @@ struct IPAActiveState {\n \t\t} automatic;\n \n \t\tbool autoEnabled;\n+\t\tuint32_t constraintMode;\n+\t\tuint32_t exposureMode;\n \t} agc;\n \n \tstruct {\n@@ -151,6 +154,8 @@ struct IPAContext {\n \tIPAActiveState activeState;\n \n \tFCQueue frameContexts;\n+\n+\tControlInfoMap::Map ctrlMap;\n };\n \n } /* namespace ipa::rkisp1 */\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex 9dc5f53c..5f74762f 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -119,7 +119,7 @@ const ControlInfoMap::Map rkisp1Controls{\n } /* namespace */\n \n IPARkISP1::IPARkISP1()\n-\t: context_({ {}, {}, {}, { kMaxFrameContexts } })\n+\t: context_({ {}, {}, {}, { kMaxFrameContexts }, {} })\n {\n }\n \n","prefixes":["09/10"]}