{"id":19854,"url":"https://patchwork.libcamera.org/api/1.1/patches/19854/?format=json","web_url":"https://patchwork.libcamera.org/patch/19854/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240405144729.2992219-5-paul.elder@ideasonboard.com>","date":"2024-04-05T14:47:28","name":"[4/5] ipa: libipa: agc: Change luminance target to piecewise linear function","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"29f942259850f6149c1634ba37965e4817828cb6","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/1.1/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19854/mbox/","series":[{"id":4251,"url":"https://patchwork.libcamera.org/api/1.1/series/4251/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4251","date":"2024-04-05T14:47:24","name":"ipa: rkisp1: Improve AGC (plumbing)","version":1,"mbox":"https://patchwork.libcamera.org/series/4251/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19854/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19854/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1240CC32CB\n\tfor ;\n\tFri, 5 Apr 2024 14:47:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 82CEF63361;\n\tFri, 5 Apr 2024 16:47:55 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AAD4E63361\n\tfor ;\n\tFri, 5 Apr 2024 16:47:52 +0200 (CEST)","from pyrite.hamster-moth.ts.net (h175-177-049-156.catv02.itscom.jp\n\t[175.177.49.156])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id BEA6363B;\n\tFri, 5 Apr 2024 16:47:12 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"oIEVdEaW\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1712328434;\n\tbh=d6IIQmH1l7yyHbcCWvhNL7wu4MUBA+yNtu0ss2BZ+4E=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=oIEVdEaWn0zSJaucA+swLHoEyNjT9kb9trQeuwzyuk8fqZ9+cb6iMRJmTuiyb9KJ/\n\tIzOeo1D2TW9pLaLbwDKqCalArvZ5JBFL3LpbdTELqhObXY5tjXnLrA4GbZFsKCjwvf\n\t0XZGEgPBuFGJi0CtdG2jvwC0nh5N+hTVDCmvbKZc=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Cc":"Paul Elder ","Subject":"[PATCH 4/5] ipa: libipa: agc: Change luminance target to piecewise\n\tlinear function","Date":"Fri, 5 Apr 2024 23:47:28 +0900","Message-Id":"<20240405144729.2992219-5-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.39.2","In-Reply-To":"<20240405144729.2992219-1-paul.elder@ideasonboard.com>","References":"<20240405144729.2992219-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Change the relative luminance target from a scalar valur to a piecewise\nlinear function that needs to be sampled by the estimate lux value.\n\nAlso chagne the rkisp1 and ipu3 IPAs according, as they use the libipa\nagc. As they both don't yet have lux modules, hardcode them to a single\nlux value for now.\n\nThis affects the format of the tuning files, but as there aren't yet any\nthis shouldn't be an issue.\n\nSigned-off-by: Paul Elder \n---\n src/ipa/ipu3/algorithms/agc.cpp | 5 ++++-\n src/ipa/libipa/agc.cpp | 22 +++++++++++++++-------\n src/ipa/libipa/agc.h | 7 ++++---\n src/ipa/rkisp1/algorithms/agc.cpp | 5 ++++-\n 4 files changed, 27 insertions(+), 12 deletions(-)","diff":"diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\nindex 08deff0c..8e07c89e 100644\n--- a/src/ipa/ipu3/algorithms/agc.cpp\n+++ b/src/ipa/ipu3/algorithms/agc.cpp\n@@ -228,12 +228,15 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \tdouble analogueGain = frameContext.sensor.gain;\n \tutils::Duration effectiveExposureValue = exposureTime * analogueGain;\n \n+\t/* \\todo Plumb in the lux value. Requires a lux algo + tuning */\n+\tdouble lux = 400;\n+\n \tutils::Duration shutterTime;\n \tdouble aGain, dGain;\n \tstd::tie(shutterTime, aGain, dGain) =\n \t\tcalculateNewEv(context.activeState.agc.constraintMode,\n \t\t\t context.activeState.agc.exposureMode, hist_,\n-\t\t\t effectiveExposureValue);\n+\t\t\t effectiveExposureValue, lux);\n \n \tLOG(IPU3Agc, Debug)\n \t\t<< \"Divided up shutter, analogue gain and digital gain are \"\ndiff --git a/src/ipa/libipa/agc.cpp b/src/ipa/libipa/agc.cpp\nindex af57a571..bcb036e6 100644\n--- a/src/ipa/libipa/agc.cpp\n+++ b/src/ipa/libipa/agc.cpp\n@@ -110,7 +110,7 @@ static constexpr double kDefaultRelativeLuminanceTarget = 0.16;\n */\n \n MeanLuminanceAgc::MeanLuminanceAgc()\n-\t: frameCount_(0), filteredExposure_(0s), relativeLuminanceTarget_(0)\n+\t: frameCount_(0), filteredExposure_(0s)\n {\n }\n \n@@ -120,8 +120,12 @@ MeanLuminanceAgc::MeanLuminanceAgc()\n */\n void MeanLuminanceAgc::parseRelativeLuminanceTarget(const YamlObject &tuningData)\n {\n-\trelativeLuminanceTarget_ =\n-\t\ttuningData[\"relativeLuminanceTarget\"].get(kDefaultRelativeLuminanceTarget);\n+\tint ret = relativeLuminanceTarget_.readYaml(tuningData[\"relativeLuminanceTarget\"]);\n+\tif (ret == 0)\n+\t\treturn;\n+\n+\tstd::vector points = { { 0, kDefaultRelativeLuminanceTarget } };\n+\trelativeLuminanceTarget_ = Pwl(points);\n }\n \n /**\n@@ -378,6 +382,7 @@ int MeanLuminanceAgc::parseExposureModes(const YamlObject &tuningData)\n /**\n * \\brief Estimate the initial gain needed to achieve a relative luminance\n * target\n+ * \\param[in] lux The lux value at which to sample the luminance target pwl\n *\n * To account for non-linearity caused by saturation, the value needs to be\n * estimated in an iterative process, as multiplying by a gain will not increase\n@@ -385,9 +390,10 @@ int MeanLuminanceAgc::parseExposureModes(const YamlObject &tuningData)\n *\n * \\return The calculated initial gain\n */\n-double MeanLuminanceAgc::estimateInitialGain()\n+double MeanLuminanceAgc::estimateInitialGain(double lux)\n {\n-\tdouble yTarget = relativeLuminanceTarget_;\n+\tdouble yTarget =\n+\t\trelativeLuminanceTarget_.eval(relativeLuminanceTarget_.domain().clamp(lux));\n \tdouble yGain = 1.0;\n \n \tfor (unsigned int i = 0; i < 8; i++) {\n@@ -476,6 +482,7 @@ utils::Duration MeanLuminanceAgc::filterExposure(utils::Duration exposureValue)\n *\t the calculated gain\n * \\param[in] effectiveExposureValue The EV applied to the frame from which the\n *\t statistics in use derive\n+ * \\param[in] lux The lux value at which to sample the luminance target pwl\n *\n * Calculate a new exposure value to try to obtain the target. The calculated\n * exposure value is filtered to prevent rapid changes from frame to frame, and\n@@ -487,7 +494,8 @@ std::tuple\n MeanLuminanceAgc::calculateNewEv(uint32_t constraintModeIndex,\n \t\t\t\t uint32_t exposureModeIndex,\n \t\t\t\t const Histogram &yHist,\n-\t\t\t\t utils::Duration effectiveExposureValue)\n+\t\t\t\t utils::Duration effectiveExposureValue,\n+\t\t\t\t double lux)\n {\n \t/*\n \t * The pipeline handler should validate that we have received an allowed\n@@ -496,7 +504,7 @@ MeanLuminanceAgc::calculateNewEv(uint32_t constraintModeIndex,\n \tstd::shared_ptr exposureModeHelper =\n \t\texposureModeHelpers_.at(exposureModeIndex);\n \n-\tdouble gain = estimateInitialGain();\n+\tdouble gain = estimateInitialGain(lux);\n \tgain = constraintClampGain(constraintModeIndex, yHist, gain);\n \n \t/*\ndiff --git a/src/ipa/libipa/agc.h b/src/ipa/libipa/agc.h\nindex 902a359a..f187dbc8 100644\n--- a/src/ipa/libipa/agc.h\n+++ b/src/ipa/libipa/agc.h\n@@ -16,6 +16,7 @@\n \n #include \"exposure_mode_helper.h\"\n #include \"histogram.h\"\n+#include \"pwl.h\"\n \n namespace libcamera {\n \n@@ -59,18 +60,18 @@ public:\n \t}\n \n \tvirtual double estimateLuminance(const double gain) = 0;\n-\tdouble estimateInitialGain();\n+\tdouble estimateInitialGain(double lux);\n \tdouble constraintClampGain(uint32_t constraintModeIndex,\n \t\t\t\t const Histogram &hist,\n \t\t\t\t double gain);\n \tutils::Duration filterExposure(utils::Duration exposureValue);\n \tstd::tuple\n \tcalculateNewEv(uint32_t constraintModeIndex, uint32_t exposureModeIndex,\n-\t\t const Histogram &yHist, utils::Duration effectiveExposureValue);\n+\t\t const Histogram &yHist, utils::Duration effectiveExposureValue, double lux);\n private:\n \tuint64_t frameCount_;\n \tutils::Duration filteredExposure_;\n-\tdouble relativeLuminanceTarget_;\n+\tPwl relativeLuminanceTarget_;\n \n \tstd::map> constraintModes_;\n \tstd::map> exposureModeHelpers_;\ndiff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex 1dfc4aaa..a1b6eb39 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -389,12 +389,15 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \tdouble analogueGain = frameContext.sensor.gain;\n \tutils::Duration effectiveExposureValue = exposureTime * analogueGain;\n \n+\t/* \\todo Plumb in the lux value. Requires a lux algo + tuning */\n+\tdouble lux = 400;\n+\n \tutils::Duration shutterTime;\n \tdouble aGain, dGain;\n \tstd::tie(shutterTime, aGain, dGain) =\n \t\tcalculateNewEv(context.activeState.agc.constraintMode,\n \t\t\t context.activeState.agc.exposureMode, hist_,\n-\t\t\t effectiveExposureValue);\n+\t\t\t effectiveExposureValue, lux);\n \n \tLOG(RkISP1Agc, Debug)\n \t\t<< \"Divided up shutter, analogue gain and digital gain are \"\n","prefixes":["4/5"]}