{"id":20310,"url":"https://patchwork.libcamera.org/api/1.1/patches/20310/?format=json","web_url":"https://patchwork.libcamera.org/patch/20310/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240614074214.3600996-2-paul.elder@ideasonboard.com>","date":"2024-06-14T07:42:13","name":"[v6,1/2] ipa: rkisp1: agc: Read histogram weights from tuning file","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"e753f01c07da4707a4a27ae97bf83dace183f8a3","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/1.1/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/20310/mbox/","series":[{"id":4392,"url":"https://patchwork.libcamera.org/api/1.1/series/4392/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4392","date":"2024-06-14T07:42:12","name":"ipa: rkisp1: Improve AGC (plumbing)","version":6,"mbox":"https://patchwork.libcamera.org/series/4392/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/20310/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/20310/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 08A32C3237\n\tfor ;\n\tFri, 14 Jun 2024 07:42:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AB93365490;\n\tFri, 14 Jun 2024 09:42:30 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id EA4F065489\n\tfor ;\n\tFri, 14 Jun 2024 09:42:28 +0200 (CEST)","from neptunite.hamster-moth.ts.net\n\t(h175-177-049-156.catv02.itscom.jp [175.177.49.156])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C10DA10C4;\n\tFri, 14 Jun 2024 09:42:11 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vb4jH4t/\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1718350934;\n\tbh=C6ldniQmD5q1I7bDiZm2NJLLxayQYyaP6l57y/MXzww=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=vb4jH4t/elYDc7AYoXrlsSx0k7WWRwAAOlxywUMVpm4SlWBzrXjSOey3qsAvq9iPe\n\t4evM8TQ9SzKeuFA2uTsMyl1SsO/OLKidWhEptkIHcL88+ARL2QJ3vJtANJ3nGcqw40\n\t58E4p8HY914HFi3rPIiecks6EhyrG3Zd0aDo7I9Q=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Cc":"Paul Elder ,\n\tStefan Klug ,\n\tDaniel Scally ","Subject":"[PATCH v6 1/2] ipa: rkisp1: agc: Read histogram weights from tuning\n\tfile","Date":"Fri, 14 Jun 2024 16:42:13 +0900","Message-Id":"<20240614074214.3600996-2-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.39.2","In-Reply-To":"<20240614074214.3600996-1-paul.elder@ideasonboard.com>","References":"<20240614074214.3600996-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add support to the rkisp1 AGC to read histogram weights from the tuning\nfile. As controls for selecting the metering mode are not yet supported,\nfor now hardcode the matrix metering mode, which is the same as what the\nAGC previously hardcoded.\n\nSigned-off-by: Paul Elder \nReviewed-by: Stefan Klug \nReviewed-by: Daniel Scally \n\n---\nChanges in v6:\n- fix predivider calculation\n- misc cleanups\n- remove support for v12 (technically it depends on what the tuning file\n provides)\n\nNo change in v5\n\nChanges in v4:\n- add debug print when parsing the yaml file\n\nChanges in v3:\n- support the new tuning file layout for interpolated matrices\n- support both v10 and v12\n\nChanges in v2:\n- add default metering mode if none are read successfully from the\n tuning file\n- compute the predivider instead of using a table\n---\n src/ipa/rkisp1/algorithms/agc.cpp | 100 +++++++++++++++++++++++++++++-\n src/ipa/rkisp1/algorithms/agc.h | 6 ++\n 2 files changed, 103 insertions(+), 3 deletions(-)","diff":"diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex 50e0690fe146..5a9c6be118d9 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -17,6 +17,8 @@\n #include \n #include \n \n+#include \"libcamera/internal/yaml_parser.h\"\n+\n #include \"libipa/histogram.h\"\n \n /**\n@@ -36,6 +38,85 @@ namespace ipa::rkisp1::algorithms {\n \n LOG_DEFINE_CATEGORY(RkISP1Agc)\n \n+int Agc::parseMeteringModes(IPAContext &context, const YamlObject &tuningData)\n+{\n+\tif (!tuningData.isDictionary()) {\n+\t\tLOG(RkISP1Agc, Error)\n+\t\t\t<< \"'AeMeteringMode' parameter not found in tuning file\";\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tfor (const auto &[key, value] : tuningData.asDict()) {\n+\t\tif (controls::AeMeteringModeNameValueMap.find(key) ==\n+\t\t controls::AeMeteringModeNameValueMap.end()) {\n+\t\t\tLOG(RkISP1Agc, Warning)\n+\t\t\t\t<< \"Skipping unknown metering mode '\" << key << \"'\";\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tstd::vector weights =\n+\t\t\tvalue.getList().value_or(std::vector{});\n+\t\tif (weights.size() != context.hw->numHistogramWeights) {\n+\t\t\tLOG(RkISP1Agc, Warning)\n+\t\t\t\t<< \"Failed to read metering mode'\" << key << \"'\";\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tmeteringModes_[controls::AeMeteringModeNameValueMap.at(key)] = weights;\n+\t}\n+\n+\tif (meteringModes_.empty()) {\n+\t\tLOG(RkISP1Agc, Warning)\n+\t\t\t<< \"No metering modes read from tuning file; defaulting to matrix\";\n+\t\tint32_t meteringModeId = controls::AeMeteringModeNameValueMap.at(\"MeteringMatrix\");\n+\t\tstd::vector weights(context.hw->numHistogramWeights, 1);\n+\n+\t\tmeteringModes_[meteringModeId] = weights;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+uint8_t Agc::computeHistogramPredivider(Size &size, enum rkisp1_cif_isp_histogram_mode mode)\n+{\n+\t/*\n+\t * The maximum number of pixels that could potentially be in one bin is\n+\t * if all the pixels of the image are in it, multiplied by 3 for the\n+\t * three color channels. The counter for each bin is 16 bits wide, so\n+\t * `factor` thus contains the number of times we'd wrap around. This is\n+\t * obviously the number of pixels that we need to skip to make sure\n+\t * that we don't wrap around, but we compute the square root of it\n+\t * instead, as the skip that we need to program is for both the x and y\n+\t * directions.\n+\t *\n+\t * Even though it looks like dividing into a counter of 65536 would\n+\t * overflow by 1, this is apparently fine according to the hardware\n+\t * documentation, and this successfully gets the expected documented\n+\t * predivider size for cases where:\n+\t * (width / predivider) * (height / predivider) * 3 == 65536.\n+\t *\n+\t * There's a bit of extra rounding math to make sure the rounding goes\n+\t * the correct direction so that the square of the step is big enough\n+\t * to encompass the `factor` number of pixels that we need to skip.\n+\t *\n+\t * \\todo Take into account weights. That is, if the weights are low\n+\t * enough we can potentially reduce the predivider to increase\n+\t * precision. This needs some investigation however, as this hardware\n+\t * behavior is undocumented and is only an educated guess.\n+\t */\n+\tint count = mode == RKISP1_CIF_ISP_HISTOGRAM_MODE_RGB_COMBINED ? 3 : 1;\n+\tdouble factor = size.width * size.height * count / 65536.0;\n+\tdouble root = std::sqrt(factor);\n+\tuint8_t predivider;\n+\n+\tif (std::pow(std::floor(root), 2) < factor)\n+\t\tpredivider = static_cast(std::ceil(root));\n+\telse\n+\t\tpredivider = static_cast(std::floor(root));\n+\n+\treturn std::clamp(predivider, 3, 127);\n+}\n+\n Agc::Agc()\n {\n \tsupportsRaw_ = true;\n@@ -59,6 +140,11 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData)\n \tif (ret)\n \t\treturn ret;\n \n+\tconst YamlObject &yamlMeteringModes = tuningData[\"AeMeteringMode\"];\n+\tret = parseMeteringModes(context, yamlMeteringModes);\n+\tif (ret)\n+\t\treturn ret;\n+\n \tcontext.ctrlMap.merge(controls());\n \n \treturn 0;\n@@ -160,6 +246,7 @@ void Agc::prepare(IPAContext &context, const uint32_t frame,\n \t\tframeContext.agc.gain = context.activeState.agc.automatic.gain;\n \t}\n \n+\t/* \\todo Remove this when we can set the below with controls */\n \tif (frame > 0)\n \t\treturn;\n \n@@ -178,14 +265,21 @@ void Agc::prepare(IPAContext &context, const uint32_t frame,\n \tparams->meas.hst_config.meas_window = context.configuration.agc.measureWindow;\n \t/* Produce the luminance histogram. */\n \tparams->meas.hst_config.mode = RKISP1_CIF_ISP_HISTOGRAM_MODE_Y_HISTOGRAM;\n+\n \t/* Set an average weighted histogram. */\n \tSpan weights{\n \t\tparams->meas.hst_config.hist_weight,\n \t\tcontext.hw->numHistogramWeights\n \t};\n-\tstd::fill(weights.begin(), weights.end(), 1);\n-\t/* Step size can't be less than 3. */\n-\tparams->meas.hst_config.histogram_predivider = 4;\n+\t/* \\todo Get this from control */\n+\tstd::vector &modeWeights = meteringModes_.at(controls::MeteringMatrix);\n+\tstd::copy(modeWeights.begin(), modeWeights.end(), weights.begin());\n+\n+\tstruct rkisp1_cif_isp_window window = params->meas.hst_config.meas_window;\n+\tSize windowSize = { window.h_size, window.v_size };\n+\tparams->meas.hst_config.histogram_predivider =\n+\t\tcomputeHistogramPredivider(windowSize,\n+\t\t\t\t\t static_cast(params->meas.hst_config.mode));\n \n \t/* Update the configuration for histogram. */\n \tparams->module_cfg_update |= RKISP1_CIF_ISP_MODULE_HST;\ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex 04b3247e1276..996fea71755c 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -44,11 +44,17 @@ public:\n \t\t ControlList &metadata) override;\n \n private:\n+\tint parseMeteringModes(IPAContext &context, const YamlObject &tuningData);\n+\tuint8_t computeHistogramPredivider(Size &size,\n+\t\t\t\t\t enum rkisp1_cif_isp_histogram_mode mode);\n+\n \tvoid fillMetadata(IPAContext &context, IPAFrameContext &frameContext,\n \t\t\t ControlList &metadata);\n \tdouble estimateLuminance(double gain) const override;\n \n \tSpan expMeans_;\n+\n+\tstd::map> meteringModes_;\n };\n \n } /* namespace ipa::rkisp1::algorithms */\n","prefixes":["v6","1/2"]}