{"id":20034,"url":"https://patchwork.libcamera.org/api/1.1/patches/20034/?format=json","web_url":"https://patchwork.libcamera.org/patch/20034/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240509175555.43121-4-paul.elder@ideasonboard.com>","date":"2024-05-09T17:55:55","name":"[v2,3/3] ipa: rkisp1: agc: Plumb mode-selection and frame duration controls","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"79b59a0e9f309387691cc8c97a39a2716c97e5ad","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/1.1/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/20034/mbox/","series":[{"id":4304,"url":"https://patchwork.libcamera.org/api/1.1/series/4304/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4304","date":"2024-05-09T17:55:52","name":"ipa: rkisp1: Improve AGC (plumbing)","version":2,"mbox":"https://patchwork.libcamera.org/series/4304/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/20034/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/20034/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5901FC3226\n\tfor ;\n\tThu, 9 May 2024 17:56:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 086C16346D;\n\tThu, 9 May 2024 19:56:18 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4818363466\n\tfor ;\n\tThu, 9 May 2024 19:56:14 +0200 (CEST)","from neptunite.hamster-moth.ts.net\n\t(h175-177-049-156.catv02.itscom.jp [175.177.49.156])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 91AA92CF2;\n\tThu, 9 May 2024 19:56:09 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"HzM834cv\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1715277370;\n\tbh=bLaW2hWawKLMRs4VQo6jg42OA1BKgXeGqIVGMdWFb0o=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=HzM834cvyywiCkFnPmYcjDUKAOCdOqFjP9gU/RWbAAIjffTpsQA+CRVNg442td8/h\n\tRNwsidR3bmAE+t/SFmBjMHW3D2nyryjx3zRpBVD0Wrlrcssr6hl94IIoJLGpAWLdD6\n\tdk3onQ4WDZDwdW7hd3rnTfvNu5+RI+GQ9fCH19qs=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Cc":"Paul Elder ","Subject":"[PATCH v2 3/3] ipa: rkisp1: agc: Plumb mode-selection and frame\n\tduration controls","Date":"Fri, 10 May 2024 02:55:55 +0900","Message-Id":"<20240509175555.43121-4-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.39.2","In-Reply-To":"<20240509175555.43121-1-paul.elder@ideasonboard.com>","References":"<20240509175555.43121-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Plumb controls for setting metering mode, exposure mode, constraint\nmode, and frame duration limits. Also report them as available controls,\nas well as in metadata.\n\nWhile at it, add the missing #include for tuple, as a std::tie is used.\n\nSigned-off-by: Paul Elder \n\n---\nChanges in v2:\n- add #include \n- don't overwrite metering/exposure/constraint mode controls to default\n if no control is supplied, and retain the old value instead (same for\n frame duration limits)\n---\n src/ipa/rkisp1/algorithms/agc.cpp | 52 ++++++++++++++++++++++++---\n src/ipa/rkisp1/algorithms/algorithm.h | 2 ++\n src/ipa/rkisp1/ipa_context.h | 10 ++++--\n src/ipa/rkisp1/rkisp1.cpp | 10 ++++++\n 4 files changed, 67 insertions(+), 7 deletions(-)","diff":"diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex bd9a6a66..74bbda1a 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -10,6 +10,8 @@\n #include \n #include \n #include \n+#include \n+#include \n \n #include \n #include \n@@ -48,6 +50,7 @@ int Agc::parseMeteringModes(IPAContext &context, const YamlObject &tuningData,\n \t\treturn -EINVAL;\n \t}\n \n+\tstd::vector availableMeteringModes;\n \tfor (const auto &[key, value] : yamlMeteringModes.asDict()) {\n \t\tif (controls::AeMeteringModeNameValueMap.find(key) ==\n \t\t controls::AeMeteringModeNameValueMap.end()) {\n@@ -66,10 +69,12 @@ int Agc::parseMeteringModes(IPAContext &context, const YamlObject &tuningData,\n \t\t\treturn -ENODATA;\n \t\t}\n \n-\t\tmeteringModes_[controls::AeMeteringModeNameValueMap.at(key)] = weights;\n+\t\tint32_t control = controls::AeMeteringModeNameValueMap.at(key);\n+\t\tmeteringModes_[control] = weights;\n+\t\tavailableMeteringModes.push_back(control);\n \t}\n \n-\tif (meteringModes_.empty()) {\n+\tif (availableMeteringModes.empty()) {\n \t\tint32_t meteringModeId = controls::AeMeteringModeNameValueMap.at(\"MeteringMatrix\");\n \t\tstd::vector weights = {\n \t\t\t1, 1, 1, 1, 1,\n@@ -80,8 +85,12 @@ int Agc::parseMeteringModes(IPAContext &context, const YamlObject &tuningData,\n \t\t};\n \n \t\tmeteringModes_[meteringModeId] = weights;\n+\t\tavailableMeteringModes.push_back(meteringModeId);\n \t}\n \n+\tAlgorithm::controls_[&controls::AeMeteringMode] =\n+\t\tControlInfo(availableMeteringModes);\n+\n \treturn 0;\n }\n \n@@ -142,6 +151,8 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData)\n \n \tcontext.ctrlMap.merge(controls());\n \n+\tAlgorithm::controls_.merge(ControlInfoMap::Map(controls()));\n+\n \treturn 0;\n }\n \n@@ -164,6 +175,7 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo)\n \n \tcontext.activeState.agc.constraintMode = constraintModes().begin()->first;\n \tcontext.activeState.agc.exposureMode = exposureModeHelpers().begin()->first;\n+\tcontext.activeState.agc.meteringMode = meteringModes_.begin()->first;\n \n \t/*\n \t * Define the measurement window for AGC as a centered rectangle\n@@ -174,7 +186,6 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo)\n \tcontext.configuration.agc.measureWindow.h_size = 3 * configInfo.outputSize.width / 4;\n \tcontext.configuration.agc.measureWindow.v_size = 3 * configInfo.outputSize.height / 4;\n \n-\t/* \\todo Run this again when FrameDurationLimits is passed in */\n \tsetLimits(context.configuration.sensor.minShutterSpeed,\n \t\t context.configuration.sensor.maxShutterSpeed,\n \t\t context.configuration.sensor.minAnalogueGain,\n@@ -228,6 +239,26 @@ void Agc::queueRequest(IPAContext &context,\n \t\tframeContext.agc.exposure = agc.manual.exposure;\n \t\tframeContext.agc.gain = agc.manual.gain;\n \t}\n+\n+\tconst auto &meteringMode = controls.get(controls::AeMeteringMode);\n+\tif (meteringMode)\n+\t\tagc.meteringMode = *meteringMode;\n+\tframeContext.agc.meteringMode = agc.meteringMode;\n+\n+\tconst auto &exposureMode = controls.get(controls::AeExposureMode);\n+\tif (exposureMode)\n+\t\tagc.exposureMode = *exposureMode;\n+\tframeContext.agc.exposureMode = agc.exposureMode;\n+\n+\tconst auto &constraintMode = controls.get(controls::AeConstraintMode);\n+\tif (constraintMode)\n+\t\tagc.constraintMode = *constraintMode;\n+\tframeContext.agc.constraintMode = agc.constraintMode;\n+\n+\tconst auto &frameDurationLimits = controls.get(controls::FrameDurationLimits);\n+\tif (frameDurationLimits)\n+\t\tagc.maxShutterSpeed = std::chrono::milliseconds((*frameDurationLimits).back());\n+\tframeContext.agc.maxShutterSpeed = agc.maxShutterSpeed;\n }\n \n /**\n@@ -266,8 +297,7 @@ void Agc::prepare(IPAContext &context, const uint32_t frame,\n \t\tparams->meas.hst_config.hist_weight,\n \t\tcontext.hw->numHistogramWeights\n \t};\n-\t/* \\todo Get this from control */\n-\tstd::vector &modeWeights = meteringModes_.at(controls::MeteringMatrix);\n+\tstd::vector &modeWeights = meteringModes_.at(frameContext.agc.meteringMode);\n \tstd::copy(modeWeights.begin(), modeWeights.end(), weights.begin());\n \n \tstd::stringstream str;\n@@ -294,6 +324,7 @@ void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext,\n \t\t\t\t * frameContext.sensor.exposure;\n \tmetadata.set(controls::AnalogueGain, frameContext.sensor.gain);\n \tmetadata.set(controls::ExposureTime, exposureTime.get());\n+\tmetadata.set(controls::AeEnable, frameContext.agc.autoEnabled);\n \n \t/* \\todo Use VBlank value calculated from each frame exposure. */\n \tuint32_t vTotal = context.configuration.sensor.size.height\n@@ -301,6 +332,10 @@ void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext,\n \tutils::Duration frameDuration = context.configuration.sensor.lineDuration\n \t\t\t\t * vTotal;\n \tmetadata.set(controls::FrameDuration, frameDuration.get());\n+\n+\tmetadata.set(controls::AeMeteringMode, frameContext.agc.meteringMode);\n+\tmetadata.set(controls::AeExposureMode, frameContext.agc.exposureMode);\n+\tmetadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode);\n }\n \n /**\n@@ -375,6 +410,13 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \t\t [](uint32_t x) { return x >> 4; });\n \texpMeans_ = { params->ae.exp_mean, context.hw->numAeCells };\n \n+\tutils::Duration maxShutterSpeed = std::min(context.configuration.sensor.maxShutterSpeed,\n+\t\t\t\t\t\t frameContext.agc.maxShutterSpeed);\n+\tsetLimits(context.configuration.sensor.minShutterSpeed,\n+\t\t maxShutterSpeed,\n+\t\t context.configuration.sensor.minAnalogueGain,\n+\t\t context.configuration.sensor.maxAnalogueGain);\n+\n \t/*\n \t * The Agc algorithm needs to know the effective exposure value that was\n \t * applied to the sensor when the statistics were collected.\ndiff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h\nindex 9454c9a1..c3a002b8 100644\n--- a/src/ipa/rkisp1/algorithms/algorithm.h\n+++ b/src/ipa/rkisp1/algorithms/algorithm.h\n@@ -25,6 +25,8 @@ public:\n \n \tbool disabled_;\n \tbool supportsRaw_;\n+\n+\tControlInfoMap::Map controls_;\n };\n \n } /* namespace ipa::rkisp1 */\ndiff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\nindex 256b75eb..26c39570 100644\n--- a/src/ipa/rkisp1/ipa_context.h\n+++ b/src/ipa/rkisp1/ipa_context.h\n@@ -68,8 +68,10 @@ struct IPAActiveState {\n \t\t} automatic;\n \n \t\tbool autoEnabled;\n-\t\tuint32_t constraintMode;\n-\t\tuint32_t exposureMode;\n+\t\tint32_t constraintMode;\n+\t\tint32_t exposureMode;\n+\t\tint32_t meteringMode;\n+\t\tutils::Duration maxShutterSpeed;\n \t} agc;\n \n \tstruct {\n@@ -111,6 +113,10 @@ struct IPAFrameContext : public FrameContext {\n \t\tuint32_t exposure;\n \t\tdouble gain;\n \t\tbool autoEnabled;\n+\t\tint32_t exposureMode;\n+\t\tint32_t constraintMode;\n+\t\tint32_t meteringMode;\n+\t\tutils::Duration maxShutterSpeed;\n \t} agc;\n \n \tstruct {\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex 963cdbab..7716c11b 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -80,6 +80,7 @@ private:\n \tstd::map mappedBuffers_;\n \n \tControlInfoMap sensorControls_;\n+\tControlInfoMap::Map algoControls_;\n \n \t/* Interface to the Camera Helper */\n \tstd::unique_ptr camHelper_;\n@@ -193,6 +194,14 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision,\n \tif (ret)\n \t\treturn ret;\n \n+\tfor (auto const &a : algorithms()) {\n+\t\tAlgorithm *algo = static_cast(a.get());\n+\n+\t\t/* \\todo Avoid merging duplicate controls */\n+\t\tif (!algo->controls_.empty())\n+\t\t\talgoControls_.merge(ControlInfoMap::Map(algo->controls_));\n+\t}\n+\n \t/* Initialize controls. */\n \tupdateControls(sensorInfo, sensorControls, ipaControls);\n \n@@ -377,6 +386,7 @@ void IPARkISP1::updateControls(const IPACameraSensorInfo &sensorInfo,\n \t\t\t ControlInfoMap *ipaControls)\n {\n \tControlInfoMap::Map ctrlMap = rkisp1Controls;\n+\tctrlMap.merge(algoControls_);\n \n \t/*\n \t * Compute exposure time limits from the V4L2_CID_EXPOSURE control\n","prefixes":["v2","3/3"]}