{"id":17328,"url":"https://patchwork.libcamera.org/api/1.1/patches/17328/?format=json","web_url":"https://patchwork.libcamera.org/patch/17328/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220908014200.28728-20-laurent.pinchart@ideasonboard.com>","date":"2022-09-08T01:41:47","name":"[libcamera-devel,v4,19/32] ipa: rkisp1: agc: Store per-frame information in frame context","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"bc723db4433ac13d2b45f8bab0123e525c8874f0","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/1.1/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17328/mbox/","series":[{"id":3476,"url":"https://patchwork.libcamera.org/api/1.1/series/3476/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3476","date":"2022-09-08T01:41:28","name":"ipa: Frame context queue, IPU3 & RkISP consolidation, and RkISP1 improvements","version":4,"mbox":"https://patchwork.libcamera.org/series/3476/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17328/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17328/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 691EFC3272\n\tfor ;\n\tThu, 8 Sep 2022 01:42:47 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1A627620DD;\n\tThu, 8 Sep 2022 03:42:47 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B1E8A620DF\n\tfor ;\n\tThu, 8 Sep 2022 03:42:44 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 403F26CC\n\tfor ;\n\tThu, 8 Sep 2022 03:42:44 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1662601367;\n\tbh=z122C3wGPVnmSw1BpNS36W2k/1kgikwymrCKG6k6QhM=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=dQF8bJbjBf3SLIg0DifO+7YYPLKYaVM5FCIy4j7xIX6mxGYjAfswPYYdb3DEUOFh2\n\t+JGoj00x6thYoiBuolrT9xEL7unt2E6vcFFxt6I4+Rkkj3QULkSygjHtO06q0fJhUy\n\teHr4Q2pFBqhxUo/hq312dWf6DjsphBHRfFjliwPiwf/QEA2jp97kRjZe67nJ46vLYx\n\tuC9P9ikHt1KwkyuvxgzaGnPuwK6Y6XJd3TK4TTR5W5+cwrz8+mSOwqTgLqQ2J+YJ8O\n\trWLZUK3kpVpJ6pdCok3iPAprIOThxSe96tKxCoycuu4Xn53iEwQ+PAPYXsXPFP2qT/\n\tbvlm5fjNhn5+A==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1662601364;\n\tbh=z122C3wGPVnmSw1BpNS36W2k/1kgikwymrCKG6k6QhM=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=oqJl8HNNtW3fCKrfhvONj4AxVRK4F1LCbpfR+bB6TUwonLsMokONp4TbqhSlJikpF\n\tCpJzuKRAUCj0gsUqZ/pi/twokMT9A2hvbeqw2gS6L8CZ/fNWc6zS9vWKqGuvEaj2Ls\n\t+xH1467gQTGsWagEskO1NfWSCKf0Ez7GddIykLuo="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"oqJl8HNN\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 8 Sep 2022 04:41:47 +0300","Message-Id":"<20220908014200.28728-20-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.35.1","In-Reply-To":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>","References":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v4 19/32] ipa: rkisp1: agc: Store\n\tper-frame information in frame context","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Rework the algorithm's usage of the active state to store the value of\ncontrols for the last queued request in the queueRequest() function, and\nstore a copy of the values in the corresponding frame context.\n\nThe frame context is used in the prepare() function to populate the ISP\nparameters with values corresponding to the right frame.\n\nSigned-off-by: Laurent Pinchart \n---\n src/ipa/rkisp1/algorithms/agc.cpp | 18 ++++++++-----\n src/ipa/rkisp1/algorithms/agc.h | 3 ++-\n src/ipa/rkisp1/ipa_context.cpp | 43 ++++++++++++++++++++-----------\n src/ipa/rkisp1/ipa_context.h | 14 ++++++----\n src/ipa/rkisp1/rkisp1.cpp | 9 ++++---\n 5 files changed, 55 insertions(+), 32 deletions(-)","diff":"diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex 4124c3272bc7..e7198169d11b 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -144,17 +144,19 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue)\n /**\n * \\brief Estimate the new exposure and gain values\n * \\param[inout] context The shared IPA Context\n+ * \\param[in] frameContext The FrameContext for this frame\n * \\param[in] yGain The gain calculated on the current brightness level\n * \\param[in] iqMeanGain The gain calculated based on the relative luminance target\n */\n-void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain)\n+void Agc::computeExposure(IPAContext &context, RkISP1FrameContext &frameContext,\n+\t\t\t double yGain, double iqMeanGain)\n {\n \tIPASessionConfiguration &configuration = context.configuration;\n \tIPAActiveState &activeState = context.activeState;\n \n \t/* Get the effective exposure and gain applied on the sensor. */\n-\tuint32_t exposure = activeState.sensor.exposure;\n-\tdouble analogueGain = activeState.sensor.gain;\n+\tuint32_t exposure = frameContext.sensor.exposure;\n+\tdouble analogueGain = frameContext.sensor.gain;\n \n \t/* Use the highest of the two gain estimates. */\n \tdouble evGain = std::max(yGain, iqMeanGain);\n@@ -285,7 +287,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const\n * new exposure and gain for the scene.\n */\n void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n-\t\t [[maybe_unused]] RkISP1FrameContext &frameContext,\n+\t\t RkISP1FrameContext &frameContext,\n \t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\n@@ -319,7 +321,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n \t\t\tbreak;\n \t}\n \n-\tcomputeExposure(context, yGain, iqMeanGain);\n+\tcomputeExposure(context, frameContext, yGain, iqMeanGain);\n \tframeCount_++;\n }\n \n@@ -327,9 +329,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n * \\copydoc libcamera::ipa::Algorithm::prepare\n */\n void Agc::prepare(IPAContext &context, const uint32_t frame,\n-\t\t [[maybe_unused]] RkISP1FrameContext &frameContext,\n-\t\t rkisp1_params_cfg *params)\n+\t\t RkISP1FrameContext &frameContext, rkisp1_params_cfg *params)\n {\n+\tframeContext.agc.exposure = context.activeState.agc.exposure;\n+\tframeContext.agc.gain = context.activeState.agc.gain;\n+\n \tif (frame > 0)\n \t\treturn;\n \ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex be8932040c8e..d6c6fb130fc1 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -34,7 +34,8 @@ public:\n \t\t const rkisp1_stat_buffer *stats) override;\n \n private:\n-\tvoid computeExposure(IPAContext &Context, double yGain, double iqMeanGain);\n+\tvoid computeExposure(IPAContext &Context, RkISP1FrameContext &frameContext,\n+\t\t\t double yGain, double iqMeanGain);\n \tutils::Duration filterExposure(utils::Duration exposureValue);\n \tdouble estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain);\n \tdouble measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const;\ndiff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp\nindex 211428717838..88e0631c8d39 100644\n--- a/src/ipa/rkisp1/ipa_context.cpp\n+++ b/src/ipa/rkisp1/ipa_context.cpp\n@@ -104,16 +104,13 @@ namespace libcamera::ipa::rkisp1 {\n * \\var IPAActiveState::agc\n * \\brief State for the Automatic Gain Control algorithm\n *\n- * The exposure and gain determined are expected to be applied to the sensor\n- * at the earliest opportunity.\n+ * The exposure and gain are the latest values computed by the AGC algorithm.\n *\n * \\var IPAActiveState::agc.exposure\n * \\brief Exposure time expressed as a number of lines\n *\n * \\var IPAActiveState::agc.gain\n * \\brief Analogue gain multiplier\n- *\n- * The gain should be adapted to the sensor specific gain code before applying.\n */\n \n /**\n@@ -181,17 +178,6 @@ namespace libcamera::ipa::rkisp1 {\n * \\brief Indicates if ISP parameters need to be updated\n */\n \n-/**\n- * \\var IPAActiveState::sensor\n- * \\brief Effective sensor values\n- *\n- * \\var IPAActiveState::sensor.exposure\n- * \\brief Exposure time expressed as a number of lines\n- *\n- * \\var IPAActiveState::sensor.gain\n- * \\brief Analogue gain multiplier\n- */\n-\n /**\n * \\struct RkISP1FrameContext\n * \\brief Per-frame context for algorithms\n@@ -199,6 +185,33 @@ namespace libcamera::ipa::rkisp1 {\n * \\todo Populate the frame context for all algorithms\n */\n \n+/**\n+ * \\var RkISP1FrameContext::agc\n+ * \\brief Automatic Gain Control parameters for this frame\n+ *\n+ * The exposure and gain are provided by the AGC algorithm, and are to be\n+ * applied to the sensor in order to take effect for this frame.\n+ *\n+ * \\var RkISP1FrameContext::agc.exposure\n+ * \\brief Exposure time expressed as a number of lines\n+ *\n+ * \\var RkISP1FrameContext::agc.gain\n+ * \\brief Analogue gain multiplier\n+ *\n+ * The gain should be adapted to the sensor specific gain code before applying.\n+ */\n+\n+/**\n+ * \\var RkISP1FrameContext::sensor\n+ * \\brief Sensor configuration that used been used for this frame\n+ *\n+ * \\var RkISP1FrameContext::sensor.exposure\n+ * \\brief Exposure time expressed as a number of lines\n+ *\n+ * \\var RkISP1FrameContext::sensor.gain\n+ * \\brief Analogue gain multiplier\n+ */\n+\n /**\n * \\struct IPAContext\n * \\brief Global IPA context data shared between all algorithms\ndiff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\nindex 4e38ef40ed3c..ecf993cd22d7 100644\n--- a/src/ipa/rkisp1/ipa_context.h\n+++ b/src/ipa/rkisp1/ipa_context.h\n@@ -82,14 +82,18 @@ struct IPAActiveState {\n \t\tuint8_t sharpness;\n \t\tbool updateParams;\n \t} filter;\n-\n-\tstruct {\n-\t\tuint32_t exposure;\n-\t\tdouble gain;\n-\t} sensor;\n };\n \n struct RkISP1FrameContext : public FrameContext {\n+\tstruct {\n+\t\tuint32_t exposure;\n+\t\tdouble gain;\n+\t} agc;\n+\n+\tstruct {\n+\t\tuint32_t exposure;\n+\t\tdouble gain;\n+\t} sensor;\n };\n \n struct IPAContext {\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex 4f6e0b0f87b9..a23cfc7fb24d 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -334,9 +334,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId\n \t\treinterpret_cast(\n \t\t\tmappedBuffers_.at(bufferId).planes()[0].data());\n \n-\tcontext_.activeState.sensor.exposure =\n+\tframeContext.sensor.exposure =\n \t\tsensorControls.get(V4L2_CID_EXPOSURE).get();\n-\tcontext_.activeState.sensor.gain =\n+\tframeContext.sensor.gain =\n \t\tcamHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get());\n \n \tunsigned int aeState = 0;\n@@ -351,8 +351,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId\n \n void IPARkISP1::setControls(unsigned int frame)\n {\n-\tuint32_t exposure = context_.activeState.agc.exposure;\n-\tuint32_t gain = camHelper_->gainCode(context_.activeState.agc.gain);\n+\tRkISP1FrameContext &frameContext = context_.frameContexts.get(frame);\n+\tuint32_t exposure = frameContext.agc.exposure;\n+\tuint32_t gain = camHelper_->gainCode(frameContext.agc.gain);\n \n \tControlList ctrls(ctrls_);\n \tctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure));\n","prefixes":["libcamera-devel","v4","19/32"]}