{"id":15941,"url":"https://patchwork.libcamera.org/api/patches/15941/?format=json","web_url":"https://patchwork.libcamera.org/patch/15941/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220517191833.333122-3-umang.jain@ideasonboard.com>","date":"2022-05-17T19:18:32","name":"[libcamera-devel,v3,2/3] ipa: libipa: Add frame context pointer in process()","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"9bb514bddb7fa39244e543eb892ff4246ca4aa03","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/15941/mbox/","series":[{"id":3118,"url":"https://patchwork.libcamera.org/api/series/3118/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3118","date":"2022-05-17T19:18:30","name":"IPAFrameContext Ring buffer","version":3,"mbox":"https://patchwork.libcamera.org/series/3118/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15941/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15941/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 26B46C0F2A\n\tfor ;\n\tTue, 17 May 2022 19:18:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AF7A965660;\n\tTue, 17 May 2022 21:18:45 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 850256041E\n\tfor ;\n\tTue, 17 May 2022 21:18:43 +0200 (CEST)","from perceval.ideasonboard.com (unknown [45.131.31.124])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 552ED8E6;\n\tTue, 17 May 2022 21:18:42 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652815125;\n\tbh=27si9dP945uWLD3Hwhd2DzgcUm0eVf4husfdps57WQQ=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=iYk+qCOwET3T9LTSjTB2gkC0gw6SkzGt6viEN8k/Hp4r1dE8WtDlB5Gkmc8Jv7AD/\n\tdr2pCOfDr5tftXgxhJybDnLn7yCLcp6wcEbJP+T2MjALKfzZokTsBTAExRtlyz55Zs\n\tl64X45hEYTJ59Cs8xcsL5DtFP6gtmmfTO41nTUMGL4OFXcIceY7yNSf+VHNrLLFk67\n\tzelwy9ZnjMf0dWl5kHH5YdHpTeRftwBZTC6anGndv0Irilmw7ALAjwNr2RTi+8i5T3\n\tlXFRUJUhNwlkQ5wWaIcdqDWJ0r+Xn/vlP8h0yBww6Cwaax8Nus7VRFyCyoV0sHx5Yf\n\toZBkwiHWVvTUA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652815123;\n\tbh=27si9dP945uWLD3Hwhd2DzgcUm0eVf4husfdps57WQQ=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=NQeMyG50aXpxCJfNN5Tk8OuzxVBIGXwg9CAWnQckUeeHo7Xlikd1WeevSiOsEOqnC\n\tBxOmZjXoMvnRg2ZMi2S7KaVyBevMTwOnaH2nTtL/JvCAMGgQEQryQ1i/RWevw/iBXr\n\t/c07xScheAyYhZ3aS9IFSq2DXbSP3qKr4tujC8BA="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"NQeMyG50\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 17 May 2022 21:18:32 +0200","Message-Id":"<20220517191833.333122-3-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","In-Reply-To":"<20220517191833.333122-1-umang.jain@ideasonboard.com>","References":"<20220517191833.333122-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 2/3] ipa: libipa: Add frame context\n\tpointer in process()","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Umang Jain via libcamera-devel ","Reply-To":"Umang Jain ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Currently we have a single structure of IPAFrameContext but\nsubsequently, we shall have a ring buffer (or similar) container\nto keep IPAFrameContext structures for each frame.\n\nIt would be a hassle to query out the frame context required for\nprocess() (since they will reside in a ring buffer) by the IPA\nfor each process. Hence, prepare the process() libipa template to\naccept a particular IPAFrameContext early on.\n\nAs for this patch, we shall pass in the pointer as nullptr, so\nthat the changes compile and keep working as-is.\n\nSigned-off-by: Umang Jain \n---\n src/ipa/ipu3/algorithms/af.cpp | 4 +++-\n src/ipa/ipu3/algorithms/af.h | 3 ++-\n src/ipa/ipu3/algorithms/agc.cpp | 4 +++-\n src/ipa/ipu3/algorithms/agc.h | 3 ++-\n src/ipa/ipu3/algorithms/algorithm.h | 4 +++-\n src/ipa/ipu3/algorithms/awb.cpp | 3 ++-\n src/ipa/ipu3/algorithms/awb.h | 3 ++-\n src/ipa/ipu3/algorithms/tone_mapping.cpp | 3 ++-\n src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++-\n src/ipa/ipu3/ipu3.cpp | 2 +-\n src/ipa/libipa/algorithm.cpp | 1 +\n src/ipa/libipa/algorithm.h | 4 +++-\n src/ipa/rkisp1/algorithms/agc.cpp | 4 +++-\n src/ipa/rkisp1/algorithms/agc.h | 3 ++-\n src/ipa/rkisp1/algorithms/algorithm.h | 4 +++-\n src/ipa/rkisp1/algorithms/awb.cpp | 4 +++-\n src/ipa/rkisp1/algorithms/awb.h | 3 ++-\n src/ipa/rkisp1/rkisp1.cpp | 2 +-\n 18 files changed, 40 insertions(+), 17 deletions(-)","diff":"diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp\nindex 8a5a6b1a..d07521a0 100644\n--- a/src/ipa/ipu3/algorithms/af.cpp\n+++ b/src/ipa/ipu3/algorithms/af.cpp\n@@ -406,6 +406,7 @@ bool Af::afIsOutOfFocus(IPAContext context)\n /**\n * \\brief Determine the max contrast image and lens position.\n * \\param[in] context The IPA context.\n+ * \\param[in] frameContext The current frame context\n * \\param[in] stats The statistics buffer of IPU3.\n *\n * Ideally, a clear image also has a relatively higher contrast. So, every\n@@ -419,7 +420,8 @@ bool Af::afIsOutOfFocus(IPAContext context)\n *\n * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing\n */\n-void Af::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n+void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats)\n {\n \t/* Evaluate the AF buffer length */\n \tuint32_t afRawBufferLen = context.configuration.af.afGrid.width *\ndiff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h\nindex b85cf941..ccf015f3 100644\n--- a/src/ipa/ipu3/algorithms/af.h\n+++ b/src/ipa/ipu3/algorithms/af.h\n@@ -32,7 +32,8 @@ public:\n \n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tvoid afCoarseScan(IPAContext &context);\ndiff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\nindex fdeec09d..383a8deb 100644\n--- a/src/ipa/ipu3/algorithms/agc.cpp\n+++ b/src/ipa/ipu3/algorithms/agc.cpp\n@@ -318,12 +318,14 @@ double Agc::estimateLuminance(IPAActiveState &activeState,\n /**\n * \\brief Process IPU3 statistics, and run AGC operations\n * \\param[in] context The shared IPA context\n+ * \\param[in] frameContext The current frame context\n * \\param[in] stats The IPU3 statistics and ISP results\n *\n * Identify the current image brightness, and use that to estimate the optimal\n * new exposure and gain for the scene.\n */\n-void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n+void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats)\n {\n \t/*\n \t * Estimate the gain needed to have the proportion of pixels in a given\ndiff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h\nindex 31420841..219a1a96 100644\n--- a/src/ipa/ipu3/algorithms/agc.h\n+++ b/src/ipa/ipu3/algorithms/agc.h\n@@ -28,7 +28,8 @@ public:\n \t~Agc() = default;\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tdouble measureBrightness(const ipu3_uapi_stats_3a *stats,\ndiff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h\nindex d2eecc78..234b2bd7 100644\n--- a/src/ipa/ipu3/algorithms/algorithm.h\n+++ b/src/ipa/ipu3/algorithms/algorithm.h\n@@ -17,7 +17,9 @@ namespace libcamera {\n \n namespace ipa::ipu3 {\n \n-using Algorithm = libcamera::ipa::Algorithm;\n+using Algorithm = libcamera::ipa::Algorithm;\n \n } /* namespace ipa::ipu3 */\n \ndiff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp\nindex ab6924eb..5c232d92 100644\n--- a/src/ipa/ipu3/algorithms/awb.cpp\n+++ b/src/ipa/ipu3/algorithms/awb.cpp\n@@ -387,7 +387,8 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats)\n /**\n * \\copydoc libcamera::ipa::Algorithm::process\n */\n-void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n+void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats)\n {\n \tcalculateWBGains(stats);\n \ndiff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h\nindex ab4b0a33..9a50a985 100644\n--- a/src/ipa/ipu3/algorithms/awb.h\n+++ b/src/ipa/ipu3/algorithms/awb.h\n@@ -40,7 +40,8 @@ public:\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \t/* \\todo Make these structs available to all the ISPs ? */\ndiff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp\nindex 7c78d0d9..f86e79b2 100644\n--- a/src/ipa/ipu3/algorithms/tone_mapping.cpp\n+++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp\n@@ -72,12 +72,13 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context,\n /**\n * \\brief Calculate the tone mapping look up table\n * \\param context The shared IPA context\n+ * \\param frameContext The current frame context\n * \\param stats The IPU3 statistics and ISP results\n *\n * The tone mapping look up table is generated as an inverse power curve from\n * our gamma setting.\n */\n-void ToneMapping::process(IPAContext &context,\n+void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n \t\t\t [[maybe_unused]] const ipu3_uapi_stats_3a *stats)\n {\n \t/*\ndiff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h\nindex b727ab1e..d7d48006 100644\n--- a/src/ipa/ipu3/algorithms/tone_mapping.h\n+++ b/src/ipa/ipu3/algorithms/tone_mapping.h\n@@ -20,7 +20,8 @@ public:\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tdouble gamma_;\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex 3b4fc911..16e5028f 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -576,7 +576,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame,\n \tControlList ctrls(controls::controls);\n \n \tfor (auto const &algo : algorithms_)\n-\t\talgo->process(context_, stats);\n+\t\talgo->process(context_, nullptr, stats);\n \n \tsetControls(frame);\n \ndiff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp\nindex 398d5372..cce2ed62 100644\n--- a/src/ipa/libipa/algorithm.cpp\n+++ b/src/ipa/libipa/algorithm.cpp\n@@ -64,6 +64,7 @@ namespace ipa {\n * \\fn Algorithm::process()\n * \\brief Process ISP statistics, and run algorithm operations\n * \\param[in] context The shared IPA context\n+ * \\param[in] frameContext The current frame's context\n * \\param[in] stats The IPA statistics and ISP results\n *\n * This function is called while camera is running for every frame processed by\ndiff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h\nindex 766aee5d..032a05b5 100644\n--- a/src/ipa/libipa/algorithm.h\n+++ b/src/ipa/libipa/algorithm.h\n@@ -10,7 +10,8 @@ namespace libcamera {\n \n namespace ipa {\n \n-template\n+template\n class Algorithm\n {\n public:\n@@ -28,6 +29,7 @@ public:\n \t}\n \n \tvirtual void process([[maybe_unused]] Context &context,\n+\t\t\t [[maybe_unused]] FrameContext *frameContext,\n \t\t\t [[maybe_unused]] const Stats *stats)\n \t{\n \t}\ndiff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex 5f4c3f93..b5a184d9 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -280,7 +280,9 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const\n * Identify the current image brightness, and use that to estimate the optimal\n * new exposure and gain for the scene.\n */\n-void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats)\n+void Agc::process(IPAContext &context,\n+\t\t [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\n \tASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP);\ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex ce1adf27..22c02779 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -29,7 +29,8 @@ public:\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, rkisp1_params_cfg *params) override;\n-\tvoid process(IPAContext &context, const rkisp1_stat_buffer *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const rkisp1_stat_buffer *stats) override;\n \n private:\n \tvoid computeExposure(IPAContext &Context, double yGain, double iqMeanGain);\ndiff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h\nindex d46c3188..68e3a44e 100644\n--- a/src/ipa/rkisp1/algorithms/algorithm.h\n+++ b/src/ipa/rkisp1/algorithms/algorithm.h\n@@ -19,7 +19,9 @@ namespace libcamera {\n \n namespace ipa::rkisp1 {\n \n-using Algorithm = libcamera::ipa::Algorithm;\n+using Algorithm = libcamera::ipa::Algorithm;\n \n } /* namespace ipa::rkisp1 */\n \ndiff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\nindex be4585c6..88441382 100644\n--- a/src/ipa/rkisp1/algorithms/awb.cpp\n+++ b/src/ipa/rkisp1/algorithms/awb.cpp\n@@ -119,7 +119,9 @@ void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params)\n /**\n * \\copydoc libcamera::ipa::Algorithm::process\n */\n-void Awb::process([[maybe_unused]] IPAContext &context, const rkisp1_stat_buffer *stats)\n+void Awb::process([[maybe_unused]] IPAContext &context,\n+\t\t [[maybe_unused]] IPAFrameContext *frameCtx,\n+\t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\n \tconst rkisp1_cif_isp_awb_stat *awb = ¶ms->awb;\ndiff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h\nindex 11946643..7647842f 100644\n--- a/src/ipa/rkisp1/algorithms/awb.h\n+++ b/src/ipa/rkisp1/algorithms/awb.h\n@@ -23,7 +23,8 @@ public:\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, rkisp1_params_cfg *params) override;\n-\tvoid process(IPAContext &context, const rkisp1_stat_buffer *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameCtx,\n+\t\t const rkisp1_stat_buffer *stats) override;\n \n private:\n \tuint32_t estimateCCT(double red, double green, double blue);\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex ef1f0d56..c818a6d7 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -272,7 +272,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId\n \tunsigned int aeState = 0;\n \n \tfor (auto const &algo : algorithms_)\n-\t\talgo->process(context_, stats);\n+\t\talgo->process(context_, nullptr, stats);\n \n \tsetControls(frame);\n \n","prefixes":["libcamera-devel","v3","2/3"]}