{"id":15948,"url":"https://patchwork.libcamera.org/api/patches/15948/?format=json","web_url":"https://patchwork.libcamera.org/patch/15948/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220518131030.421225-3-umang.jain@ideasonboard.com>","date":"2022-05-18T13:10:29","name":"[libcamera-devel,v4,2/3] ipa: libipa: Add frame context pointer in process()","commit_ref":"8b291bce82f7cc8307e8ef55ff20e3f41462fa3f","pull_url":null,"state":"accepted","archived":false,"hash":"9bb514bddb7fa39244e543eb892ff4246ca4aa03","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/15948/mbox/","series":[{"id":3121,"url":"https://patchwork.libcamera.org/api/series/3121/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3121","date":"2022-05-18T13:10:27","name":"[libcamera-devel,v4,1/3] ipa: ipu3: Rework IPAFrameContext","version":4,"mbox":"https://patchwork.libcamera.org/series/3121/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15948/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15948/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 44CE7C3256\n\tfor ;\n\tWed, 18 May 2022 13:10:45 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 756FF65658;\n\tWed, 18 May 2022 15:10:44 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B62B465656\n\tfor ;\n\tWed, 18 May 2022 15:10:41 +0200 (CEST)","from perceval.ideasonboard.com (unknown [45.131.31.124])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0FB96E50;\n\tWed, 18 May 2022 15:10:41 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1652879444;\n\tbh=Qda6Wkdmk9B85sanqlXJcxj6JNor0Ye3TnC2+lv8Qhg=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=wUJFLxg8qJNIAHuCx1WP6AeQ6sKdCs7vlbktY4kNwSKFoLXxSHdmzHgOlwr5nh9Y0\n\tCRVCFUSCTodJCSH7YjpMZ1Ly+37jnwHbs9A0WiBJHN9cMa7F0nmsfY0JJ4xY0XQc/4\n\tnhvTYHTAwIbPKCxUmWOcyZ8T6Z0LBMDbkITdd76ywWqlWeTuWRqba9kKD6Qa293tb+\n\tI87LBiV1d10Fe69FMAvAbzFKdUXSR0vhL8+h1smvJKmtrzmF/hlvBCbd7anedDDTH0\n\t2GHKHZfEkJUgFKOGAKjoQK/BmGCCgB9MgzRgUI6dhFt4YgvKWBfg97XUUXtcpmYZHq\n\tgFrwKvdgx/buA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1652879441;\n\tbh=Qda6Wkdmk9B85sanqlXJcxj6JNor0Ye3TnC2+lv8Qhg=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=ECKLyi75Ece89W7gCeoPYzCGgvkKYzfVYjlM01/PQardXPVebTBrCdVU78lh+Va5w\n\t+7RlIfONanWbkieWwSX5wa6ZlsnUX61mKExCvCNVX8n2tCFM1LxYdVrVUXucG62KaT\n\ts6y+pgLC3nbhvi4EpR1lPqGrBN9fJJ5Ud16uWKpA="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"ECKLyi75\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 18 May 2022 15:10:29 +0200","Message-Id":"<20220518131030.421225-3-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","In-Reply-To":"<20220518131030.421225-1-umang.jain@ideasonboard.com>","References":"<20220518131030.421225-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v4 2/3] ipa: libipa: Add frame context\n\tpointer in process()","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Umang Jain via libcamera-devel ","Reply-To":"Umang Jain ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Currently we have a single structure of IPAFrameContext but\nsubsequently, we shall have a ring buffer (or similar) container\nto keep IPAFrameContext structures for each frame.\n\nIt would be a hassle to query out the frame context required for\nprocess() (since they will reside in a ring buffer) by the IPA\nfor each process. Hence, prepare the process() libipa template to\naccept a particular IPAFrameContext early on.\n\nAs for this patch, we shall pass in the pointer as nullptr, so\nthat the changes compile and keep working as-is.\n\nSigned-off-by: Umang Jain \nReviewed-by: Jacopo Mondi \nReviewed-by: Jean-Michel Hautbois \nReviewed-by: Kieran Bingham \n---\n src/ipa/ipu3/algorithms/af.cpp | 4 +++-\n src/ipa/ipu3/algorithms/af.h | 3 ++-\n src/ipa/ipu3/algorithms/agc.cpp | 4 +++-\n src/ipa/ipu3/algorithms/agc.h | 3 ++-\n src/ipa/ipu3/algorithms/algorithm.h | 4 +++-\n src/ipa/ipu3/algorithms/awb.cpp | 3 ++-\n src/ipa/ipu3/algorithms/awb.h | 3 ++-\n src/ipa/ipu3/algorithms/tone_mapping.cpp | 3 ++-\n src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++-\n src/ipa/ipu3/ipu3.cpp | 2 +-\n src/ipa/libipa/algorithm.cpp | 1 +\n src/ipa/libipa/algorithm.h | 4 +++-\n src/ipa/rkisp1/algorithms/agc.cpp | 4 +++-\n src/ipa/rkisp1/algorithms/agc.h | 3 ++-\n src/ipa/rkisp1/algorithms/algorithm.h | 4 +++-\n src/ipa/rkisp1/algorithms/awb.cpp | 4 +++-\n src/ipa/rkisp1/algorithms/awb.h | 3 ++-\n src/ipa/rkisp1/rkisp1.cpp | 2 +-\n 18 files changed, 40 insertions(+), 17 deletions(-)","diff":"diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp\nindex 8a5a6b1a..d07521a0 100644\n--- a/src/ipa/ipu3/algorithms/af.cpp\n+++ b/src/ipa/ipu3/algorithms/af.cpp\n@@ -406,6 +406,7 @@ bool Af::afIsOutOfFocus(IPAContext context)\n /**\n * \\brief Determine the max contrast image and lens position.\n * \\param[in] context The IPA context.\n+ * \\param[in] frameContext The current frame context\n * \\param[in] stats The statistics buffer of IPU3.\n *\n * Ideally, a clear image also has a relatively higher contrast. So, every\n@@ -419,7 +420,8 @@ bool Af::afIsOutOfFocus(IPAContext context)\n *\n * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing\n */\n-void Af::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n+void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats)\n {\n \t/* Evaluate the AF buffer length */\n \tuint32_t afRawBufferLen = context.configuration.af.afGrid.width *\ndiff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h\nindex b85cf941..ccf015f3 100644\n--- a/src/ipa/ipu3/algorithms/af.h\n+++ b/src/ipa/ipu3/algorithms/af.h\n@@ -32,7 +32,8 @@ public:\n \n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tvoid afCoarseScan(IPAContext &context);\ndiff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\nindex fdeec09d..383a8deb 100644\n--- a/src/ipa/ipu3/algorithms/agc.cpp\n+++ b/src/ipa/ipu3/algorithms/agc.cpp\n@@ -318,12 +318,14 @@ double Agc::estimateLuminance(IPAActiveState &activeState,\n /**\n * \\brief Process IPU3 statistics, and run AGC operations\n * \\param[in] context The shared IPA context\n+ * \\param[in] frameContext The current frame context\n * \\param[in] stats The IPU3 statistics and ISP results\n *\n * Identify the current image brightness, and use that to estimate the optimal\n * new exposure and gain for the scene.\n */\n-void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n+void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats)\n {\n \t/*\n \t * Estimate the gain needed to have the proportion of pixels in a given\ndiff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h\nindex 31420841..219a1a96 100644\n--- a/src/ipa/ipu3/algorithms/agc.h\n+++ b/src/ipa/ipu3/algorithms/agc.h\n@@ -28,7 +28,8 @@ public:\n \t~Agc() = default;\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tdouble measureBrightness(const ipu3_uapi_stats_3a *stats,\ndiff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h\nindex d2eecc78..234b2bd7 100644\n--- a/src/ipa/ipu3/algorithms/algorithm.h\n+++ b/src/ipa/ipu3/algorithms/algorithm.h\n@@ -17,7 +17,9 @@ namespace libcamera {\n \n namespace ipa::ipu3 {\n \n-using Algorithm = libcamera::ipa::Algorithm;\n+using Algorithm = libcamera::ipa::Algorithm;\n \n } /* namespace ipa::ipu3 */\n \ndiff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp\nindex ab6924eb..5c232d92 100644\n--- a/src/ipa/ipu3/algorithms/awb.cpp\n+++ b/src/ipa/ipu3/algorithms/awb.cpp\n@@ -387,7 +387,8 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats)\n /**\n * \\copydoc libcamera::ipa::Algorithm::process\n */\n-void Awb::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n+void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats)\n {\n \tcalculateWBGains(stats);\n \ndiff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h\nindex ab4b0a33..9a50a985 100644\n--- a/src/ipa/ipu3/algorithms/awb.h\n+++ b/src/ipa/ipu3/algorithms/awb.h\n@@ -40,7 +40,8 @@ public:\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \t/* \\todo Make these structs available to all the ISPs ? */\ndiff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp\nindex 7c78d0d9..f86e79b2 100644\n--- a/src/ipa/ipu3/algorithms/tone_mapping.cpp\n+++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp\n@@ -72,12 +72,13 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context,\n /**\n * \\brief Calculate the tone mapping look up table\n * \\param context The shared IPA context\n+ * \\param frameContext The current frame context\n * \\param stats The IPU3 statistics and ISP results\n *\n * The tone mapping look up table is generated as an inverse power curve from\n * our gamma setting.\n */\n-void ToneMapping::process(IPAContext &context,\n+void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n \t\t\t [[maybe_unused]] const ipu3_uapi_stats_3a *stats)\n {\n \t/*\ndiff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h\nindex b727ab1e..d7d48006 100644\n--- a/src/ipa/ipu3/algorithms/tone_mapping.h\n+++ b/src/ipa/ipu3/algorithms/tone_mapping.h\n@@ -20,7 +20,8 @@ public:\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n-\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tdouble gamma_;\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex 3b4fc911..16e5028f 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -576,7 +576,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame,\n \tControlList ctrls(controls::controls);\n \n \tfor (auto const &algo : algorithms_)\n-\t\talgo->process(context_, stats);\n+\t\talgo->process(context_, nullptr, stats);\n \n \tsetControls(frame);\n \ndiff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp\nindex 398d5372..cce2ed62 100644\n--- a/src/ipa/libipa/algorithm.cpp\n+++ b/src/ipa/libipa/algorithm.cpp\n@@ -64,6 +64,7 @@ namespace ipa {\n * \\fn Algorithm::process()\n * \\brief Process ISP statistics, and run algorithm operations\n * \\param[in] context The shared IPA context\n+ * \\param[in] frameContext The current frame's context\n * \\param[in] stats The IPA statistics and ISP results\n *\n * This function is called while camera is running for every frame processed by\ndiff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h\nindex 766aee5d..032a05b5 100644\n--- a/src/ipa/libipa/algorithm.h\n+++ b/src/ipa/libipa/algorithm.h\n@@ -10,7 +10,8 @@ namespace libcamera {\n \n namespace ipa {\n \n-template\n+template\n class Algorithm\n {\n public:\n@@ -28,6 +29,7 @@ public:\n \t}\n \n \tvirtual void process([[maybe_unused]] Context &context,\n+\t\t\t [[maybe_unused]] FrameContext *frameContext,\n \t\t\t [[maybe_unused]] const Stats *stats)\n \t{\n \t}\ndiff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex 5f4c3f93..b5a184d9 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -280,7 +280,9 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const\n * Identify the current image brightness, and use that to estimate the optimal\n * new exposure and gain for the scene.\n */\n-void Agc::process(IPAContext &context, const rkisp1_stat_buffer *stats)\n+void Agc::process(IPAContext &context,\n+\t\t [[maybe_unused]] IPAFrameContext *frameContext,\n+\t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\n \tASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP);\ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex ce1adf27..22c02779 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -29,7 +29,8 @@ public:\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, rkisp1_params_cfg *params) override;\n-\tvoid process(IPAContext &context, const rkisp1_stat_buffer *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\t\t const rkisp1_stat_buffer *stats) override;\n \n private:\n \tvoid computeExposure(IPAContext &Context, double yGain, double iqMeanGain);\ndiff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h\nindex d46c3188..68e3a44e 100644\n--- a/src/ipa/rkisp1/algorithms/algorithm.h\n+++ b/src/ipa/rkisp1/algorithms/algorithm.h\n@@ -19,7 +19,9 @@ namespace libcamera {\n \n namespace ipa::rkisp1 {\n \n-using Algorithm = libcamera::ipa::Algorithm;\n+using Algorithm = libcamera::ipa::Algorithm;\n \n } /* namespace ipa::rkisp1 */\n \ndiff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\nindex be4585c6..88441382 100644\n--- a/src/ipa/rkisp1/algorithms/awb.cpp\n+++ b/src/ipa/rkisp1/algorithms/awb.cpp\n@@ -119,7 +119,9 @@ void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params)\n /**\n * \\copydoc libcamera::ipa::Algorithm::process\n */\n-void Awb::process([[maybe_unused]] IPAContext &context, const rkisp1_stat_buffer *stats)\n+void Awb::process([[maybe_unused]] IPAContext &context,\n+\t\t [[maybe_unused]] IPAFrameContext *frameCtx,\n+\t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\n \tconst rkisp1_cif_isp_awb_stat *awb = ¶ms->awb;\ndiff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h\nindex 11946643..7647842f 100644\n--- a/src/ipa/rkisp1/algorithms/awb.h\n+++ b/src/ipa/rkisp1/algorithms/awb.h\n@@ -23,7 +23,8 @@ public:\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, rkisp1_params_cfg *params) override;\n-\tvoid process(IPAContext &context, const rkisp1_stat_buffer *stats) override;\n+\tvoid process(IPAContext &context, IPAFrameContext *frameCtx,\n+\t\t const rkisp1_stat_buffer *stats) override;\n \n private:\n \tuint32_t estimateCCT(double red, double green, double blue);\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex ef1f0d56..c818a6d7 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -272,7 +272,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId\n \tunsigned int aeState = 0;\n \n \tfor (auto const &algo : algorithms_)\n-\t\talgo->process(context_, stats);\n+\t\talgo->process(context_, nullptr, stats);\n \n \tsetControls(frame);\n \n","prefixes":["libcamera-devel","v4","2/3"]}