{"id":16141,"url":"https://patchwork.libcamera.org/api/1.1/patches/16141/?format=json","web_url":"https://patchwork.libcamera.org/patch/16141/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220603132259.188845-5-umang.jain@ideasonboard.com>","date":"2022-06-03T13:22:59","name":"[libcamera-devel,v1,4/4] ipa: algorithm: process() should take in frame number instead of context","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"643172c20060499aa6d347320c59deada6282288","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/1.1/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/16141/mbox/","series":[{"id":3157,"url":"https://patchwork.libcamera.org/api/1.1/series/3157/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3157","date":"2022-06-03T13:22:55","name":"Move frame contexts queue to separate class","version":1,"mbox":"https://patchwork.libcamera.org/series/3157/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/16141/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/16141/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CA62FBD160\n\tfor ;\n\tFri, 3 Jun 2022 13:23:11 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 7C35465643;\n\tFri, 3 Jun 2022 15:23:11 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 011506040E\n\tfor ;\n\tFri, 3 Jun 2022 15:23:09 +0200 (CEST)","from perceval.ideasonboard.com\n\t(static-127-186-62-95.ipcom.comunitel.net [95.62.186.127])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 487EA10DD;\n\tFri, 3 Jun 2022 15:23:08 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1654262591;\n\tbh=puTHWg7/2ymLbpmeqoS7ZDesw3mfgzg4V9arUFG5dn4=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=YaSqfbK9cHIVLzX7sTXgMdt0+yHNSOBsyqYiZsH5XxCOH50IJ+opzww/qHObuqcwG\n\tldK+iaJ1H+dQgWm/nQYOLYTbt+VzaDvtYFfg3Ws/j+R+J9ReANsX62U3S+bkBceQAM\n\tQ5X/jvFDF77dd60MaWLRWvp3Q5kxnzOo3AaOPF7LHD5lOND+GdEuRzlKB6KLiJCxSb\n\t88YvCi2Cq6etVN2ydu3w2QNDYFLIQOTahvAXuaNkzxwKoc7mwIybnVugh10MJFZeWg\n\tt4WXb8vyf7YRj3/5l2Oz0aP3Qo2CU5omyNY2icEcWUCPgSbIenygZgeLXKM7F08i8t\n\tOPZcl9h94XkSw==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1654262588;\n\tbh=puTHWg7/2ymLbpmeqoS7ZDesw3mfgzg4V9arUFG5dn4=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=Si+xXqDfX1rSKTxSEMGSCpXocIgXES0fJ60z/AWzL82KWjWxui4i2S05knrfkLj7n\n\tY9DXuKgz/vzY5Msabaejfl2YwWNVGcHELgHKYI8EUgqfbQt5ZsOuTI5so4nNp4c18y\n\tdeevYSUH3xThv/VKo7D2xdJQ9khCeNHK5NVksTXY="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"Si+xXqDf\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 3 Jun 2022 15:22:59 +0200","Message-Id":"<20220603132259.188845-5-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","In-Reply-To":"<20220603132259.188845-1-umang.jain@ideasonboard.com>","References":"<20220603132259.188845-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v1 4/4] ipa: algorithm: process() should\n\ttake in frame number instead of context","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Umang Jain via libcamera-devel ","Reply-To":"Umang Jain ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This will enable algorithms to extract a frame context from the FCQueue\nusing the frame number. This is required because algorithms can operate\non different frames at different points in time (according to frame latency\nthey need to respect).\n\nSigned-off-by: Umang Jain \nReviewed-by: Paul Elder \nReviewed-by: Jean-Michel Hautbois \n---\n src/ipa/ipu3/algorithms/af.cpp | 4 ++--\n src/ipa/ipu3/algorithms/af.h | 2 +-\n src/ipa/ipu3/algorithms/agc.cpp | 13 +++++++------\n src/ipa/ipu3/algorithms/agc.h | 4 ++--\n src/ipa/ipu3/algorithms/algorithm.h | 4 ++--\n src/ipa/ipu3/algorithms/awb.cpp | 2 +-\n src/ipa/ipu3/algorithms/awb.h | 2 +-\n src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 ++--\n src/ipa/ipu3/algorithms/tone_mapping.h | 2 +-\n src/ipa/ipu3/ipu3.cpp | 2 +-\n src/ipa/libipa/algorithm.cpp | 2 +-\n src/ipa/libipa/algorithm.h | 6 +++---\n src/ipa/rkisp1/algorithms/agc.cpp | 3 +--\n src/ipa/rkisp1/algorithms/agc.h | 2 +-\n src/ipa/rkisp1/algorithms/algorithm.h | 5 ++---\n src/ipa/rkisp1/algorithms/awb.cpp | 2 +-\n src/ipa/rkisp1/algorithms/awb.h | 2 +-\n src/ipa/rkisp1/rkisp1.cpp | 2 +-\n 18 files changed, 31 insertions(+), 32 deletions(-)","diff":"diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp\nindex d07521a0..228c2e81 100644\n--- a/src/ipa/ipu3/algorithms/af.cpp\n+++ b/src/ipa/ipu3/algorithms/af.cpp\n@@ -406,7 +406,7 @@ bool Af::afIsOutOfFocus(IPAContext context)\n /**\n * \\brief Determine the max contrast image and lens position.\n * \\param[in] context The IPA context.\n- * \\param[in] frameContext The current frame context\n+ * \\param[in] frame The frame number in consideration\n * \\param[in] stats The statistics buffer of IPU3.\n *\n * Ideally, a clear image also has a relatively higher contrast. So, every\n@@ -420,7 +420,7 @@ bool Af::afIsOutOfFocus(IPAContext context)\n *\n * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing\n */\n-void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+void Af::process(IPAContext &context, [[maybe_unused]] uint32_t frame,\n \t\t const ipu3_uapi_stats_3a *stats)\n {\n \t/* Evaluate the AF buffer length */\ndiff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h\nindex ccf015f3..69aeb3e2 100644\n--- a/src/ipa/ipu3/algorithms/af.h\n+++ b/src/ipa/ipu3/algorithms/af.h\n@@ -32,7 +32,7 @@ public:\n \n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n-\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\tvoid process(IPAContext &context, uint32_t frame,\n \t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\ndiff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\nindex f16be534..3dc11fe2 100644\n--- a/src/ipa/ipu3/algorithms/agc.cpp\n+++ b/src/ipa/ipu3/algorithms/agc.cpp\n@@ -180,14 +180,16 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue)\n /**\n * \\brief Estimate the new exposure and gain values\n * \\param[inout] frameContext The shared IPA frame Context\n+ * \\param [in] frame The frame number in consideration\n * \\param[in] yGain The gain calculated based on the relative luminance target\n * \\param[in] iqMeanGain The gain calculated based on the relative luminance target\n */\n-void Agc::computeExposure(IPAContext &context, IPAFrameContext *frameContext,\n-\t\t\t double yGain, double iqMeanGain)\n+void Agc::computeExposure(IPAContext &context, uint32_t frame, double yGain,\n+\t\t\t double iqMeanGain)\n {\n \tconst IPASessionConfiguration &configuration = context.configuration;\n \t/* Get the effective exposure and gain applied on the sensor. */\n+\tIPAFrameContext *frameContext = context.frameContexts.get(frame);\n \tuint32_t exposure = frameContext->sensor.exposure;\n \tdouble analogueGain = frameContext->sensor.gain;\n \n@@ -317,14 +319,13 @@ double Agc::estimateLuminance(IPAActiveState &activeState,\n /**\n * \\brief Process IPU3 statistics, and run AGC operations\n * \\param[in] context The shared IPA context\n- * \\param[in] frameContext The current frame context\n+ * \\param[in] frame The frame number in consideration\n * \\param[in] stats The IPU3 statistics and ISP results\n *\n * Identify the current image brightness, and use that to estimate the optimal\n * new exposure and gain for the scene.\n */\n-void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n-\t\t const ipu3_uapi_stats_3a *stats)\n+void Agc::process(IPAContext &context, uint32_t frame, const ipu3_uapi_stats_3a *stats)\n {\n \t/*\n \t * Estimate the gain needed to have the proportion of pixels in a given\n@@ -359,7 +360,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameCo\n \t\t\tbreak;\n \t}\n \n-\tcomputeExposure(context, frameContext, yGain, iqMeanGain);\n+\tcomputeExposure(context, frame, yGain, iqMeanGain);\n \tframeCount_++;\n }\n \ndiff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h\nindex 105ae0f2..8ef2a2d8 100644\n--- a/src/ipa/ipu3/algorithms/agc.h\n+++ b/src/ipa/ipu3/algorithms/agc.h\n@@ -28,14 +28,14 @@ public:\n \t~Agc() = default;\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n-\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\tvoid process(IPAContext &context, uint32_t frame,\n \t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\n \tdouble measureBrightness(const ipu3_uapi_stats_3a *stats,\n \t\t\t\t const ipu3_uapi_grid_config &grid) const;\n \tutils::Duration filterExposure(utils::Duration currentExposure);\n-\tvoid computeExposure(IPAContext &context, IPAFrameContext *frameContext,\n+\tvoid computeExposure(IPAContext &context, uint32_t frame,\n \t\t\t double yGain, double iqMeanGain);\n \tdouble estimateLuminance(IPAActiveState &activeState,\n \t\t\t\t const ipu3_uapi_grid_config &grid,\ndiff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h\nindex 234b2bd7..3e0c60d3 100644\n--- a/src/ipa/ipu3/algorithms/algorithm.h\n+++ b/src/ipa/ipu3/algorithms/algorithm.h\n@@ -17,8 +17,8 @@ namespace libcamera {\n \n namespace ipa::ipu3 {\n \n-using Algorithm = libcamera::ipa::Algorithm;\n \n } /* namespace ipa::ipu3 */\ndiff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp\nindex 5c232d92..35f35ebe 100644\n--- a/src/ipa/ipu3/algorithms/awb.cpp\n+++ b/src/ipa/ipu3/algorithms/awb.cpp\n@@ -387,7 +387,7 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats)\n /**\n * \\copydoc libcamera::ipa::Algorithm::process\n */\n-void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+void Awb::process(IPAContext &context, [[maybe_unused]] uint32_t frame,\n \t\t const ipu3_uapi_stats_3a *stats)\n {\n \tcalculateWBGains(stats);\ndiff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h\nindex 9a50a985..f263540c 100644\n--- a/src/ipa/ipu3/algorithms/awb.h\n+++ b/src/ipa/ipu3/algorithms/awb.h\n@@ -40,7 +40,7 @@ public:\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n-\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\tvoid process(IPAContext &context, uint32_t frame,\n \t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\ndiff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp\nindex f86e79b2..6e4024cd 100644\n--- a/src/ipa/ipu3/algorithms/tone_mapping.cpp\n+++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp\n@@ -72,13 +72,13 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context,\n /**\n * \\brief Calculate the tone mapping look up table\n * \\param context The shared IPA context\n- * \\param frameContext The current frame context\n+ * \\param frame The frame number in consideration\n * \\param stats The IPU3 statistics and ISP results\n *\n * The tone mapping look up table is generated as an inverse power curve from\n * our gamma setting.\n */\n-void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext,\n+void ToneMapping::process(IPAContext &context, [[maybe_unused]] uint32_t frame,\n \t\t\t [[maybe_unused]] const ipu3_uapi_stats_3a *stats)\n {\n \t/*\ndiff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h\nindex d7d48006..61474085 100644\n--- a/src/ipa/ipu3/algorithms/tone_mapping.h\n+++ b/src/ipa/ipu3/algorithms/tone_mapping.h\n@@ -20,7 +20,7 @@ public:\n \n \tint configure(IPAContext &context, const IPAConfigInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, ipu3_uapi_params *params) override;\n-\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\tvoid process(IPAContext &context, uint32_t frame,\n \t\t const ipu3_uapi_stats_3a *stats) override;\n \n private:\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex 63aa9b56..2c811b84 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -582,7 +582,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame,\n \tControlList ctrls(controls::controls);\n \n \tfor (auto const &algo : algorithms_)\n-\t\talgo->process(context_, frameContext, stats);\n+\t\talgo->process(context_, frame, stats);\n \n \tsetControls(frame);\n \ndiff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp\nindex cce2ed62..1b318b5d 100644\n--- a/src/ipa/libipa/algorithm.cpp\n+++ b/src/ipa/libipa/algorithm.cpp\n@@ -64,7 +64,7 @@ namespace ipa {\n * \\fn Algorithm::process()\n * \\brief Process ISP statistics, and run algorithm operations\n * \\param[in] context The shared IPA context\n- * \\param[in] frameContext The current frame's context\n+ * \\param[in] frame The frame number in consideration\n * \\param[in] stats The IPA statistics and ISP results\n *\n * This function is called while camera is running for every frame processed by\ndiff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h\nindex 032a05b5..d1ca0883 100644\n--- a/src/ipa/libipa/algorithm.h\n+++ b/src/ipa/libipa/algorithm.h\n@@ -10,8 +10,8 @@ namespace libcamera {\n \n namespace ipa {\n \n-template\n+template\n+\n class Algorithm\n {\n public:\n@@ -29,7 +29,7 @@ public:\n \t}\n \n \tvirtual void process([[maybe_unused]] Context &context,\n-\t\t\t [[maybe_unused]] FrameContext *frameContext,\n+\t\t\t [[maybe_unused]] uint32_t frame,\n \t\t\t [[maybe_unused]] const Stats *stats)\n \t{\n \t}\ndiff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp\nindex b5a184d9..9418b533 100644\n--- a/src/ipa/rkisp1/algorithms/agc.cpp\n+++ b/src/ipa/rkisp1/algorithms/agc.cpp\n@@ -280,8 +280,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const\n * Identify the current image brightness, and use that to estimate the optimal\n * new exposure and gain for the scene.\n */\n-void Agc::process(IPAContext &context,\n-\t\t [[maybe_unused]] IPAFrameContext *frameContext,\n+void Agc::process(IPAContext &context, [[maybe_unused]] uint32_t frame,\n \t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\ndiff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h\nindex 22c02779..a749f1c3 100644\n--- a/src/ipa/rkisp1/algorithms/agc.h\n+++ b/src/ipa/rkisp1/algorithms/agc.h\n@@ -29,7 +29,7 @@ public:\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, rkisp1_params_cfg *params) override;\n-\tvoid process(IPAContext &context, IPAFrameContext *frameContext,\n+\tvoid process(IPAContext &context, uint32_t frame,\n \t\t const rkisp1_stat_buffer *stats) override;\n \n private:\ndiff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h\nindex 68e3a44e..069c744c 100644\n--- a/src/ipa/rkisp1/algorithms/algorithm.h\n+++ b/src/ipa/rkisp1/algorithms/algorithm.h\n@@ -19,9 +19,8 @@ namespace libcamera {\n \n namespace ipa::rkisp1 {\n \n-using Algorithm = libcamera::ipa::Algorithm;\n+using Algorithm = libcamera::ipa::Algorithm;\n \n } /* namespace ipa::rkisp1 */\n \ndiff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\nindex 88441382..1e7fd11a 100644\n--- a/src/ipa/rkisp1/algorithms/awb.cpp\n+++ b/src/ipa/rkisp1/algorithms/awb.cpp\n@@ -120,7 +120,7 @@ void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params)\n * \\copydoc libcamera::ipa::Algorithm::process\n */\n void Awb::process([[maybe_unused]] IPAContext &context,\n-\t\t [[maybe_unused]] IPAFrameContext *frameCtx,\n+\t\t [[maybe_unused]] uint32_t frame,\n \t\t const rkisp1_stat_buffer *stats)\n {\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\ndiff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h\nindex 7647842f..82921f34 100644\n--- a/src/ipa/rkisp1/algorithms/awb.h\n+++ b/src/ipa/rkisp1/algorithms/awb.h\n@@ -23,7 +23,7 @@ public:\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n \tvoid prepare(IPAContext &context, rkisp1_params_cfg *params) override;\n-\tvoid process(IPAContext &context, IPAFrameContext *frameCtx,\n+\tvoid process(IPAContext &context, uint32_t frame,\n \t\t const rkisp1_stat_buffer *stats) override;\n \n private:\ndiff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp\nindex 7d6f0b39..33a10dab 100644\n--- a/src/ipa/rkisp1/rkisp1.cpp\n+++ b/src/ipa/rkisp1/rkisp1.cpp\n@@ -272,7 +272,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId\n \tunsigned int aeState = 0;\n \n \tfor (auto const &algo : algorithms_)\n-\t\talgo->process(context_, nullptr, stats);\n+\t\talgo->process(context_, frame, stats);\n \n \tsetControls(frame);\n \n","prefixes":["libcamera-devel","v1","4/4"]}