{"id":13391,"url":"https://patchwork.libcamera.org/api/1.1/patches/13391/?format=json","web_url":"https://patchwork.libcamera.org/patch/13391/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210818155403.268694-5-jeanmichel.hautbois@ideasonboard.com>","date":"2021-08-18T15:53:58","name":"[libcamera-devel,v3,4/9] ipa: ipu3: Introduce modular algorithm","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"76ba2c0d7736cf25b264949a0b38ae601b6b2b1c","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/1.1/people/75/?format=json","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/13391/mbox/","series":[{"id":2370,"url":"https://patchwork.libcamera.org/api/1.1/series/2370/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2370","date":"2021-08-18T15:53:54","name":"IPU3: Introduce modularity for algorithms","version":3,"mbox":"https://patchwork.libcamera.org/series/2370/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13391/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13391/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 28C93BD87C\n\tfor ;\n\tWed, 18 Aug 2021 15:54:16 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 89EB768890;\n\tWed, 18 Aug 2021 17:54:15 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DAAF7688A5\n\tfor ;\n\tWed, 18 Aug 2021 17:54:09 +0200 (CEST)","from tatooine.ideasonboard.com (unknown\n\t[IPv6:2a01:e0a:169:7140:946e:2bbb:370e:41e4])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8A1EFEE;\n\tWed, 18 Aug 2021 17:54:09 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"mslvsvhd\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1629302049;\n\tbh=y+QeKCpspQTZXZ16dntwKNgRxMQVPK309nlXoVMPIAI=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=mslvsvhdcD7dMyO+Zo8DOBsGBeOvTYEYSWSOB3JfoIFtmu9WMzvsmb6OvlRxsLEqJ\n\tA9C5x6aBtDUNOsUIbefnCZniMYJu2y60gxwm9cpXRzly/oZxcoXNnKS6Zi+DxutZGB\n\tUt4hj2RiQ/pt/JOZrvVtxs2/scD4sJeQszwxw2Eg=","From":"Jean-Michel Hautbois ","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 18 Aug 2021 17:53:58 +0200","Message-Id":"<20210818155403.268694-5-jeanmichel.hautbois@ideasonboard.com>","X-Mailer":"git-send-email 2.30.2","In-Reply-To":"<20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com>","References":"<20210818155403.268694-1-jeanmichel.hautbois@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 4/9] ipa: ipu3: Introduce modular\n\talgorithm","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Implement a new modular framework for algorithms with a common context\nstructure that is passed to each algorithm through a common API.\n\nThis patch:\n- removes all the local references from IPAIPU3 and uses IPAContext\n- implements the list of pointers and the loop at configure call on each\n algorithm\n- loops in fillParams on each prepare() call on the algorithm list\n- loops in prepareStats on each process() call on the algorithm list\n\nSigned-off-by: Jean-Michel Hautbois \n---\n src/ipa/ipu3/ipa_context.h | 27 +++--------\n src/ipa/ipu3/ipu3.cpp | 94 +++++++++++++++++++++++++++++++++-----\n 2 files changed, 89 insertions(+), 32 deletions(-)","diff":"diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h\nindex 76534f7f..46291d9e 100644\n--- a/src/ipa/ipu3/ipa_context.h\n+++ b/src/ipa/ipu3/ipa_context.h\n@@ -10,37 +10,22 @@\n \n #include \n \n+#include \n+\n namespace libcamera {\n \n namespace ipa::ipu3 {\n \n-/**\n- * Fixed configuration of the IPA\n- *\n- * This structure contains data set at the configure state of the IPA.\n- * It can't be modified while streaming.\n- */\n struct IPAConfiguration {\n+\tstruct Grid {\n+\t\tipu3_uapi_grid_config bdsGrid;\n+\t\tSize bdsOutputSize;\n+\t} grid;\n };\n \n-/**\n- * Context of a frame for each algorithms\n- *\n- * This may be stored in a way that is associated with a given request\n- * lifetime, though for now a single instance is used.\n- * It contains informations updated while streaming by the algorithms.\n- */\n struct IPAFrameContext {\n };\n \n-/**\n- * Context information shared between the algorithms\n- *\n- * The global context structure contains:\n- * - a configuration structure set while the IPA is configured, and not\n- * modified when streaming\n- * - a frameContext structure updated by the algorithms while streaming\n- */\n struct IPAContext {\n \tIPAConfiguration configuration;\n \tIPAFrameContext frameContext;\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex c34fa460..dc6f0735 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -29,6 +29,7 @@\n \n #include \"libcamera/internal/mapped_framebuffer.h\"\n \n+#include \"algorithms/algorithm.h\"\n #include \"ipu3_agc.h\"\n #include \"ipu3_awb.h\"\n #include \"libipa/camera_sensor_helper.h\"\n@@ -36,6 +37,56 @@\n static constexpr uint32_t kMaxCellWidthPerSet = 160;\n static constexpr uint32_t kMaxCellHeightPerSet = 56;\n \n+/**\n+ * \\file ipa_context.h\n+ * \\brief Context information shared between the algorithms\n+ */\n+\n+/**\n+ * \\struct IPAConfiguration\n+ * \\brief Fixed configuration of the IPA\n+ *\n+ * This structure contains data set at the configure state of the IPA.\n+ * It can't be modified while streaming.\n+ */\n+\n+/**\n+ * \\struct IPAConfiguration::Grid\n+ * \\brief Grid configuration of the IPA\n+ *\n+ * This structure contains all the parameters needed to use the statistics.\n+ *\n+ */\n+\n+/**\n+ * \\var IPAConfiguration::Grid::bdsGrid\n+ * \\brief Bayer Down Scaler grid plane config used by the kernel\n+ */\n+\n+/**\n+ * \\var IPAConfiguration::Grid::bdsOutputSize\n+ * \\brief BDS output size configured by the pipeline handler\n+ */\n+\n+/**\n+ * \\struct IPAFrameContext\n+ * \\brief Context of a frame for each algorithms\n+ *\n+ * This may be stored in a way that is associated with a given request\n+ * lifetime, though for now a single instance is used.\n+ * It contains informations updated while streaming by the algorithms.\n+ */\n+\n+/**\n+ * \\struct IPAContext\n+ * \\brief Context information shared between the algorithms\n+ *\n+ * The global context structure contains:\n+ * - a configuration structure set while the IPA is configured, and not\n+ * modified when streaming\n+ * - a frameContext structure updated by the algorithms while streaming\n+ */\n+\n namespace libcamera {\n \n LOG_DEFINE_CATEGORY(IPAIPU3)\n@@ -91,10 +142,12 @@ private:\n \t/* Interface to the Camera Helper */\n \tstd::unique_ptr camHelper_;\n \n+\t/* Maintain the algorithms used by the IPA */\n+\tstd::list> algorithms_;\n+\n \t/* Local parameter storage */\n+\tstruct IPAContext context_;\n \tstruct ipu3_uapi_params params_;\n-\n-\tstruct ipu3_uapi_grid_config bdsGrid_;\n };\n \n /**\n@@ -191,7 +244,13 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize)\n \tuint32_t minError = std::numeric_limits::max();\n \tSize best;\n \tSize bestLog2;\n-\tbdsGrid_ = {};\n+\tipu3_uapi_grid_config &bdsGrid = context_.configuration.grid.bdsGrid;\n+\n+\t/* Set the BDS output size in the IPAConfiguration structure */\n+\tcontext_.configuration.grid.bdsOutputSize = bdsOutputSize;\n+\n+\tbdsGrid.x_start = 0;\n+\tbdsGrid.y_start = 0;\n \n \tfor (uint32_t widthShift = 3; widthShift <= 7; ++widthShift) {\n \t\tuint32_t width = std::min(kMaxCellWidthPerSet,\n@@ -215,14 +274,14 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize)\n \t\t}\n \t}\n \n-\tbdsGrid_.width = best.width >> bestLog2.width;\n-\tbdsGrid_.block_width_log2 = bestLog2.width;\n-\tbdsGrid_.height = best.height >> bestLog2.height;\n-\tbdsGrid_.block_height_log2 = bestLog2.height;\n+\tbdsGrid.width = best.width >> bestLog2.width;\n+\tbdsGrid.block_width_log2 = bestLog2.width;\n+\tbdsGrid.height = best.height >> bestLog2.height;\n+\tbdsGrid.block_height_log2 = bestLog2.height;\n \n \tLOG(IPAIPU3, Debug) << \"Best grid found is: (\"\n-\t\t\t << (int)bdsGrid_.width << \" << \" << (int)bdsGrid_.block_width_log2 << \") x (\"\n-\t\t\t << (int)bdsGrid_.height << \" << \" << (int)bdsGrid_.block_height_log2 << \")\";\n+\t\t\t << (int)bdsGrid.width << \" << \" << (int)bdsGrid.block_width_log2 << \") x (\"\n+\t\t\t << (int)bdsGrid.height << \" << \" << (int)bdsGrid.block_height_log2 << \")\";\n }\n \n int IPAIPU3::configure(const IPAConfigInfo &configInfo)\n@@ -264,15 +323,22 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo)\n \n \tdefVBlank_ = itVBlank->second.def().get();\n \n+\t/* Clean context and IPU3 parameters at configuration */\n \tparams_ = {};\n+\tcontext_ = {};\n \n \tcalculateBdsGrid(configInfo.bdsOutputSize);\n+\tfor (auto const &algo : algorithms_) {\n+\t\tint ret = algo->configure(context_, configInfo);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n \n \tawbAlgo_ = std::make_unique();\n-\tawbAlgo_->initialise(params_, configInfo.bdsOutputSize, bdsGrid_);\n+\tawbAlgo_->initialise(params_, context_.configuration.grid.bdsOutputSize, context_.configuration.grid.bdsGrid);\n \n \tagcAlgo_ = std::make_unique();\n-\tagcAlgo_->initialise(bdsGrid_, sensorInfo_);\n+\tagcAlgo_->initialise(context_.configuration.grid.bdsGrid, sensorInfo_);\n \n \treturn 0;\n }\n@@ -346,6 +412,9 @@ void IPAIPU3::processControls([[maybe_unused]] unsigned int frame,\n \n void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params)\n {\n+\tfor (auto const &algo : algorithms_)\n+\t\talgo->prepare(context_, params_);\n+\n \tif (agcAlgo_->updateControls())\n \t\tawbAlgo_->updateWbParameters(params_, agcAlgo_->gamma());\n \n@@ -363,6 +432,9 @@ void IPAIPU3::parseStatistics(unsigned int frame,\n {\n \tControlList ctrls(controls::controls);\n \n+\tfor (auto const &algo : algorithms_)\n+\t\talgo->process(context_, stats);\n+\n \tdouble gain = camHelper_->gain(gain_);\n \tagcAlgo_->process(stats, exposure_, gain);\n \tgain_ = camHelper_->gainCode(gain);\n","prefixes":["libcamera-devel","v3","4/9"]}