{"id":21855,"url":"https://patchwork.libcamera.org/api/patches/21855/?format=json","web_url":"https://patchwork.libcamera.org/patch/21855/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241107114819.57599-11-dan.scally@ideasonboard.com>","date":"2024-11-07T11:48:18","name":"[v3,10/11] ipa: mali-c55: Add Lens Shading Correction algorithm","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"1121d632e147a80e79591946a1b231e0faa45974","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/people/156/?format=json","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21855/mbox/","series":[{"id":4777,"url":"https://patchwork.libcamera.org/api/series/4777/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4777","date":"2024-11-07T11:48:08","name":"Add Mali-C55 IPA Module and Algorithms","version":3,"mbox":"https://patchwork.libcamera.org/series/4777/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21855/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21855/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9FB99C323E\n\tfor ;\n\tThu, 7 Nov 2024 11:48:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id ECA70654A4;\n\tThu, 7 Nov 2024 12:48:40 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B7B4765482\n\tfor ;\n\tThu, 7 Nov 2024 12:48:30 +0100 (CET)","from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B0074A47;\n\tThu, 7 Nov 2024 12:48:21 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"fPB/4jt3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1730980102;\n\tbh=Z2P9JHer368QYSn91XPFxtoU1FLE2Fw+Ti4DuwaryWs=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=fPB/4jt3WAiLpuV0bteuWTGbZ7tUiNJEJ9nUYYtYDsj8SdFfmrSIsZ4evXI0agnc9\n\troNYk577G0EsjU6jOJjK/bVykdpR9GNKftwzQqrC41g3ZPEfjcG+OWWpprLCNdFdms\n\tMya8fRjzGynyFtRQXo7iUlv2nFujuYUuoPAkoJ+U=","From":"Daniel Scally ","To":"libcamera-devel@lists.libcamera.org","Cc":"Anthony.McGivern@arm.com, Daniel Scally ,\n\tKieran Bingham ,\n\tNayden Kanchev ,\n\tJacopo Mondi ","Subject":"[PATCH v3 10/11] ipa: mali-c55: Add Lens Shading Correction\n\talgorithm","Date":"Thu, 7 Nov 2024 11:48:18 +0000","Message-Id":"<20241107114819.57599-11-dan.scally@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20241107114819.57599-1-dan.scally@ideasonboard.com>","References":"<20241107114819.57599-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Add a lens shading correction algorithm to the mali-c55 IPA. This\nalgorithm parses tables from Yaml in a easy to follow format before\nmunging them into Arm's interleaved mesh to be copied to the ISP.\nA colour temperature estimate from the AGC statistics is used to\nselect the appropriate table to apply; this can be some interpolation\nof two tables, in which case the colour temperature estimate is also\nused to derive the coefficient that does the blending.\n\nReviewed-by: Kieran Bingham \nAcked-by: Nayden Kanchev \nCo-developed-by: Jacopo Mondi \nSigned-off-by: Jacopo Mondi \nSigned-off-by: Daniel Scally \n---\nChanges in v3:\n\n\t- Added a comment explaining the setting of the mesh size variable\n\nChanges in v2:\n\n\t- Use the union rather than reinterpret_cast<>() to abstract the block\n\n src/ipa/mali-c55/algorithms/lsc.cpp | 216 ++++++++++++++++++++++++\n src/ipa/mali-c55/algorithms/lsc.h | 45 +++++\n src/ipa/mali-c55/algorithms/meson.build | 1 +\n 3 files changed, 262 insertions(+)\n create mode 100644 src/ipa/mali-c55/algorithms/lsc.cpp\n create mode 100644 src/ipa/mali-c55/algorithms/lsc.h","diff":"diff --git a/src/ipa/mali-c55/algorithms/lsc.cpp b/src/ipa/mali-c55/algorithms/lsc.cpp\nnew file mode 100644\nindex 00000000..c5afc04d\n--- /dev/null\n+++ b/src/ipa/mali-c55/algorithms/lsc.cpp\n@@ -0,0 +1,216 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Ideas On Board Oy\n+ *\n+ * lsc.cpp - Mali-C55 Lens shading correction algorithm\n+ */\n+\n+#include \"lsc.h\"\n+\n+#include \"libcamera/internal/yaml_parser.h\"\n+\n+namespace libcamera {\n+\n+namespace ipa::mali_c55::algorithms {\n+\n+LOG_DEFINE_CATEGORY(MaliC55Lsc)\n+\n+int Lsc::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData)\n+{\n+\tif (!tuningData.contains(\"meshScale\")) {\n+\t\tLOG(MaliC55Lsc, Error) << \"meshScale missing from tuningData\";\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tmeshScale_ = tuningData[\"meshScale\"].get(0);\n+\n+\tconst YamlObject &yamlSets = tuningData[\"sets\"];\n+\tif (!yamlSets.isList()) {\n+\t\tLOG(MaliC55Lsc, Error) << \"LSC tables missing or invalid\";\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tsize_t tableSize = 0;\n+\tconst auto &sets = yamlSets.asList();\n+\tfor (const auto &yamlSet : sets) {\n+\t\tuint32_t ct = yamlSet[\"ct\"].get(0);\n+\n+\t\tif (!ct) {\n+\t\t\tLOG(MaliC55Lsc, Error) << \"Invalid colour temperature\";\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tif (std::count(colourTemperatures_.begin(),\n+\t\t\t colourTemperatures_.end(), ct)) {\n+\t\t\tLOG(MaliC55Lsc, Error)\n+\t\t\t\t<< \"Multiple sets found for colour temperature\";\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tstd::vector rTable =\n+\t\t\tyamlSet[\"r\"].getList().value_or(std::vector{});\n+\t\tstd::vector gTable =\n+\t\t\tyamlSet[\"g\"].getList().value_or(std::vector{});\n+\t\tstd::vector bTable =\n+\t\t\tyamlSet[\"b\"].getList().value_or(std::vector{});\n+\n+\t\t/*\n+\t\t * Some validation to do; only 16x16 and 32x32 tables of\n+\t\t * coefficients are acceptable, and all tables across all of the\n+\t\t * sets must be the same size. The first time we encounter a\n+\t\t * table we check that it is an acceptable size and if so make\n+\t\t * sure all other tables are of equal size.\n+\t\t */\n+\t\tif (!tableSize) {\n+\t\t\tif (rTable.size() != 256 && rTable.size() != 1024) {\n+\t\t\t\tLOG(MaliC55Lsc, Error)\n+\t\t\t\t\t<< \"Invalid table size for colour temperature \" << ct;\n+\t\t\t\treturn -EINVAL;\n+\t\t\t}\n+\t\t\ttableSize = rTable.size();\n+\t\t}\n+\n+\t\tif (rTable.size() != tableSize ||\n+\t\t gTable.size() != tableSize ||\n+\t\t bTable.size() != tableSize) {\n+\t\t\tLOG(MaliC55Lsc, Error)\n+\t\t\t\t<< \"Invalid or mismatched table size for colour temperature \" << ct;\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tif (colourTemperatures_.size() >= 3) {\n+\t\t\tLOG(MaliC55Lsc, Error)\n+\t\t\t\t<< \"A maximum of 3 colour temperatures are supported\";\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tfor (unsigned int i = 0; i < tableSize; i++) {\n+\t\t\tmesh_[kRedOffset + i] |=\n+\t\t\t\t(rTable[i] << (colourTemperatures_.size() * 8));\n+\t\t\tmesh_[kGreenOffset + i] |=\n+\t\t\t\t(gTable[i] << (colourTemperatures_.size() * 8));\n+\t\t\tmesh_[kBlueOffset + i] |=\n+\t\t\t\t(bTable[i] << (colourTemperatures_.size() * 8));\n+\t\t}\n+\n+\t\tcolourTemperatures_.push_back(ct);\n+\t}\n+\n+\t/*\n+\t * The mesh has either 16x16 or 32x32 nodes, we tell the driver which it\n+\t * is based on the number of values in the tuning data's table.\n+\t */\n+\tif (tableSize == 256)\n+\t\tmeshSize_ = 15;\n+\telse\n+\t\tmeshSize_ = 31;\n+\n+\treturn 0;\n+}\n+\n+size_t Lsc::fillConfigParamsBlock(mali_c55_params_block block) const\n+{\n+\tblock.header->type = MALI_C55_PARAM_MESH_SHADING_CONFIG;\n+\tblock.header->flags = MALI_C55_PARAM_BLOCK_FL_NONE;\n+\tblock.header->size = sizeof(struct mali_c55_params_mesh_shading_config);\n+\n+\tblock.shading_config->mesh_show = false;\n+\tblock.shading_config->mesh_scale = meshScale_;\n+\tblock.shading_config->mesh_page_r = 0;\n+\tblock.shading_config->mesh_page_g = 1;\n+\tblock.shading_config->mesh_page_b = 2;\n+\tblock.shading_config->mesh_width = meshSize_;\n+\tblock.shading_config->mesh_height = meshSize_;\n+\n+\tstd::copy(mesh_.begin(), mesh_.end(), block.shading_config->mesh);\n+\n+\treturn block.header->size;\n+}\n+\n+size_t Lsc::fillSelectionParamsBlock(mali_c55_params_block block, uint8_t bank,\n+\t\t\t\t uint8_t alpha) const\n+{\n+\tblock.header->type = MALI_C55_PARAM_MESH_SHADING_SELECTION;\n+\tblock.header->flags = MALI_C55_PARAM_BLOCK_FL_NONE;\n+\tblock.header->size = sizeof(struct mali_c55_params_mesh_shading_selection);\n+\n+\tblock.shading_selection->mesh_alpha_bank_r = bank;\n+\tblock.shading_selection->mesh_alpha_bank_g = bank;\n+\tblock.shading_selection->mesh_alpha_bank_b = bank;\n+\tblock.shading_selection->mesh_alpha_r = alpha;\n+\tblock.shading_selection->mesh_alpha_g = alpha;\n+\tblock.shading_selection->mesh_alpha_b = alpha;\n+\tblock.shading_selection->mesh_strength = 0x1000; /* Otherwise known as 1.0 */\n+\n+\treturn block.header->size;\n+}\n+\n+std::tuple Lsc::findBankAndAlpha(uint32_t ct) const\n+{\n+\tunsigned int i;\n+\n+\tct = std::clamp(ct, colourTemperatures_.front(),\n+\t\t\t\t colourTemperatures_.back());\n+\n+\tfor (i = 0; i < colourTemperatures_.size() - 1; i++) {\n+\t\tif (ct >= colourTemperatures_[i] &&\n+\t\t ct <= colourTemperatures_[i + 1])\n+\t\t\tbreak;\n+\t}\n+\n+\t/*\n+\t * With the clamping, we're guaranteed an index into colourTemperatures_\n+\t * that's <= colourTemperatures_.size() - 1.\n+\t */\n+\tuint8_t alpha = (255 * (ct - colourTemperatures_[i])) /\n+\t\t\t(colourTemperatures_[i + 1] - colourTemperatures_[i]);\n+\n+\treturn { i, alpha };\n+}\n+\n+void Lsc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame,\n+\t\t [[maybe_unused]] IPAFrameContext &frameContext,\n+\t\t mali_c55_params_buffer *params)\n+{\n+\t/*\n+\t * For each frame we assess the colour temperature of the **last** frame\n+\t * and then select an appropriately blended table of coefficients based\n+\t * on that ct. As a bit of a shortcut, if we've only a single table the\n+\t * handling is somewhat simpler; if it's the first frame we just select\n+\t * that table and if we're past the first frame then we can just do\n+\t * nothing - the config will never change.\n+\t */\n+\tuint32_t temperatureK = context.activeState.agc.temperatureK;\n+\tuint8_t bank, alpha;\n+\n+\tif (colourTemperatures_.size() == 1) {\n+\t\tif (frame > 0)\n+\t\t\treturn;\n+\n+\t\tbank = 0;\n+\t\talpha = 0;\n+\t} else {\n+\t\tstd::tie(bank, alpha) = findBankAndAlpha(temperatureK);\n+\t}\n+\n+\tmali_c55_params_block block;\n+\tblock.data = ¶ms->data[params->total_size];\n+\n+\tparams->total_size += fillSelectionParamsBlock(block, bank, alpha);\n+\n+\tif (frame > 0)\n+\t\treturn;\n+\n+\t/*\n+\t * If this is the first frame, we need to load the parsed coefficient\n+\t * tables from tuning data to the ISP.\n+\t */\n+\tblock.data = ¶ms->data[params->total_size];\n+\tparams->total_size += fillConfigParamsBlock(block);\n+}\n+\n+REGISTER_IPA_ALGORITHM(Lsc, \"Lsc\")\n+\n+} /* namespace ipa::mali_c55::algorithms */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/mali-c55/algorithms/lsc.h b/src/ipa/mali-c55/algorithms/lsc.h\nnew file mode 100644\nindex 00000000..e613277a\n--- /dev/null\n+++ b/src/ipa/mali-c55/algorithms/lsc.h\n@@ -0,0 +1,45 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2024, Ideas On Board Oy\n+ *\n+ * lsc.h - Mali-C55 Lens shading correction algorithm\n+ */\n+\n+#include \n+#include \n+\n+#include \"algorithm.h\"\n+\n+namespace libcamera {\n+\n+namespace ipa::mali_c55::algorithms {\n+\n+class Lsc : public Algorithm\n+{\n+public:\n+\tLsc() = default;\n+\t~Lsc() = default;\n+\n+\tint init(IPAContext &context, const YamlObject &tuningData) override;\n+\tvoid prepare(IPAContext &context, const uint32_t frame,\n+\t\t IPAFrameContext &frameContext,\n+\t\t mali_c55_params_buffer *params) override;\n+private:\n+\tstatic constexpr unsigned int kRedOffset = 0;\n+\tstatic constexpr unsigned int kGreenOffset = 1024;\n+\tstatic constexpr unsigned int kBlueOffset = 2048;\n+\n+\tsize_t fillConfigParamsBlock(mali_c55_params_block block) const;\n+\tsize_t fillSelectionParamsBlock(mali_c55_params_block block,\n+\t\t\t\t\tuint8_t bank, uint8_t alpha) const;\n+\tstd::tuple findBankAndAlpha(uint32_t ct) const;\n+\n+\tstd::vector mesh_ = std::vector(3072);\n+\tstd::vector colourTemperatures_;\n+\tuint32_t meshScale_;\n+\tuint32_t meshSize_;\n+};\n+\n+} /* namespace ipa::mali_c55::algorithms */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/mali-c55/algorithms/meson.build b/src/ipa/mali-c55/algorithms/meson.build\nindex f11791aa..1665da07 100644\n--- a/src/ipa/mali-c55/algorithms/meson.build\n+++ b/src/ipa/mali-c55/algorithms/meson.build\n@@ -4,4 +4,5 @@ mali_c55_ipa_algorithms = files([\n 'agc.cpp',\n 'awb.cpp',\n 'blc.cpp',\n+ 'lsc.cpp',\n ])\n","prefixes":["v3","10/11"]}