[{"id":29834,"web_url":"https://patchwork.libcamera.org/comment/29834/","msgid":"<171811528553.1148289.13427747030866283474@ping.linuxembedded.co.uk>","date":"2024-06-11T14:14:45","subject":"Re: [PATCH v7 2/3] ipa: libipa: Add MatrixInterpolator class","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Paul Elder (2024-06-11 15:02:06)\n> Add a class to encapsulate the functionality of fetching a matrix based\n> on an integer key, and interpolating if there is no exact match. This is\n> expected to be used by both color correction matrices / crosstalk\n> correction as well as lens shading correction.\n> \n> A cache is included only for exact matches of the key. The caller is\n> expected to decide the tolererance for rounding.\n> \n> Signed-off-by: Paul Elder \n> Reviewed-by: Stefan Klug \n> \n> ---\n> Changes in v7:\n> - remove cache\n> - fix copyright and license\n\n\nReviewed-by: Kieran Bingham \n\n> \n> Changes in v6:\n> - fix doxygen\n> \n> Changes in v5:\n> - improve documentation\n> - replace count() with find() != end() (in get())\n> - add cache\n> \n> Changes in v4:\n> - remove stray semicolons\n> - read from the new yaml layout (which mirrors what we have at the\n> moment for lsc), and add keys to make it more flexible\n> \n> Changes in v3:\n> - add a constructor that takes a map of unsigned int -> matrix\n> - s/unit/identity\n> - clear matrices on reset and when reading from yaml\n> - add assert at get()\n> \n> Changes in v2:\n> - initialize to identity matrix\n> - add a function to reset to identity matrix\n> - other minor fixes\n> ---\n> src/ipa/libipa/matrix_interpolator.cpp | 110 ++++++++++++++++++++++\n> src/ipa/libipa/matrix_interpolator.h | 124 +++++++++++++++++++++++++\n> src/ipa/libipa/meson.build | 2 +\n> 3 files changed, 236 insertions(+)\n> create mode 100644 src/ipa/libipa/matrix_interpolator.cpp\n> create mode 100644 src/ipa/libipa/matrix_interpolator.h\n> \n> diff --git a/src/ipa/libipa/matrix_interpolator.cpp b/src/ipa/libipa/matrix_interpolator.cpp\n> new file mode 100644\n> index 000000000000..04ca177f72c6\n> --- /dev/null\n> +++ b/src/ipa/libipa/matrix_interpolator.cpp\n> @@ -0,0 +1,110 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Paul Elder \n> + *\n> + * Helper class for interpolating maps of matrices\n> + */\n> +#include \"matrix_interpolator.h\"\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/yaml_parser.h\"\n> +\n> +#include \"matrix.h\"\n> +\n> +/**\n> + * \\file matrix_interpolator.h\n> + * \\brief Helper class for interpolating maps of matrices\n> + */\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(MatrixInterpolator)\n> +\n> +namespace ipa {\n> +\n> +/**\n> + * \\class MatrixInterpolator\n> + * \\brief Class for storing, retrieving, and interpolating matrices\n> + * \\tparam T Type of numerical values to be stored in the matrices\n> + * \\tparam R Number of rows in the matrices\n> + * \\tparam C Number of columns in the matrices\n> + *\n> + * The main use case is to pass a map from color temperatures to corresponding\n> + * matrices (eg. color correction), and then requesting a matrix for a specific\n> + * color temperature. This class will abstract away the interpolation portion.\n> + */\n> +\n> +/**\n> + * \\fn MatrixInterpolator::MatrixInterpolator(const std::map> &matrices)\n> + * \\brief Construct a matrix interpolator from a map of matrices\n> + * \\param matrices Map from which to construct the matrix interpolator\n> + */\n> +\n> +/**\n> + * \\fn MatrixInterpolator::reset()\n> + * \\brief Reset the matrix interpolator content to a single identity matrix\n> + */\n> +\n> +/**\n> + * \\fn int MatrixInterpolator::readYaml()\n> + * \\brief Initialize an MatrixInterpolator instance from yaml\n> + * \\tparam T Type of data stored in the matrices\n> + * \\tparam R Number of rows of the matrices\n> + * \\tparam C Number of columns of the matrices\n> + * \\param[in] yaml The yaml object that contains the map of unsigned integers to matrices\n> + * \\param[in] key_name The name of the key in the yaml object\n> + * \\param[in] matrix_name The name of the matrix in the yaml object\n> + *\n> + * The yaml object is expected to be a list of maps. Each map has two or more\n> + * pairs: one of \\a key_name to the key value (usually color temperature), and\n> + * one or more of \\a matrix_name to the matrix. This is a bit difficult to\n> + * explain, so here is an example (in python, as it is easier to parse than\n> + * yaml):\n> + * [\n> + * {\n> + * 'ct': 2860,\n> + * 'ccm': [ 2.12089, -0.52461, -0.59629,\n> + * -0.85342, 2.80445, -0.95103,\n> + * -0.26897, -1.14788, 2.41685 ],\n> + * 'offsets': [ 0, 0, 0 ]\n> + * },\n> + *\n> + * {\n> + * 'ct': 2960,\n> + * 'ccm': [ 2.26962, -0.54174, -0.72789,\n> + * -0.77008, 2.60271, -0.83262,\n> + * -0.26036, -1.51254, 2.77289 ],\n> + * 'offsets': [ 0, 0, 0 ]\n> + * },\n> + *\n> + * {\n> + * 'ct': 3603,\n> + * 'ccm': [ 2.18644, -0.66148, -0.52496,\n> + * -0.77828, 2.69474, -0.91645,\n> + * -0.25239, -0.83059, 2.08298 ],\n> + * 'offsets': [ 0, 0, 0 ]\n> + * },\n> + * ]\n> + *\n> + * In this case, \\a key_name would be 'ct', and \\a matrix_name can be either\n> + * 'ccm' or 'offsets'. This way multiple matrix interpolators can be defined in\n> + * one set of color temperature ranges in the tuning file, and they can be\n> + * retrieved separately with the \\a matrix_name parameter.\n> + *\n> + * \\return Zero on success, negative error code otherwise\n> + */\n> +\n> +/**\n> + * \\fn Matrix MatrixInterpolator::get(unsigned int key)\n> + * \\brief Retrieve a matrix from the list of matrices, interpolating if necessary\n> + * \\param[in] key The unsigned integer key of the matrix to retrieve\n> + * \\return The matrix corresponding to the color temperature\n> + */\n> +\n> +} /* namespace ipa */\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/ipa/libipa/matrix_interpolator.h b/src/ipa/libipa/matrix_interpolator.h\n> new file mode 100644\n> index 000000000000..36827faec9ae\n> --- /dev/null\n> +++ b/src/ipa/libipa/matrix_interpolator.h\n> @@ -0,0 +1,124 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Paul Elder \n> + *\n> + * Helper class for interpolating maps of matrices\n> + */\n> +\n> +#pragma once\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/yaml_parser.h\"\n> +\n> +#include \"matrix.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(MatrixInterpolator)\n> +\n> +namespace ipa {\n> +\n> +#ifndef __DOXYGEN__\n> +template + std::enable_if_t> * = nullptr>\n> +#else\n> +template\n> +#endif /* __DOXYGEN__ */\n> +class MatrixInterpolator\n> +{\n> +public:\n> + MatrixInterpolator()\n> + {\n> + reset();\n> + }\n> +\n> + MatrixInterpolator(const std::map> &matrices)\n> + {\n> + for (const auto &pair : matrices)\n> + matrices_[pair.first] = pair.second;\n> + }\n> +\n> + ~MatrixInterpolator() {}\n> +\n> + void reset()\n> + {\n> + Matrix identity;\n> + matrices_.clear();\n> + matrices_[0] = identity;\n> + }\n> +\n> + int readYaml(const libcamera::YamlObject &yaml,\n> + const std::string &key_name,\n> + const std::string &matrix_name)\n> + {\n> + int ret;\n> +\n> + matrices_.clear();\n> +\n> + if (!yaml.isList()) {\n> + LOG(MatrixInterpolator, Error) << \"yaml object must be a list\";\n> + return -EINVAL;\n> + }\n> +\n> + for (const auto &value : yaml.asList()) {\n> + unsigned int ct = std::stoul(value[key_name].get(\"\"));\n> + Matrix matrix;\n> + if ((ret = matrix.readYaml(value[matrix_name])) < 0) {\n> + LOG(MatrixInterpolator, Error) << \"Failed to read matrix\";\n> + return ret;\n> + }\n> +\n> + matrices_[ct] = matrix;\n> +\n> + LOG(MatrixInterpolator, Debug)\n> + << \"Read matrix '\" << matrix_name << \"' for key '\"\n> + << key_name << \"' \" << ct << \": \"\n> + << matrices_[ct].toString();\n> + }\n> +\n> + if (matrices_.size() < 1) {\n> + LOG(MatrixInterpolator, Error) << \"Need at least one matrix\";\n> + return -EINVAL;\n> + }\n> +\n> + return 0;\n> + }\n> +\n> + Matrix get(unsigned int ct)\n> + {\n> + ASSERT(matrices_.size() > 0);\n> +\n> + if (matrices_.size() == 1 ||\n> + ct <= matrices_.begin()->first)\n> + return matrices_.begin()->second;\n> +\n> + if (ct >= matrices_.rbegin()->first)\n> + return matrices_.rbegin()->second;\n> +\n> + if (matrices_.find(ct) != matrices_.end())\n> + return matrices_[ct];\n> +\n> + /* The above four guarantee that this will succeed */\n> + auto iter = matrices_.upper_bound(ct);\n> + unsigned int ctUpper = iter->first;\n> + unsigned int ctLower = (--iter)->first;\n> +\n> + double lambda = (ct - ctLower) / static_cast(ctUpper - ctLower);\n> + Matrix ret =\n> + lambda * matrices_[ctUpper] + (1.0 - lambda) * matrices_[ctLower];\n> + return ret;\n> + }\n> +\n> +private:\n> + std::map> matrices_;\n> +};\n> +\n> +} /* namespace ipa */\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build\n> index 067d0d273e0a..a3349f0dd2c2 100644\n> --- a/src/ipa/libipa/meson.build\n> +++ b/src/ipa/libipa/meson.build\n> @@ -8,6 +8,7 @@ libipa_headers = files([\n> 'fc_queue.h',\n> 'histogram.h',\n> 'matrix.h',\n> + 'matrix_interpolator.h',\n> 'module.h',\n> ])\n> \n> @@ -19,6 +20,7 @@ libipa_sources = files([\n> 'fc_queue.cpp',\n> 'histogram.cpp',\n> 'matrix.cpp',\n> + 'matrix_interpolator.cpp',\n> 'module.cpp',\n> ])\n> \n> -- \n> 2.39.2\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5F7E6C31E9\n\tfor ;\n\tTue, 11 Jun 2024 14:14:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 81F4765455;\n\tTue, 11 Jun 2024 16:14:50 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B05EC61A26\n\tfor ;\n\tTue, 11 Jun 2024 16:14:48 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D861429A;\n\tTue, 11 Jun 2024 16:14:35 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"oe/w7RPE\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1718115275;\n\tbh=BkMhVsOr8xYZgpRH1HAe13PlJa33OBA69+P2lUmxFfA=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=oe/w7RPEVr0dcL+K9wRVoBLYGhchjYWVxihfmJaakJnTaWIl2rLvKms0ONGljAnyO\n\tRkfOArWbw82D3wqNvbxlJIVRxM+ehxd8jGuxxs3WWfao72M4OnVoh4ked3kCXebzcm\n\tSCgHWbd9z6VUHxevbNFPQdONDM5QEotcFeU4VMD4=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20240611140207.520083-3-paul.elder@ideasonboard.com>","References":"<20240611140207.520083-1-paul.elder@ideasonboard.com>\n\t<20240611140207.520083-3-paul.elder@ideasonboard.com>","Subject":"Re: [PATCH v7 2/3] ipa: libipa: Add MatrixInterpolator class","From":"Kieran Bingham ","Cc":"Paul Elder ,\n\tStefan Klug ","To":"Paul Elder ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Tue, 11 Jun 2024 15:14:45 +0100","Message-ID":"<171811528553.1148289.13427747030866283474@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]