[{"id":30956,"web_url":"https://patchwork.libcamera.org/comment/30956/","msgid":"","date":"2024-08-29T10:38:19","subject":"Re: [PATCH v1 1/8] ipa: libipa: Add generic Interpolator class","submitter":{"id":133,"url":"https://patchwork.libcamera.org/api/people/133/","name":"Pőcze Barnabás","email":"pobrn@protonmail.com"},"content":"Hi\n\n\n2024. augusztus 26., hétfő 18:21 keltezéssel, Stefan Klug írta:\n\n> The MatrixInterpolator is great for interpolation of matrices for different\n> color temperatures. It has however one limitation - it can only handle\n> matrices. For LSC it would be great to interpolate the LSC tables (or even\n> polynomials) using the same approach. Add a generic Interpolator class based on\n> the existing MatrixInterpolator. This calss can be adapted to any other type\n> using partial template specialization.\n> \n> Signed-off-by: Stefan Klug \n> ---\n> src/ipa/libipa/interpolator.cpp | 139 ++++++++++++++++++++++++++++++++\n> src/ipa/libipa/interpolator.h | 139 ++++++++++++++++++++++++++++++++\n> src/ipa/libipa/meson.build | 2 +\n> 3 files changed, 280 insertions(+)\n> create mode 100644 src/ipa/libipa/interpolator.cpp\n> create mode 100644 src/ipa/libipa/interpolator.h\n> \n> diff --git a/src/ipa/libipa/interpolator.cpp b/src/ipa/libipa/interpolator.cpp\n> new file mode 100644\n> index 000000000000..98462d1509da\n> --- /dev/null\n> +++ b/src/ipa/libipa/interpolator.cpp\n> @@ -0,0 +1,139 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Paul Elder \n> + *\n> + * Helper class for interpolating maps of matrices\n> + */\n> +#include \"interpolator.h\"\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/yaml_parser.h\"\n> +\n> +#include \"interpolator.h\"\n> +\n> +/**\n> + * \\file interpolator.h\n> + * \\brief Helper class for linear interpolating a set of objects\n> + */\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(Interpolator)\n> +\n> +namespace ipa {\n> +\n> +/**\n> + * \\class Interpolator\n> + * \\brief Class for storing, retrieving, and interpolating objects\n> + * \\tparam T Type of objects stored in the interpolator\n> + *\n> + * The main use case is to pass a map from color temperatures to corresponding\n> + * objects (eg. matrices for color correction), and then requesting a\n> + * interpolated object for a specific color temperature. This class will\n> + * abstract away the interpolation portion.\n> + */\n> +\n> +/**\n> + * \\fn Interpolator::Interpolator()\n> + * \\brief Construct a empty interpolator\n> + */\n> +\n> +/**\n> + * \\fn Interpolator::Interpolator(const std::map &data)\n> + * \\brief Construct a interpolator from a map of objects\n> + * \\param data Map from which to construct the interpolator\n> + */\n> +\n> +/**\n> + * \\fn Interpolator::Interpolator(const std::map &&data)\n> + * \\brief Construct a interpolator from a map of objects\n> + * \\param data Map from which to construct the interpolator\n> + */\n> +\n> +/**\n> + * \\fn int Interpolator::readYaml(const libcamera::YamlObject &yaml,\n> +\t\t const std::string &key_name,\n> +\t\t const std::string &value_name)\n> + * \\brief Initialize an Interpolator instance from yaml\n> + * \\tparam T Type of data stored in the interpolator\n> + * \\param[in] yaml The yaml object that contains the map of unsigned integers to objects\n> + * \\param[in] key_name The name of the key in the yaml object\n> + * \\param[in] value_name The name of the matrix in the yaml object\n> + *\n> + * The yaml object is expected to be a list of maps. Each map has two or more\n> + * pairs: one of \\a key_name to the key value (usually color temperature), and\n> + * one or more of \\a value_name to the object. This is a bit difficult to\n> + * explain, so here is an example (in python, as it is easier to parse than\n> + * yaml):\n> + * [\n> + * {\n> + * 'ct': 2860,\n> + * 'ccm': [ 2.12089, -0.52461, -0.59629,\n> + * -0.85342, 2.80445, -0.95103,\n> + * -0.26897, -1.14788, 2.41685 ],\n> + * 'offsets': [ 0, 0, 0 ]\n> + * },\n> + *\n> + * {\n> + * 'ct': 2960,\n> + * 'ccm': [ 2.26962, -0.54174, -0.72789,\n> + * -0.77008, 2.60271, -0.83262,\n> + * -0.26036, -1.51254, 2.77289 ],\n> + * 'offsets': [ 0, 0, 0 ]\n> + * },\n> + *\n> + * {\n> + * 'ct': 3603,\n> + * 'ccm': [ 2.18644, -0.66148, -0.52496,\n> + * -0.77828, 2.69474, -0.91645,\n> + * -0.25239, -0.83059, 2.08298 ],\n> + * 'offsets': [ 0, 0, 0 ]\n> + * },\n> + * ]\n> + *\n> + * In this case, \\a key_name would be 'ct', and \\a value_name can be either\n> + * 'ccm' or 'offsets'. This way multiple interpolators can be defined in\n> + * one set of color temperature ranges in the tuning file, and they can be\n> + * retrieved separately with the \\a value_name parameter.\n> + *\n> + * \\return Zero on success, negative error code otherwise\n> + */\n> +\n> +/**\n> + * \\fn void setQuantization(const unsigned int q)\n> + * \\brief Set the quantization value\n> + * \\param[in] q The quantization value\n> + *\n> + * Sets the quantization value. When this is set key gets quantized to this size,\n> + * before doing the interpolation. This can help in reducing the number of\n> + * updated pushed to the hardware.\n> + *\n> + * Note that normally a threshold needs to be combined with quantization.\n> + * Otherwise a value that swings around the edge of the quantization step will\n> + * lead to constant updates.\n> + */\n> +\n> +/**\n> + * \\fn void setData(std::map &&data)\n> + * \\brief Set the internal map\n> + *\n> + * Overwrites the internal map using move semantics.\n> + */\n> +\n> +/**\n> + * \\fn const T &getInterpolated(unsigned int key, unsigned int *quantizedKey = nullptr)\n> + * \\brief Retrieve a a interpolated value for the given key\n> + * \\param[in] key The unsigned integer key of the object to retrieve\n> + * \\param[out] quantizedKey If provided, the key value after quantization\n> + * \\return The object corresponding to the key. The object is cached internally,\n> + * so on successive calls with the same key (after quantization) interpolation\n> + * is not recalculated.\n> + */\n> +\n> +} /* namespace ipa */\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/ipa/libipa/interpolator.h b/src/ipa/libipa/interpolator.h\n> new file mode 100644\n> index 000000000000..b0241c35bbab\n> --- /dev/null\n> +++ b/src/ipa/libipa/interpolator.h\n> @@ -0,0 +1,139 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Paul Elder \n> + *\n> + * Helper class for interpolating maps of objects\n> + */\n> +\n> +#pragma once\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/yaml_parser.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(Interpolator)\n> +\n> +namespace ipa {\n> +\n> +template\n> +class Interpolator\n> +{\n> +public:\n> +\tInterpolator()\n> +\t{\n> +\t\tquantization_ = 0;\n> +\t}\n\nInterpolator() = default;\n\n\n> +\n> +\tInterpolator(const std::map &data)\n> +\t{\n> +\t\tquantization_ = 0;\n> +\t\tdata_ = data;\n> +\t}\n\nInterpolator(const std::map &data)\n\t: data_(data)\n{ }\n\n\n> +\n> +\tInterpolator(std::map &&data)\n> +\t{\n> +\t\tquantization_ = 0;\n> +\t\tdata_ = std::move(data);\n> +\t}\n\nInterpolator(std::map &&data)\n\t: data_(std::move(data))\n{ }\n\n\n> +\n> +\t~Interpolator() {}\n\n~Interpolator() = default;\n\n\n> +\n> +\tint readYaml(const libcamera::YamlObject &yaml,\n> +\t\t const std::string &key_name,\n> +\t\t const std::string &value_name)\n> +\t{\n> +\t\tdata_.clear();\n> +\n> +\t\tif (!yaml.isList()) {\n> +\t\t\tLOG(Interpolator, Error) << \"yaml object must be a list\";\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> +\n> +\t\tfor (const auto &value : yaml.asList()) {\n> +\t\t\tunsigned int ct = std::stoul(value[key_name].get(\"\"));\n> +\t\t\tstd::optional data =\n> +\t\t\t\tvalue[value_name].get();\n> +\t\t\tif (!data) {\n> +\t\t\t\treturn -EINVAL;\n> +\t\t\t}\n> +\n> +\t\t\tdata_[ct] = *data;\n> +\t\t}\n> +\n> +\t\tif (data_.size() < 1) {\n> +\t\t\tLOG(Interpolator, Error) << \"Need at least one element\";\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> +\n> +\t\treturn 0;\n> +\t}\n> +\n> +\tvoid setQuantization(const unsigned int q)\n> +\t{\n> +\t\tquantization_ = q;\n> +\t}\n> +\n> +\tvoid setData(std::map &&data)\n> +\t{\n> +\t\tdata_ = std::move(data);\n> +\t}\n> +\n> +\tconst T &getInterpolated(unsigned int key, unsigned int *quantizedKey = nullptr)\n> +\t{\n> +\t\tASSERT(data_.size() > 0);\n> +\n> +\t\tif (quantization_ > 0)\n> +\t\t\tkey = std::lround(key / static_cast(quantization_)) * quantization_;\n> +\n> +\t\tif (quantizedKey)\n> +\t\t\t*quantizedKey = key;\n> +\n> +\t\tif (lastInterpolatedKey_.has_value() &&\n> +\t\t *lastInterpolatedKey_ == key)\n> +\t\t\treturn lastInterpolatedValue_;\n\nsetData() should reset the cache. By the way, is caching worth it?\n\n\n> +\n> +\t\tif (data_.size() == 1 ||\n> +\t\t key <= data_.begin()->first)\n> +\t\t\treturn data_.begin()->second;\n> +\n> +\t\tif (key >= data_.rbegin()->first)\n> +\t\t\treturn data_.rbegin()->second;\n> +\n> +\t\tif (data_.find(key) != data_.end())\n> +\t\t\treturn data_[key];\n> +\n> +\t\t/* The above four guarantee that this will succeed */\n> +\t\tauto iter = data_.upper_bound(key);\n> +\t\tunsigned int ctUpper = iter->first;\n> +\t\tunsigned int ctLower = (--iter)->first;\n> +\n> +\t\tdouble lambda = (key - ctLower) / static_cast(ctUpper - ctLower);\n> +\t\tinterpolate(data_[ctLower], data_[ctUpper], lastInterpolatedValue_, lambda);\n> +\t\tlastInterpolatedKey_ = key;\n\nThe above has 5 separate map lookups, I am wondering if something like this could work:\n\n auto it = data_.lower_bound(key);\n\n if (it == data_.begin())\n return it->second;\n\n if (it == data_.end())\n return std::prev(it)->second;\n\n if (it->first == key)\n return it->second;\n\n auto it2 = std::prev(it);\n\n double lambda = (key - it2->first) / static_cast(it->first - it2->first);\n T val = it2->second * (1 - lambda) + it->second * lambda;\n\n\n> +\n> +\t\treturn lastInterpolatedValue_;\n> +\t}\n> +\n> +\tvoid interpolate(const T &a, const T &b, T &dest, double lambda)\n> +\t{\n> +\t\tdest = a * (1.0 - lambda) + b * lambda;\n> +\t}\n> +\n> +private:\n> +\tstd::map data_;\n> +\tT lastInterpolatedValue_;\n> +\tstd::optional lastInterpolatedKey_;\n> +\tunsigned int quantization_;\n\nSince `quantization_` is initialized to 0 in all constructors, I think it might\nbe best to say ` = 0` here.\n\n\nRegards,\nBarnabás Pőcze\n\n\n> +};\n> +\n> +} /* namespace ipa */\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build\n> index eff8ce2660f1..2c2712a7d252 100644\n> --- a/src/ipa/libipa/meson.build\n> +++ b/src/ipa/libipa/meson.build\n> @@ -7,6 +7,7 @@ libipa_headers = files([\n> 'exposure_mode_helper.h',\n> 'fc_queue.h',\n> 'histogram.h',\n> + 'interpolator.h',\n> 'matrix.h',\n> 'matrix_interpolator.h',\n> 'module.h',\n> @@ -21,6 +22,7 @@ libipa_sources = files([\n> 'exposure_mode_helper.cpp',\n> 'fc_queue.cpp',\n> 'histogram.cpp',\n> + 'interpolator.cpp',\n> 'matrix.cpp',\n> 'matrix_interpolator.cpp',\n> 'module.cpp',\n> --\n> 2.43.0\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 42A3BC324C\n\tfor ;\n\tThu, 29 Aug 2024 10:38:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4EF6763461;\n\tThu, 29 Aug 2024 12:38:27 +0200 (CEST)","from mail-4316.protonmail.ch (mail-4316.protonmail.ch\n\t[185.70.43.16])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1DB0C63456\n\tfor ;\n\tThu, 29 Aug 2024 12:38:25 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=protonmail.com header.i=@protonmail.com\n\theader.b=\"V/jqdxzS\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;\n\ts=protonmail3; t=1724927904; x=1725187104;\n\tbh=0NSzqU7AJyLqBmlhONv9qQjQsulF1rEdlSDkKHrDRSo=;\n\th=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:\n\tFeedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:\n\tMessage-ID:BIMI-Selector;\n\tb=V/jqdxzSAvJbJuNLKcOOs+I8hrzUGvvlZyPyrDg52/Nh/xhGU6hLa523t8ZL8XZgs\n\t8KTjFM0s8MxcZYI/JlUpaQqbAczcNjyM4FjEIYCv10m3macMIgWohBuawR0mUOZHSg\n\tJBZit8YkzWSZ2zwKaCuNEQNROx2jeTZOtTHsIP3Ky0BNFPOQrLUudpfU+y7/h+NSoT\n\t7waUs4MVHKxwvZkjrvYbAoADSw4Ku9g6R1Kb3/xxbS1g/J4xBEvXP6z30FnC4Aw9tb\n\trGd25Rw98w8QrhfkdeJduOULLPUaSClMXpIDR5o8OPXPqLjnu1+zietgqH8Hx2mZ+N\n\tLmAE12kBY3YIg==","Date":"Thu, 29 Aug 2024 10:38:19 +0000","To":"Stefan Klug ","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH v1 1/8] ipa: libipa: Add generic Interpolator class","Message-ID":"","In-Reply-To":"<20240826152224.362773-2-stefan.klug@ideasonboard.com>","References":"<20240826152224.362773-1-stefan.klug@ideasonboard.com>\n\t<20240826152224.362773-2-stefan.klug@ideasonboard.com>","Feedback-ID":"20568564:user:proton","X-Pm-Message-ID":"483f83e3735cedd3fff3ab482baf79ad73f37e19","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]