From patchwork Fri Jun 14 11:37:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 20318 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E6201C3237 for ; Fri, 14 Jun 2024 11:38:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9B07F65497; Fri, 14 Jun 2024 13:38:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wDdO/bG/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AA18D65491 for ; Fri, 14 Jun 2024 13:38:09 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (h175-177-049-156.catv02.itscom.jp [175.177.49.156]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A08393D5; Fri, 14 Jun 2024 13:37:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1718365075; bh=VnlyZH+Zq7nAfM3PS3FqqrruPbi0vukgucEvIlgogOs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wDdO/bG/f7OmMi95u3ovubmqRFnJiyKvzjUfcXlvu6XXbDMDHgK3tGGCHwZL11aw4 faWAfa4fASdfkkE2Ug/MAwyPG5Ba3ozYNZFhp8HxvwDJkTWx2cGElaqxNwizm+ZZgE C1Zm0RPwnq0RYv7xPFjxBpTVcW8cK9fiDCRh8aYU= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , Stefan Klug , Kieran Bingham Subject: [PATCH v9 2/3] ipa: libipa: Add MatrixInterpolator class Date: Fri, 14 Jun 2024 20:37:56 +0900 Message-Id: <20240614113757.109452-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240614113757.109452-1-paul.elder@ideasonboard.com> References: <20240614113757.109452-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a class to encapsulate the functionality of fetching a matrix based on an integer key, and interpolating if there is no exact match. This is expected to be used by both color correction matrices / crosstalk correction as well as lens shading correction. A cache is included only for exact matches of the key. The caller is expected to decide the tolererance for rounding. Signed-off-by: Paul Elder Reviewed-by: Stefan Klug Reviewed-by: Kieran Bingham --- No change in v9 Changes in v8: - use yaml get function - use Matrix::identity() (instead of the default constructor, now that it makes a zero matrix instead of an identity matrix) Changes in v7: - remove cache - fix copyright and license Changes in v6: - fix doxygen Changes in v5: - improve documentation - replace count() with find() != end() (in get()) - add cache Changes in v4: - remove stray semicolons - read from the new yaml layout (which mirrors what we have at the moment for lsc), and add keys to make it more flexible Changes in v3: - add a constructor that takes a map of unsigned int -> matrix - s/unit/identity - clear matrices on reset and when reading from yaml - add assert at get() Changes in v2: - initialize to identity matrix - add a function to reset to identity matrix - other minor fixes --- src/ipa/libipa/matrix_interpolator.cpp | 110 ++++++++++++++++++++++ src/ipa/libipa/matrix_interpolator.h | 122 +++++++++++++++++++++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 234 insertions(+) create mode 100644 src/ipa/libipa/matrix_interpolator.cpp create mode 100644 src/ipa/libipa/matrix_interpolator.h diff --git a/src/ipa/libipa/matrix_interpolator.cpp b/src/ipa/libipa/matrix_interpolator.cpp new file mode 100644 index 000000000000..04ca177f72c6 --- /dev/null +++ b/src/ipa/libipa/matrix_interpolator.cpp @@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Paul Elder + * + * Helper class for interpolating maps of matrices + */ +#include "matrix_interpolator.h" + +#include +#include + +#include + +#include "libcamera/internal/yaml_parser.h" + +#include "matrix.h" + +/** + * \file matrix_interpolator.h + * \brief Helper class for interpolating maps of matrices + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(MatrixInterpolator) + +namespace ipa { + +/** + * \class MatrixInterpolator + * \brief Class for storing, retrieving, and interpolating matrices + * \tparam T Type of numerical values to be stored in the matrices + * \tparam R Number of rows in the matrices + * \tparam C Number of columns in the matrices + * + * The main use case is to pass a map from color temperatures to corresponding + * matrices (eg. color correction), and then requesting a matrix for a specific + * color temperature. This class will abstract away the interpolation portion. + */ + +/** + * \fn MatrixInterpolator::MatrixInterpolator(const std::map> &matrices) + * \brief Construct a matrix interpolator from a map of matrices + * \param matrices Map from which to construct the matrix interpolator + */ + +/** + * \fn MatrixInterpolator::reset() + * \brief Reset the matrix interpolator content to a single identity matrix + */ + +/** + * \fn int MatrixInterpolator::readYaml() + * \brief Initialize an MatrixInterpolator instance from yaml + * \tparam T Type of data stored in the matrices + * \tparam R Number of rows of the matrices + * \tparam C Number of columns of the matrices + * \param[in] yaml The yaml object that contains the map of unsigned integers to matrices + * \param[in] key_name The name of the key in the yaml object + * \param[in] matrix_name The name of the matrix in the yaml object + * + * The yaml object is expected to be a list of maps. Each map has two or more + * pairs: one of \a key_name to the key value (usually color temperature), and + * one or more of \a matrix_name to the matrix. This is a bit difficult to + * explain, so here is an example (in python, as it is easier to parse than + * yaml): + * [ + * { + * 'ct': 2860, + * 'ccm': [ 2.12089, -0.52461, -0.59629, + * -0.85342, 2.80445, -0.95103, + * -0.26897, -1.14788, 2.41685 ], + * 'offsets': [ 0, 0, 0 ] + * }, + * + * { + * 'ct': 2960, + * 'ccm': [ 2.26962, -0.54174, -0.72789, + * -0.77008, 2.60271, -0.83262, + * -0.26036, -1.51254, 2.77289 ], + * 'offsets': [ 0, 0, 0 ] + * }, + * + * { + * 'ct': 3603, + * 'ccm': [ 2.18644, -0.66148, -0.52496, + * -0.77828, 2.69474, -0.91645, + * -0.25239, -0.83059, 2.08298 ], + * 'offsets': [ 0, 0, 0 ] + * }, + * ] + * + * In this case, \a key_name would be 'ct', and \a matrix_name can be either + * 'ccm' or 'offsets'. This way multiple matrix interpolators can be defined in + * one set of color temperature ranges in the tuning file, and they can be + * retrieved separately with the \a matrix_name parameter. + * + * \return Zero on success, negative error code otherwise + */ + +/** + * \fn Matrix MatrixInterpolator::get(unsigned int key) + * \brief Retrieve a matrix from the list of matrices, interpolating if necessary + * \param[in] key The unsigned integer key of the matrix to retrieve + * \return The matrix corresponding to the color temperature + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/matrix_interpolator.h b/src/ipa/libipa/matrix_interpolator.h new file mode 100644 index 000000000000..087c4fd14e4f --- /dev/null +++ b/src/ipa/libipa/matrix_interpolator.h @@ -0,0 +1,122 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Paul Elder + * + * Helper class for interpolating maps of matrices + */ + +#pragma once + +#include +#include +#include +#include + +#include + +#include "libcamera/internal/yaml_parser.h" + +#include "matrix.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(MatrixInterpolator) + +namespace ipa { + +#ifndef __DOXYGEN__ +template> * = nullptr> +#else +template +#endif /* __DOXYGEN__ */ +class MatrixInterpolator +{ +public: + MatrixInterpolator() + { + reset(); + } + + MatrixInterpolator(const std::map> &matrices) + { + for (const auto &pair : matrices) + matrices_[pair.first] = pair.second; + } + + ~MatrixInterpolator() {} + + void reset() + { + matrices_.clear(); + matrices_[0] = Matrix::identity(); + } + + int readYaml(const libcamera::YamlObject &yaml, + const std::string &key_name, + const std::string &matrix_name) + { + matrices_.clear(); + + if (!yaml.isList()) { + LOG(MatrixInterpolator, Error) << "yaml object must be a list"; + return -EINVAL; + } + + for (const auto &value : yaml.asList()) { + unsigned int ct = std::stoul(value[key_name].get("")); + std::optional> matrix = + value[matrix_name].get>(); + if (!matrix) { + LOG(MatrixInterpolator, Error) << "Failed to read matrix"; + return -EINVAL; + } + + matrices_[ct] = *matrix; + + LOG(MatrixInterpolator, Debug) + << "Read matrix '" << matrix_name << "' for key '" + << key_name << "' " << ct << ": " + << matrices_[ct].toString(); + } + + if (matrices_.size() < 1) { + LOG(MatrixInterpolator, Error) << "Need at least one matrix"; + return -EINVAL; + } + + return 0; + } + + Matrix get(unsigned int ct) + { + ASSERT(matrices_.size() > 0); + + if (matrices_.size() == 1 || + ct <= matrices_.begin()->first) + return matrices_.begin()->second; + + if (ct >= matrices_.rbegin()->first) + return matrices_.rbegin()->second; + + if (matrices_.find(ct) != matrices_.end()) + return matrices_[ct]; + + /* The above four guarantee that this will succeed */ + auto iter = matrices_.upper_bound(ct); + unsigned int ctUpper = iter->first; + unsigned int ctLower = (--iter)->first; + + double lambda = (ct - ctLower) / static_cast(ctUpper - ctLower); + Matrix ret = + lambda * matrices_[ctUpper] + (1.0 - lambda) * matrices_[ctLower]; + return ret; + } + +private: + std::map> matrices_; +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 1952dccf6236..eff8ce2660f1 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -8,6 +8,7 @@ libipa_headers = files([ 'fc_queue.h', 'histogram.h', 'matrix.h', + 'matrix_interpolator.h', 'module.h', 'pwl.h', 'vector.h', @@ -21,6 +22,7 @@ libipa_sources = files([ 'fc_queue.cpp', 'histogram.cpp', 'matrix.cpp', + 'matrix_interpolator.cpp', 'module.cpp', 'pwl.cpp', 'vector.cpp',