From patchwork Wed Mar 19 16:11:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22979 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 20986C32FE for ; Wed, 19 Mar 2025 16:12:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CCBB868950; Wed, 19 Mar 2025 17:12:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UsT7bEDo"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F24CD68945 for ; Wed, 19 Mar 2025 17:11:59 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 04DBE8FA; Wed, 19 Mar 2025 17:10:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400617; bh=oFIl/d/8UPVKsZ0KLKeqRsE/24LTr+AMeld281uk9Cw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UsT7bEDoBvpHt1cIj7RiEusG2Ettnm1gbXeJaOLl6hLSpvzec+dskJzdsBe58RseC hAG1m+y25gcs03vS6034jYW8AysI7X3WETTquMpIUgMkAvDUPczb8Jrv4dO5gfDpGj s5wSkx3D1oVCM971lJLHP/ObbVt5D8jA2Z8jfyo8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 01/17] libcamera: matrix: Replace SFINAE with static_asserts Date: Wed, 19 Mar 2025 17:11:06 +0100 Message-ID: <20250319161152.63625-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SFINAE is difficult to read and not needed in these cases. Replace it with static_asserts. The idea came from [1] where it is stated: "The use of enable_if seems misguided to me. SFINAE is useful for the situation where we consider multiple candidates for something (overloads or class template specializations) and try to choose the correct one, without causing compilation to fail." [1]: https://stackoverflow.com/questions/62109526/c-friend-template-that-use-sfinae Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- include/libcamera/internal/matrix.h | 42 ++++++++--------------------- 1 file changed, 11 insertions(+), 31 deletions(-) diff --git a/include/libcamera/internal/matrix.h b/include/libcamera/internal/matrix.h index a055e6926c94..8399be583f28 100644 --- a/include/libcamera/internal/matrix.h +++ b/include/libcamera/internal/matrix.h @@ -19,14 +19,11 @@ namespace libcamera { LOG_DECLARE_CATEGORY(Matrix) -#ifndef __DOXYGEN__ -template> * = nullptr> -#else template -#endif /* __DOXYGEN__ */ class Matrix { + static_assert(std::is_arithmetic_v, "Matrix type must be arithmetic"); + public: Matrix() { @@ -78,13 +75,10 @@ public: return Span{ &data_.data()[i * Cols], Cols }; } -#ifndef __DOXYGEN__ - template>> -#else template -#endif /* __DOXYGEN__ */ - Matrix &operator*=(U d) + constexpr Matrix &operator*=(U d) { + static_assert(std::is_arithmetic_v, "Multiplier must be arithmetic"); for (unsigned int i = 0; i < Rows * Cols; i++) data_[i] *= d; return *this; @@ -94,14 +88,10 @@ private: std::array data_; }; -#ifndef __DOXYGEN__ -template> * = nullptr> -#else template -#endif /* __DOXYGEN__ */ -Matrix operator*(T d, const Matrix &m) +constexpr Matrix operator*(T d, const Matrix &m) { + static_assert(std::is_arithmetic_v, "Multiplier must be arithmetic"); Matrix result; for (unsigned int i = 0; i < Rows; i++) { @@ -112,27 +102,17 @@ Matrix operator*(T d, const Matrix &m) return result; } -#ifndef __DOXYGEN__ -template> * = nullptr> -#else template -#endif /* __DOXYGEN__ */ -Matrix operator*(const Matrix &m, T d) +constexpr Matrix operator*(const Matrix &m, T d) { + static_assert(std::is_arithmetic_v, "Multiplier must be arithmetic"); return d * m; } -#ifndef __DOXYGEN__ -template * = nullptr> -#else -template -#endif /* __DOXYGEN__ */ -Matrix operator*(const Matrix &m1, const Matrix &m2) +template +constexpr Matrix operator*(const Matrix &m1, const Matrix &m2) { + static_assert(C1 == R2, "Matrix dimensions must match for multiplication"); Matrix result; for (unsigned int i = 0; i < R1; i++) { From patchwork Wed Mar 19 16:11:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22980 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9DC3EC32FE for ; Wed, 19 Mar 2025 16:12:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5DF3A68956; Wed, 19 Mar 2025 17:12:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OQxsZKpU"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F11C768945 for ; Wed, 19 Mar 2025 17:12:02 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 14DB58FA; Wed, 19 Mar 2025 17:10:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400620; bh=GX+nwPV4aQPLp2kqpsBwud/hZduuE8qLSo/rMFQK/ZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OQxsZKpUf2yAoodyn6lmrnezgcjoi8Aqv3Sp38SGUHUHH4upsddVryasTDhXrCLHM CFihc3UbWGb9I3Id5A1Tw47daG5R1SL2vDFQNckGhScHMU3wqOoZy8Elsg23uDhsA8 VyPtjJML42rtDT9WU+D24W3VTio5+Y134o4oOQeg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 02/17] libcamera: matrix: Make most functions constexpr Date: Wed, 19 Mar 2025 17:11:07 +0100 Message-ID: <20250319161152.63625-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" By zero-initializing the data_ member we can make most functions constexpr which will come in handy in upcoming patches. Note that this is due to C++17. In C++20 we will be able to leave data_ unintialized for constexpr. The Matrix(std::array) version of the constructor can not be constexpr because std::copy only became constexpr in C++20. Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- include/libcamera/internal/matrix.h | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/include/libcamera/internal/matrix.h b/include/libcamera/internal/matrix.h index 8399be583f28..2e7929e9060c 100644 --- a/include/libcamera/internal/matrix.h +++ b/include/libcamera/internal/matrix.h @@ -25,9 +25,8 @@ class Matrix static_assert(std::is_arithmetic_v, "Matrix type must be arithmetic"); public: - Matrix() + constexpr Matrix() { - data_.fill(static_cast(0)); } Matrix(const std::array &data) @@ -35,7 +34,7 @@ public: std::copy(data.begin(), data.end(), data_.begin()); } - static Matrix identity() + static constexpr Matrix identity() { Matrix ret; for (size_t i = 0; i < std::min(Rows, Cols); i++) @@ -63,14 +62,14 @@ public: return out.str(); } - Span data() const { return data_; } + constexpr Span data() const { return data_; } - Span operator[](size_t i) const + constexpr Span operator[](size_t i) const { return Span{ &data_.data()[i * Cols], Cols }; } - Span operator[](size_t i) + constexpr Span operator[](size_t i) { return Span{ &data_.data()[i * Cols], Cols }; } @@ -85,7 +84,7 @@ public: } private: - std::array data_; + std::array data_{}; }; template @@ -130,7 +129,7 @@ constexpr Matrix operator*(const Matrix &m1, const Matrix< } template -Matrix operator+(const Matrix &m1, const Matrix &m2) +constexpr Matrix operator+(const Matrix &m1, const Matrix &m2) { Matrix result; From patchwork Wed Mar 19 16:11:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22981 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8B61FC32FE for ; Wed, 19 Mar 2025 16:12:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3839C68956; Wed, 19 Mar 2025 17:12:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="v+vnFvqP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BFD7968950 for ; Wed, 19 Mar 2025 17:12:05 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D00548FA; Wed, 19 Mar 2025 17:10:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400622; bh=UwDt89XFx7PIoRfW4bceJOVjymlThJ3lPWKM60WujCU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=v+vnFvqPZTqH+eJIYmw1HKqOINy8QJy37uk9alOODZ2Pz0q+vQiYSwXxpIWA+B/dF m0e2Ffs7EWLjNWNU8eomQilNbR3pHXdFRaz+kP0lDuXTD2+qZXWRE8saCTI53VIY7i VoauD4OUGE0oHTXhdGdDcb4681CIsWK2xDLE+PR4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 03/17] libcamera: matrix: Add a Span based constructor Date: Wed, 19 Mar 2025 17:11:08 +0100 Message-ID: <20250319161152.63625-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When one wants to create a Matrix from existing data, currently the only way is via std::array. Add a Span based constructor to allow creation from vectors and alike. Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- include/libcamera/internal/matrix.h | 5 +++++ src/libcamera/matrix.cpp | 10 ++++++++++ 2 files changed, 15 insertions(+) diff --git a/include/libcamera/internal/matrix.h b/include/libcamera/internal/matrix.h index 2e7929e9060c..9b80521e3cb0 100644 --- a/include/libcamera/internal/matrix.h +++ b/include/libcamera/internal/matrix.h @@ -34,6 +34,11 @@ public: std::copy(data.begin(), data.end(), data_.begin()); } + Matrix(const Span &data) + { + std::copy(data.begin(), data.end(), data_.begin()); + } + static constexpr Matrix identity() { Matrix ret; diff --git a/src/libcamera/matrix.cpp b/src/libcamera/matrix.cpp index e7e027225666..6dca7498cab3 100644 --- a/src/libcamera/matrix.cpp +++ b/src/libcamera/matrix.cpp @@ -41,6 +41,16 @@ LOG_DEFINE_CATEGORY(Matrix) * number of rows and columns of the matrix (Rows x Cols). */ +/** + * \fn Matrix::Matrix(const Span &data) + * \brief Construct a matrix from supplied data + * \param[in] data Data from which to construct a matrix + * + * \a data is a one-dimensional Span and will be turned into a matrix in + * row-major order. The size of \a data must be equal to the product of the + * number of rows and columns of the matrix (Rows x Cols). + */ + /** * \fn Matrix::identity() * \brief Construct an identity matrix From patchwork Wed Mar 19 16:11:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22982 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2A39CC32FE for ; Wed, 19 Mar 2025 16:12:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D7A7D6895B; Wed, 19 Mar 2025 17:12:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HQGEtgHn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 987AF68950 for ; Wed, 19 Mar 2025 17:12:08 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A1FF38FA; Wed, 19 Mar 2025 17:10:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400625; bh=Wav+/7DNoCoKWjCuh5eKfA2krY3LS9nUcXcQ3tbPgbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HQGEtgHncSgzwHAFvlib+/sZeOkooh1S3/wYl/Xg/iBdQepUC8RvLkaGRCYmEFfT5 S8hVSYRatAuUhXNHPiM8CEGP0pKgI/JGn1kKb6iQc0M5/cQ/MHKRqummJnZiF10+fl ugVU1TuMERPooxHNF31QvVkYwBcpKMzlSZyr+Xes= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 04/17] libcamera: vector: Add a Span based constructor Date: Wed, 19 Mar 2025 17:11:09 +0100 Message-ID: <20250319161152.63625-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that Matrix has a Span based constructor, add the same for Vector. Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- include/libcamera/internal/vector.h | 12 ++++++++---- src/libcamera/vector.cpp | 8 ++++++++ 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/include/libcamera/internal/vector.h b/include/libcamera/internal/vector.h index a67a09474204..66cc5ac988c2 100644 --- a/include/libcamera/internal/vector.h +++ b/include/libcamera/internal/vector.h @@ -40,10 +40,14 @@ public: data_.fill(scalar); } - constexpr Vector(const std::array &data) + Vector(const std::array &data) { - for (unsigned int i = 0; i < Rows; i++) - data_[i] = data[i]; + std::copy(data.begin(), data.end(), data_.begin()); + } + + Vector(const Span &data) + { + std::copy(data.begin(), data.end(), data_.begin()); } const T &operator[](size_t i) const @@ -285,7 +289,7 @@ private: return *this; } - std::array data_; + std::array data_{}; }; template diff --git a/src/libcamera/vector.cpp b/src/libcamera/vector.cpp index 85ca2208245a..435f2fc62f8b 100644 --- a/src/libcamera/vector.cpp +++ b/src/libcamera/vector.cpp @@ -44,6 +44,14 @@ LOG_DEFINE_CATEGORY(Vector) * The size of \a data must be equal to the dimension size Rows of the vector. */ +/** + * \fn Vector::Vector(const Span &data) + * \brief Construct vector from supplied data + * \param data Data from which to construct a vector + * + * The size of \a data must be equal to the dimension size Rows of the vector. + */ + /** * \fn T Vector::operator[](size_t i) const * \brief Index to an element in the vector From patchwork Wed Mar 19 16:11:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22983 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 018E2C32FE for ; Wed, 19 Mar 2025 16:12:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A92B6895B; Wed, 19 Mar 2025 17:12:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uvH2Li6K"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE4D268950 for ; Wed, 19 Mar 2025 17:12:11 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EC4588FA; Wed, 19 Mar 2025 17:10:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400629; bh=Gfx7x+E8Sx24+3cf5pWOkHZEsFcd/OJpxqWgEUGDWOQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uvH2Li6KYtr+IhpDRImz0Kft4waOW8/iusEVnphGvEtIowZy6A1JOygQq54tJUmAg ZwtZMnjU95P3xEzKvjyBlx13xwWd0FdPgifZNbta1p9Pt0v8vB9UENBJPENtXIm1SQ GiDyuQO0sKUPPsPjTUId5nyCzeeazWIQkJFBHNRA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v2 05/17] libcamera: matrix: Add inverse() function Date: Wed, 19 Mar 2025 17:11:10 +0100 Message-ID: <20250319161152.63625-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For calculations in upcoming algorithm patches, the inverse of a matrix is required. Add an implementation of the inverse() function for square matrices. Signed-off-by: Stefan Klug Signed-off-by: Laurent Pinchart --- Changes in v2: - Replaced the implementation by a generic one provided by Laurent that supports arbitrary square matrices instead of 2x2 and 3x3 only. - Moved the implementation into the cpp file. --- include/libcamera/internal/matrix.h | 16 +++ src/libcamera/matrix.cpp | 160 ++++++++++++++++++++++++++++ 2 files changed, 176 insertions(+) diff --git a/include/libcamera/internal/matrix.h b/include/libcamera/internal/matrix.h index 9b80521e3cb0..6e3c190286fe 100644 --- a/include/libcamera/internal/matrix.h +++ b/include/libcamera/internal/matrix.h @@ -19,6 +19,11 @@ namespace libcamera { LOG_DECLARE_CATEGORY(Matrix) +#ifndef __DOXYGEN__ +template +bool matrixInvert(Span dataIn, Span dataOut, unsigned int dim); +#endif /* __DOXYGEN__ */ + template class Matrix { @@ -88,6 +93,17 @@ public: return *this; } + Matrix inverse(bool *ok = nullptr) const + { + static_assert(Rows == Cols, "Matrix must be square"); + + Matrix inverse; + bool res = matrixInvert(Span(data_), Span(inverse.data_), Rows); + if (ok) + *ok = res; + return inverse; + } + private: std::array data_{}; }; diff --git a/src/libcamera/matrix.cpp b/src/libcamera/matrix.cpp index 6dca7498cab3..8590f8efeff3 100644 --- a/src/libcamera/matrix.cpp +++ b/src/libcamera/matrix.cpp @@ -7,6 +7,12 @@ #include "libcamera/internal/matrix.h" +#include +#include +#include +#include +#include + #include /** @@ -87,6 +93,20 @@ LOG_DEFINE_CATEGORY(Matrix) * \return Row \a i from the matrix, as a Span */ +/** + * \fn Matrix::inverse(bool *ok) const + * \param[out] ok Indicate if the matrix was successfully inverted + * \brief Compute the inverse of the matrix + * + * This function computes the inverse of the matrix. It is only implemented for + * matrices of float and double types. If \a ok is provided it will be set to a + * boolean value to indicate of the inversion was successful. This can be used + * to check if the matrix is singular, in which case the function will return + * an identity matrix. + * + * \return The inverse of the matrix + */ + /** * \fn Matrix::operator[](size_t i) * \copydoc Matrix::operator[](size_t i) const @@ -142,6 +162,146 @@ LOG_DEFINE_CATEGORY(Matrix) */ #ifndef __DOXYGEN__ +template +bool matrixInvert(Span dataIn, Span dataOut, unsigned int dim) +{ + /* + * Convenience class to access matrix data, providing a row-major (i,j) + * element accessor through the call operator, and the ability to swap + * rows without modifying the backing storage. + */ + class MatrixAccessor + { + public: + MatrixAccessor(Span data, unsigned int rows, unsigned int cols) + : data_(data), swap_(rows), rows_(rows), cols_(cols) + { + std::iota(swap_.begin(), swap_.end(), T{ 0 }); + } + + T &operator()(unsigned int row, unsigned int col) + { + assert(row < rows_ && col < cols_); + return data_[index(row, col)]; + } + + void swap(unsigned int a, unsigned int b) + { + assert(a < rows_ && a < cols_); + std::swap(swap_[a], swap_[b]); + } + + private: + unsigned int index(unsigned int row, unsigned int col) const + { + return swap_[row] * cols_ + col; + } + + Span data_; + std::vector swap_; + unsigned int rows_; + unsigned int cols_; + }; + + /* + * Matrix inversion using Gaussian elimination. + * + * Start by augmenting the original matrix with an identiy matrix of + * the same size. + */ + std::vector data(dim * dim * 2); + MatrixAccessor matrix(data, dim, dim * 2); + + for (unsigned int i = 0; i < dim; ++i) { + for (unsigned int j = 0; j < dim; ++j) + matrix(i, j) = dataIn[i * dim + j]; + matrix(i, i + dim) = T{ 1 }; + } + + /* Start by triangularizing the input . */ + for (unsigned int pivot = 0; pivot < dim; ++pivot) { + /* + * Locate the next pivot. To improve numerical stability, use + * the row with the largest value in the pivot's column. + */ + unsigned int row; + T maxValue{ 0 }; + + for (unsigned int i = pivot; i < dim; ++i) { + T value = std::abs(matrix(i, pivot)); + if (maxValue < value) { + maxValue = value; + row = i; + } + } + + /* + * If no pivot is found in the column, the matrix is not + * invertible. Return an identity matrix. + */ + if (maxValue == 0) { + std::fill(dataOut.begin(), dataOut.end(), T{ 0 }); + for (unsigned int i = 0; i < dim; ++i) + dataOut[i * dim + i] = T{ 1 }; + return false; + } + + /* Swap rows to bring the pivot in the right location. */ + matrix.swap(pivot, row); + + /* Process all rows below the pivot to zero the pivot column. */ + const T pivotValue = matrix(pivot, pivot); + + for (unsigned int i = pivot + 1; i < dim; ++i) { + const T factor = matrix(i, pivot) / pivotValue; + + /* + * We know the element in the pivot column will be 0, + * hardcode it instead of computing it. + */ + matrix(i, pivot) = T{ 0 }; + + for (unsigned int j = pivot + 1; j < dim * 2; ++j) + matrix(i, j) -= matrix(pivot, j) * factor; + } + } + + /* + * Then diagonalize the input, walking the diagonal backwards. There's + * no need to update the input matrix, as all the values we would write + * in the top-right triangle aren't used in further calculations (and + * would all by definition be zero). + */ + for (unsigned int pivot = dim - 1; pivot > 0; --pivot) { + const T pivotValue = matrix(pivot, pivot); + + for (unsigned int i = 0; i < pivot; ++i) { + const T factor = matrix(i, pivot) / pivotValue; + + for (unsigned int j = dim; j < dim * 2; ++j) + matrix(i, j) -= matrix(pivot, j) * factor; + } + } + + /* + * Finally, normalize the diagonal and store the result in the output + * data. + */ + for (unsigned int i = 0; i < dim; ++i) { + const T factor = matrix(i, i); + + for (unsigned int j = 0; j < dim; ++j) + dataOut[i * dim + j] = matrix(i, j + dim) / factor; + } + + return true; +} + +template bool matrixInvert(Span dataIn, Span dataOut, + unsigned int dim); +template bool matrixInvert(Span data, Span dataOut, + unsigned int dim); + /* * The YAML data shall be a list of numerical values. Its size shall be equal * to the product of the number of rows and columns of the matrix (Rows x From patchwork Wed Mar 19 16:11:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22984 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D28E2C32FE for ; Wed, 19 Mar 2025 16:12:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8C6096895B; Wed, 19 Mar 2025 17:12:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rjbRkjC0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C1606895E for ; Wed, 19 Mar 2025 17:12:15 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 093E355A; Wed, 19 Mar 2025 17:10:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400632; bh=7ui3aA8gW02gVxm/GJdR3NMOLlwcyBYwdtwXO6ui/Xw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rjbRkjC06OCfy21GhC2XXOrAaBbKHjfIrmalyFpWttL4ckp+S4Mbbdm3C+TbumQQk oDZDLvVNrV+3VqieWFbCrrGrbeRo0/Hcp7HOYQ7Jybr6HmVeiNR6vjUzog0vHgmjnA bvZLVkVVRaZKPyCuDHT2MOHNGijFpjfVZP3aNVKw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 06/17] test: Add minimal test for Matrix Date: Wed, 19 Mar 2025 17:11:11 +0100 Message-ID: <20250319161152.63625-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a few tests for the Matrix class. This is not full fledged but at least a starter. Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- test/matrix.cpp | 53 ++++++++++++++++++++++++++++++++++++++++++++++++ test/meson.build | 1 + 2 files changed, 54 insertions(+) create mode 100644 test/matrix.cpp diff --git a/test/matrix.cpp b/test/matrix.cpp new file mode 100644 index 000000000000..93a3b95db2dc --- /dev/null +++ b/test/matrix.cpp @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Vector tests + */ + +#include "libcamera/internal/matrix.h" + +#include +#include + +#include "test.h" + +using namespace libcamera; + +#define ASSERT_EQ(a, b) \ + if ((a) != (b)) { \ + std::cout << #a " != " #b << " (line " << __LINE__ << ")" \ + << std::endl; \ + return TestFail; \ + } + +class VectorTest : public Test +{ +protected: + int run() + { + Matrix m1; + + ASSERT_EQ(m1[0][0], 0.0); + ASSERT_EQ(m1[0][1], 0.0); + + constexpr Matrix m2 = Matrix().identity(); + ASSERT_EQ(m2[0][0], 1.0); + ASSERT_EQ(m2[0][1], 0.0); + ASSERT_EQ(m2[1][0], 0.0); + ASSERT_EQ(m2[1][1], 1.0); + + Matrix m3{ { 2.0, 0.0, 0.0, 2.0 } }; + Matrix m4 = m3.inverse(); + + Matrix m5 = m3 * m4; + ASSERT_EQ(m5[0][0], 1.0); + ASSERT_EQ(m5[0][1], 0.0); + ASSERT_EQ(m5[1][0], 0.0); + ASSERT_EQ(m5[1][1], 1.0); + + return TestPass; + } +}; + +TEST_REGISTER(VectorTest) diff --git a/test/meson.build b/test/meson.build index 4095664994fd..52f04364e4fc 100644 --- a/test/meson.build +++ b/test/meson.build @@ -60,6 +60,7 @@ internal_tests = [ {'name': 'file', 'sources': ['file.cpp']}, {'name': 'flags', 'sources': ['flags.cpp']}, {'name': 'hotplug-cameras', 'sources': ['hotplug-cameras.cpp']}, + {'name': 'matrix', 'sources': ['matrix.cpp']}, {'name': 'message', 'sources': ['message.cpp']}, {'name': 'object', 'sources': ['object.cpp']}, {'name': 'object-delete', 'sources': ['object-delete.cpp']}, From patchwork Wed Mar 19 16:11:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22985 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5C756C32FE for ; Wed, 19 Mar 2025 16:12:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1D02F6895B; Wed, 19 Mar 2025 17:12:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tm/mIn4/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B3026895E for ; Wed, 19 Mar 2025 17:12:17 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7B21D55A; Wed, 19 Mar 2025 17:10:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400634; bh=mhubXdXAgTxwsfjurIPZD4zCKZWHheK7trJwbcR4wEk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tm/mIn4/a4gi8JiVHcLg+WfCp+k888vW+ILWhmc1SEc9sl2ffzoB+lsm0D/px7AqI IGtPNJ+PfSToQd85hjtL9Eo+dt4z3j11O1YnXQQPJod+tKdw7IMKIj3JxIAwU1jIK5 2uQXoH8DpTUEL060w2wq+C9/V0TX5lYwDzPIvUmk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v2 07/17] libcamera: matrix: Extend multiplication operator to heterogenous types Date: Wed, 19 Mar 2025 17:11:12 +0100 Message-ID: <20250319161152.63625-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart It is useful to multiply matrices of heterogneous types, for instance float and double. Extend the multiplication operator to support this, avoiding the need to convert one of the matrices. The type of the returned matrix is selected automatically to avoid loosing precision. Signed-off-by: Laurent Pinchart --- Changes in v2: - Added this patch --- include/libcamera/internal/matrix.h | 10 ++++++---- src/libcamera/matrix.cpp | 5 +++-- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/matrix.h b/include/libcamera/internal/matrix.h index 6e3c190286fe..688c6fd498f3 100644 --- a/include/libcamera/internal/matrix.h +++ b/include/libcamera/internal/matrix.h @@ -8,6 +8,7 @@ #include #include +#include #include #include @@ -129,15 +130,16 @@ constexpr Matrix operator*(const Matrix &m, T d) return d * m; } -template -constexpr Matrix operator*(const Matrix &m1, const Matrix &m2) +template +constexpr Matrix, R1, C2> operator*(const Matrix &m1, + const Matrix &m2) { static_assert(C1 == R2, "Matrix dimensions must match for multiplication"); - Matrix result; + Matrix, R1, C2> result; for (unsigned int i = 0; i < R1; i++) { for (unsigned int j = 0; j < C2; j++) { - T sum = 0; + std::common_type_t sum = 0; for (unsigned int k = 0; k < C1; k++) sum += m1[i][k] * m2[k][j]; diff --git a/src/libcamera/matrix.cpp b/src/libcamera/matrix.cpp index 8590f8efeff3..f9117357dc7e 100644 --- a/src/libcamera/matrix.cpp +++ b/src/libcamera/matrix.cpp @@ -138,11 +138,12 @@ LOG_DEFINE_CATEGORY(Matrix) */ /** - * \fn Matrix operator*(const Matrix &m1, const Matrix &m2) + * \fn operator*(const Matrix &m1, const Matrix &m2) * \brief Matrix multiplication - * \tparam T Type of numerical values in the matrices + * \tparam T1 Type of numerical values in the first matrix * \tparam R1 Number of rows in the first matrix * \tparam C1 Number of columns in the first matrix + * \tparam T2 Type of numerical values in the secont matrix * \tparam R2 Number of rows in the second matrix * \tparam C2 Number of columns in the second matrix * \param m1 Multiplicand matrix From patchwork Wed Mar 19 16:11:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22986 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E45ACC32FE for ; Wed, 19 Mar 2025 16:12:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 97A5568964; Wed, 19 Mar 2025 17:12:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VjCBjuu2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DB9468962 for ; Wed, 19 Mar 2025 17:12:20 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 569368FA; Wed, 19 Mar 2025 17:10:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400637; bh=br8rN+1GhOXoViyWSjCXtOrJsJ/Gofo1vO1Isf1ZNqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VjCBjuu2ESnSqR/W1VH5VB4waiZhrqCRDVXpnvp/zZXiRHer1m1nt1OXVyqsPt8hm pNwkBVfquzNgnrRM4HKtsM5G3xGCEvDhbaWjDeX66pnDrEMpJqIIMVUEPkYIpuGqTJ T2lGGodi4l2c1fX3gX+IX+Y7wvpPORDvV8NR0VBc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v2 08/17] libcamera: vector: Extend matrix multiplication operator to heterogenous types Date: Wed, 19 Mar 2025 17:11:13 +0100 Message-ID: <20250319161152.63625-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart It is useful to multiply matrices and vectors of heterogeneous types, for instance float and double. Extend the multiplication operator to support this, avoiding the need to convert one of the operations. The type of the returned vector is selected automatically to avoid loosing precision. Signed-off-by: Laurent Pinchart Signed-off-by: Stefan Klug Changes in v2: - Added this patch --- include/libcamera/internal/vector.h | 9 +++++---- src/libcamera/vector.cpp | 5 +++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/vector.h b/include/libcamera/internal/vector.h index 66cc5ac988c2..d518de9689a3 100644 --- a/include/libcamera/internal/vector.h +++ b/include/libcamera/internal/vector.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -295,13 +296,13 @@ private: template using RGB = Vector; -template -Vector operator*(const Matrix &m, const Vector &v) +template +Vector, Rows> operator*(const Matrix &m, const Vector &v) { - Vector result; + Vector, Rows> result; for (unsigned int i = 0; i < Rows; i++) { - T sum = 0; + std::common_type_t sum = 0; for (unsigned int j = 0; j < Cols; j++) sum += m[i][j] * v[j]; result[i] = sum; diff --git a/src/libcamera/vector.cpp b/src/libcamera/vector.cpp index 435f2fc62f8b..c9419da8c663 100644 --- a/src/libcamera/vector.cpp +++ b/src/libcamera/vector.cpp @@ -308,9 +308,10 @@ LOG_DEFINE_CATEGORY(Vector) */ /** - * \fn Vector operator*(const Matrix &m, const Vector &v) + * \fn operator*(const Matrix &m, const Vector &v) * \brief Multiply a matrix by a vector - * \tparam T Numerical type of the contents of the matrix and vector + * \tparam T Numerical type of the contents of the matrix + * \tparam U Numerical type of the contents of the vector * \tparam Rows The number of rows in the matrix * \tparam Cols The number of columns in the matrix (= rows in the vector) * \param m The matrix From patchwork Wed Mar 19 16:11:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22987 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AC2ECC32FE for ; Wed, 19 Mar 2025 16:12:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6519968967; Wed, 19 Mar 2025 17:12:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CRU7mRDz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D0BE68962 for ; Wed, 19 Mar 2025 17:12:23 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8758C8FA; Wed, 19 Mar 2025 17:10:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400640; bh=wLcGUSOBZBOfcfD9gdaPSiufSRlJxTeXCp+gmRkIErE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CRU7mRDzqulBnp+sUQzmYVFSGctyvfdIF2YyABitGi8Ic8LSRYqvqA0iGlhnOw2M1 mv9tB6bWMveyLZvbcDUlh3BmDUVYb9hexvLVezPzhS2958wIIzsRcEnLa0sOO7VLhP tHh7tyntGe/5pUTmD2XA2eX2gInOSC/uQZPrNRdo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 09/17] ipa: rkisp1: Refactor automatic/manual structure in IPAActiveState Date: Wed, 19 Mar 2025 17:11:14 +0100 Message-ID: <20250319161152.63625-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Swap gains and automatic/manual in the IPAActiveState structure. This is in preparation to adding another member, which is easier in the new structure. The patch contains no functional changes. Signed-off-by: Stefan Klug --- Changes in v2: - Use one named struct instead of two anonymous ones --- src/ipa/rkisp1/algorithms/awb.cpp | 24 ++++++++++++------------ src/ipa/rkisp1/ipa_context.h | 10 ++++++---- 2 files changed, 18 insertions(+), 16 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index eafe93081bb1..a9759e53f593 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -124,8 +124,8 @@ int Awb::init(IPAContext &context, const YamlObject &tuningData) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.activeState.awb.gains.manual = RGB{ 1.0 }; - context.activeState.awb.gains.automatic = + context.activeState.awb.manual.gains = RGB{ 1.0 }; + context.activeState.awb.automatic.gains = awbAlgo_->gainsFromColourTemperature(kDefaultColourTemperature); context.activeState.awb.autoEnabled = true; context.activeState.awb.temperatureK = kDefaultColourTemperature; @@ -173,8 +173,8 @@ void Awb::queueRequest(IPAContext &context, const auto &colourTemperature = controls.get(controls::ColourTemperature); bool update = false; if (colourGains) { - awb.gains.manual.r() = (*colourGains)[0]; - awb.gains.manual.b() = (*colourGains)[1]; + awb.manual.gains.r() = (*colourGains)[0]; + awb.manual.gains.b() = (*colourGains)[1]; /* * \todo Colour temperature reported in metadata is now * incorrect, as we can't deduce the temperature from the gains. @@ -183,17 +183,17 @@ void Awb::queueRequest(IPAContext &context, update = true; } else if (colourTemperature) { const auto &gains = awbAlgo_->gainsFromColourTemperature(*colourTemperature); - awb.gains.manual.r() = gains.r(); - awb.gains.manual.b() = gains.b(); + awb.manual.gains.r() = gains.r(); + awb.manual.gains.b() = gains.b(); awb.temperatureK = *colourTemperature; update = true; } if (update) LOG(RkISP1Awb, Debug) - << "Set colour gains to " << awb.gains.manual; + << "Set colour gains to " << awb.manual.gains; - frameContext.awb.gains = awb.gains.manual; + frameContext.awb.gains = awb.manual.gains; frameContext.awb.temperatureK = awb.temperatureK; } @@ -208,7 +208,7 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, * most up-to-date automatic values we can read. */ if (frameContext.awb.autoEnabled) { - frameContext.awb.gains = context.activeState.awb.gains.automatic; + frameContext.awb.gains = context.activeState.awb.automatic.gains; frameContext.awb.temperatureK = context.activeState.awb.temperatureK; } @@ -325,14 +325,14 @@ void Awb::process(IPAContext &context, /* Filter the values to avoid oscillations. */ double speed = 0.2; awbResult.gains = awbResult.gains * speed + - activeState.awb.gains.automatic * (1 - speed); + activeState.awb.automatic.gains * (1 - speed); - activeState.awb.gains.automatic = awbResult.gains; + activeState.awb.automatic.gains = awbResult.gains; LOG(RkISP1Awb, Debug) << std::showpoint << "Means " << rgbMeans << ", gains " - << activeState.awb.gains.automatic << ", temp " + << activeState.awb.automatic.gains << ", temp " << activeState.awb.temperatureK << "K"; } diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 474f7036f003..6bc922a82971 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -89,10 +89,12 @@ struct IPAActiveState { } agc; struct { - struct { - RGB manual; - RGB automatic; - } gains; + struct AwbState { + RGB gains; + }; + + AwbState manual; + AwbState automatic; unsigned int temperatureK; bool autoEnabled; From patchwork Wed Mar 19 16:11:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22988 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 39C7FC32FE for ; Wed, 19 Mar 2025 16:12:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E643168966; Wed, 19 Mar 2025 17:12:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BiCOAwWQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 12EE668962 for ; Wed, 19 Mar 2025 17:12:27 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2726E55A; Wed, 19 Mar 2025 17:10:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400644; bh=7yqQmpCCDzAK3bhr7VJjJZpMNjdv/M+Z6prXZurl3NY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BiCOAwWQj1mXbkFfHuWH07j+oJ6Hzkpc0Ktq1pmZZQ6wP/+gN0LLV3T2YuMpe0KP5 u1BGP9lkKLjUooTg9rMYp8WTcHgMJenzf+Ybp0QSO40n6/p5G7mQw54AZlcvlZuQnl +D0yJMLjePkGUedojvqWomZ88l4h2lqLvuR0s894= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 10/17] ipa: rkisp1: ccm/lsc: Fix CCM/LSC based on manual color temperature Date: Wed, 19 Mar 2025 17:11:15 +0100 Message-ID: <20250319161152.63625-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In RkISP1Awb::process(), the color temperature in the active state is updated every time new statistics are available. The CCM/LSC algorithms use that value in prepare() to update the CCM/LSC. This is not correct if the color temperature was specified manually and leads to visible flicker even when AwbEnable is set to false. To fix that, track the auto and manual color temperature separately in active state. In Awb::prepare() the current frame context is updated with the corresponding value from active state. Change the algorithms to fetch the color temperature from the frame context instead of the active state in prepare(). Fixes: 02308809548d ("ipa: rkisp1: awb: Implement ColourTemperature control") Signed-off-by: Stefan Klug --- Changes in v2: - None --- src/ipa/rkisp1/algorithms/awb.cpp | 18 ++++++++++-------- src/ipa/rkisp1/algorithms/ccm.cpp | 2 +- src/ipa/rkisp1/algorithms/lsc.cpp | 6 +++--- src/ipa/rkisp1/ipa_context.h | 2 +- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index a9759e53f593..5e067e50cd52 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -128,7 +128,8 @@ int Awb::configure(IPAContext &context, context.activeState.awb.automatic.gains = awbAlgo_->gainsFromColourTemperature(kDefaultColourTemperature); context.activeState.awb.autoEnabled = true; - context.activeState.awb.temperatureK = kDefaultColourTemperature; + context.activeState.awb.manual.temperatureK = kDefaultColourTemperature; + context.activeState.awb.automatic.temperatureK = kDefaultColourTemperature; /* * Define the measurement window for AWB as a centered rectangle @@ -185,7 +186,7 @@ void Awb::queueRequest(IPAContext &context, const auto &gains = awbAlgo_->gainsFromColourTemperature(*colourTemperature); awb.manual.gains.r() = gains.r(); awb.manual.gains.b() = gains.b(); - awb.temperatureK = *colourTemperature; + awb.manual.temperatureK = *colourTemperature; update = true; } @@ -194,7 +195,7 @@ void Awb::queueRequest(IPAContext &context, << "Set colour gains to " << awb.manual.gains; frameContext.awb.gains = awb.manual.gains; - frameContext.awb.temperatureK = awb.temperatureK; + frameContext.awb.temperatureK = awb.manual.temperatureK; } /** @@ -208,8 +209,9 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, * most up-to-date automatic values we can read. */ if (frameContext.awb.autoEnabled) { - frameContext.awb.gains = context.activeState.awb.automatic.gains; - frameContext.awb.temperatureK = context.activeState.awb.temperatureK; + auto &awb = context.activeState.awb; + frameContext.awb.gains = awb.automatic.gains; + frameContext.awb.temperatureK = awb.automatic.temperatureK; } auto gainConfig = params->block(); @@ -309,10 +311,10 @@ void Awb::process(IPAContext &context, RkISP1AwbStats awbStats{ rgbMeans }; AwbResult awbResult = awbAlgo_->calculateAwb(awbStats, frameContext.lux.lux); - activeState.awb.temperatureK = awbResult.colourTemperature; + activeState.awb.automatic.temperatureK = awbResult.colourTemperature; /* Metadata shall contain the up to date measurement */ - metadata.set(controls::ColourTemperature, activeState.awb.temperatureK); + metadata.set(controls::ColourTemperature, activeState.awb.automatic.temperatureK); /* * Clamp the gain values to the hardware, which expresses gains as Q2.8 @@ -333,7 +335,7 @@ void Awb::process(IPAContext &context, << std::showpoint << "Means " << rgbMeans << ", gains " << activeState.awb.automatic.gains << ", temp " - << activeState.awb.temperatureK << "K"; + << activeState.awb.automatic.temperatureK << "K"; } RGB Awb::calculateRgbMeans(const IPAFrameContext &frameContext, const rkisp1_cif_isp_awb_stat *awb) const diff --git a/src/ipa/rkisp1/algorithms/ccm.cpp b/src/ipa/rkisp1/algorithms/ccm.cpp index eb8ca39e56a8..2e5e91006b55 100644 --- a/src/ipa/rkisp1/algorithms/ccm.cpp +++ b/src/ipa/rkisp1/algorithms/ccm.cpp @@ -88,7 +88,7 @@ void Ccm::setParameters(struct rkisp1_cif_isp_ctk_config &config, void Ccm::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) { - uint32_t ct = context.activeState.awb.temperatureK; + uint32_t ct = frameContext.awb.temperatureK; /* * \todo The colour temperature will likely be noisy, add filtering to diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index e47aa2f0727e..e7301bfec863 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -404,12 +404,12 @@ void LensShadingCorrection::copyTable(rkisp1_cif_isp_lsc_config &config, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void LensShadingCorrection::prepare(IPAContext &context, +void LensShadingCorrection::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, RkISP1Params *params) { - uint32_t ct = context.activeState.awb.temperatureK; + uint32_t ct = frameContext.awb.temperatureK; if (std::abs(static_cast(ct) - static_cast(lastAppliedCt_)) < kColourTemperatureChangeThreshhold) return; diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 6bc922a82971..769e9f114e23 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -91,12 +91,12 @@ struct IPAActiveState { struct { struct AwbState { RGB gains; + unsigned int temperatureK; }; AwbState manual; AwbState automatic; - unsigned int temperatureK; bool autoEnabled; } awb; From patchwork Wed Mar 19 16:11:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22989 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E400DC32FE for ; Wed, 19 Mar 2025 16:12:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 94A386896C; Wed, 19 Mar 2025 17:12:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LIdCnPre"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 57C3F68962 for ; Wed, 19 Mar 2025 17:12:29 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 777F455A; Wed, 19 Mar 2025 17:10:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400646; bh=96p0ZlNnEnooA92EeItd8ynm89pE3xwAD+FvHp4hNtc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LIdCnPre+r2CnJQAYNm/rmiaSGzmBzmPpK9XR1bTEH2bxOpVhdQZC+VuF/6KZO6Xi +JstepYqV4qOWGZhM6tG0P03wJ+lSQAefxL1buRmPl0nhza5MZTi0BUnrj4y4UWvT0 e1w3aw2nPCDQKBB2ZM0ocFBYZiLpyRtPgvKRumPI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 11/17] ipa: rkisp1: Implement manual ColourCorrectionMatrix control Date: Wed, 19 Mar 2025 17:11:16 +0100 Message-ID: <20250319161152.63625-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a manual ColourCorrectionMatrix control. This was already discussed while implementing manual colour temperature but was never implemented. The control allows to manually specify the CCM when AwbEnable is false. Signed-off-by: Stefan Klug --- Changes in v2: - None --- src/ipa/rkisp1/algorithms/ccm.cpp | 59 ++++++++++++++++++++++++++++--- src/ipa/rkisp1/algorithms/ccm.h | 6 ++++ src/ipa/rkisp1/ipa_context.h | 3 +- 3 files changed, 62 insertions(+), 6 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/ccm.cpp b/src/ipa/rkisp1/algorithms/ccm.cpp index 2e5e91006b55..303ac3dd2fe2 100644 --- a/src/ipa/rkisp1/algorithms/ccm.cpp +++ b/src/ipa/rkisp1/algorithms/ccm.cpp @@ -36,17 +36,25 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Ccm) +constexpr Matrix kIdentity3x3 = Matrix::identity(); + /** * \copydoc libcamera::ipa::Algorithm::init */ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) { + auto &cmap = context.ctrlMap; + cmap[&controls::ColourCorrectionMatrix] = ControlInfo( + ControlValue(-8.0f), + ControlValue(7.993f), + ControlValue(kIdentity3x3.data())); + int ret = ccm_.readYaml(tuningData["ccms"], "ct", "ccm"); if (ret < 0) { LOG(RkISP1Ccm, Warning) << "Failed to parse 'ccm' " << "parameter from tuning file; falling back to unit matrix"; - ccm_.setData({ { 0, Matrix::identity() } }); + ccm_.setData({ { 0, kIdentity3x3 } }); } ret = offsets_.readYaml(tuningData["ccms"], "ct", "offsets"); @@ -61,13 +69,48 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData return 0; } +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + auto &as = context.activeState; + as.ccm.manual = kIdentity3x3; + as.ccm.automatic = ccm_.getInterpolated(as.awb.automatic.temperatureK); + LOG(RkISP1Ccm, Debug) << "init matrix " << as.ccm.manual; + return 0; +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* Nothing to do here, the ccm will be calculated in prepare() */ + if (frameContext.awb.autoEnabled) + return; + + auto &ccm = context.activeState.ccm; + + const auto &colourTemperature = controls.get(controls::ColourTemperature); + const auto &ccmMatrix = controls.get(controls::ColourCorrectionMatrix); + if (ccmMatrix) + ccm.manual = Matrix(*ccmMatrix); + else if (colourTemperature) + ccm.manual = ccm_.getInterpolated(*colourTemperature); + + LOG(RkISP1Ccm, Debug) << "queueRequest matrix " << ccm.manual; + frameContext.ccm.ccm = ccm.manual; +} + void Ccm::setParameters(struct rkisp1_cif_isp_ctk_config &config, const Matrix &matrix, const Matrix &offsets) { /* * 4 bit integer and 7 bit fractional, ranging from -8 (0x400) to - * +7.992 (0x3ff) + * +7.9921875 (0x3ff) */ for (unsigned int i = 0; i < 3; i++) { for (unsigned int j = 0; j < 3; j++) @@ -88,14 +131,20 @@ void Ccm::setParameters(struct rkisp1_cif_isp_ctk_config &config, void Ccm::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) { - uint32_t ct = frameContext.awb.temperatureK; + if (!frameContext.awb.autoEnabled) { + auto config = params->block(); + config.setEnabled(true); + setParameters(*config, frameContext.ccm.ccm, Matrix()); + return; + } + uint32_t ct = frameContext.awb.temperatureK; /* * \todo The colour temperature will likely be noisy, add filtering to * avoid updating the CCM matrix all the time. */ if (frame > 0 && ct == ct_) { - frameContext.ccm.ccm = context.activeState.ccm.ccm; + frameContext.ccm.ccm = context.activeState.ccm.automatic; return; } @@ -103,7 +152,7 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame, Matrix ccm = ccm_.getInterpolated(ct); Matrix offsets = offsets_.getInterpolated(ct); - context.activeState.ccm.ccm = ccm; + context.activeState.ccm.automatic = ccm; frameContext.ccm.ccm = ccm; auto config = params->block(); diff --git a/src/ipa/rkisp1/algorithms/ccm.h b/src/ipa/rkisp1/algorithms/ccm.h index a5d9a9a45e5d..c301e6e531c8 100644 --- a/src/ipa/rkisp1/algorithms/ccm.h +++ b/src/ipa/rkisp1/algorithms/ccm.h @@ -26,6 +26,12 @@ public: ~Ccm() = default; int init(IPAContext &context, const YamlObject &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) override; diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 769e9f114e23..f0d504215d34 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -101,7 +101,8 @@ struct IPAActiveState { } awb; struct { - Matrix ccm; + Matrix manual; + Matrix automatic; } ccm; struct { From patchwork Wed Mar 19 16:11:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22990 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9005AC32FE for ; Wed, 19 Mar 2025 16:12:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 43F4568962; Wed, 19 Mar 2025 17:12:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CSwOuKgB"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AC7E6896E for ; Wed, 19 Mar 2025 17:12:32 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 521888FA; Wed, 19 Mar 2025 17:10:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400649; bh=sNWizC2Z4QwnejZ9TRAllyVdNWR0kF74gIlyotl2gH0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CSwOuKgBAgJ+nxvLO1/8JdShflBg7K5XI6E+TVz1jATYdubwZhjbrsOXHJAXpvGXW k4m497ff8BF6eRbPvMRiirLYFT9JFmJ1KiZbV37+3OdzQ+gtuUxAR0M0n3UBkJ26eH i5++588fkPBWllVm7hhZ3qrKLIcou1drsXt45xTg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 12/17] libipa: awb: Make result of gainsFromColourTemp optional Date: Wed, 19 Mar 2025 17:11:17 +0100 Message-ID: <20250319161152.63625-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the grey world AWB case, if no colour gains are contained in the tuning file, the color gains get reset to 1 when the colour temperature is set manually. This is unexpected and undesirable. Allow the gainsFromColourTemp() function to return a std::nullopt to handle that case. Signed-off-by: Stefan Klug --- Changes in v2: - Added this patch --- src/ipa/libipa/awb.cpp | 2 +- src/ipa/libipa/awb.h | 2 +- src/ipa/libipa/awb_bayes.cpp | 4 ++-- src/ipa/libipa/awb_bayes.h | 2 +- src/ipa/libipa/awb_grey.cpp | 6 +++--- src/ipa/libipa/awb_grey.h | 2 +- src/ipa/rkisp1/algorithms/awb.cpp | 16 +++++++++++----- 7 files changed, 20 insertions(+), 14 deletions(-) diff --git a/src/ipa/libipa/awb.cpp b/src/ipa/libipa/awb.cpp index 925fac232709..214bac8b56a6 100644 --- a/src/ipa/libipa/awb.cpp +++ b/src/ipa/libipa/awb.cpp @@ -114,7 +114,7 @@ namespace ipa { * does not take any statistics into account. It is used to compute the colour * gains when the user manually specifies a colour temperature. * - * \return The colour gains + * \return The colour gains or std::nullopt if the conversion is not possible */ /** diff --git a/src/ipa/libipa/awb.h b/src/ipa/libipa/awb.h index 4bab7a451ce5..789a1d534f9a 100644 --- a/src/ipa/libipa/awb.h +++ b/src/ipa/libipa/awb.h @@ -39,7 +39,7 @@ public: virtual int init(const YamlObject &tuningData) = 0; virtual AwbResult calculateAwb(const AwbStats &stats, unsigned int lux) = 0; - virtual RGB gainsFromColourTemperature(double colourTemperature) = 0; + virtual std::optional> gainsFromColourTemperature(double colourTemperature) = 0; const ControlInfoMap::Map &controls() const { diff --git a/src/ipa/libipa/awb_bayes.cpp b/src/ipa/libipa/awb_bayes.cpp index d1d0eaf0e68f..d2bcbd83d7f8 100644 --- a/src/ipa/libipa/awb_bayes.cpp +++ b/src/ipa/libipa/awb_bayes.cpp @@ -270,7 +270,7 @@ void AwbBayes::handleControls(const ControlList &controls) } } -RGB AwbBayes::gainsFromColourTemperature(double colourTemperature) +std::optional> AwbBayes::gainsFromColourTemperature(double colourTemperature) { /* * \todo In the RaspberryPi code, the ct curve was interpolated in @@ -278,7 +278,7 @@ RGB AwbBayes::gainsFromColourTemperature(double colourTemperature) * intuitive, as the gains are in linear space. But I can't prove it. */ const auto &gains = colourGainCurve_.getInterpolated(colourTemperature); - return { { gains[0], 1.0, gains[1] } }; + return RGB{ { gains[0], 1.0, gains[1] } }; } AwbResult AwbBayes::calculateAwb(const AwbStats &stats, unsigned int lux) diff --git a/src/ipa/libipa/awb_bayes.h b/src/ipa/libipa/awb_bayes.h index bb933038d6d2..47ef3cce4d58 100644 --- a/src/ipa/libipa/awb_bayes.h +++ b/src/ipa/libipa/awb_bayes.h @@ -27,7 +27,7 @@ public: int init(const YamlObject &tuningData) override; AwbResult calculateAwb(const AwbStats &stats, unsigned int lux) override; - RGB gainsFromColourTemperature(double temperatureK) override; + std::optional> gainsFromColourTemperature(double temperatureK) override; void handleControls(const ControlList &controls) override; private: diff --git a/src/ipa/libipa/awb_grey.cpp b/src/ipa/libipa/awb_grey.cpp index d3d2132b8869..d252edb2b6c6 100644 --- a/src/ipa/libipa/awb_grey.cpp +++ b/src/ipa/libipa/awb_grey.cpp @@ -98,15 +98,15 @@ AwbResult AwbGrey::calculateAwb(const AwbStats &stats, [[maybe_unused]] unsigned * \return The colour gains if a colour temperature curve is available, * [1, 1, 1] otherwise. */ -RGB AwbGrey::gainsFromColourTemperature(double colourTemperature) +std::optional> AwbGrey::gainsFromColourTemperature(double colourTemperature) { if (!colourGainCurve_) { LOG(Awb, Error) << "No gains defined"; - return RGB({ 1.0, 1.0, 1.0 }); + return std::nullopt; } auto gains = colourGainCurve_->getInterpolated(colourTemperature); - return { { gains[0], 1.0, gains[1] } }; + return RGB{ { gains[0], 1.0, gains[1] } }; } } /* namespace ipa */ diff --git a/src/ipa/libipa/awb_grey.h b/src/ipa/libipa/awb_grey.h index 7ec7bfa5da9a..f82a368d11cc 100644 --- a/src/ipa/libipa/awb_grey.h +++ b/src/ipa/libipa/awb_grey.h @@ -25,7 +25,7 @@ public: int init(const YamlObject &tuningData) override; AwbResult calculateAwb(const AwbStats &stats, unsigned int lux) override; - RGB gainsFromColourTemperature(double colourTemperature) override; + std::optional> gainsFromColourTemperature(double colourTemperature) override; private: std::optional>> colourGainCurve_; diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 5e067e50cd52..58b8370d2c61 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -125,8 +125,12 @@ int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { context.activeState.awb.manual.gains = RGB{ 1.0 }; - context.activeState.awb.automatic.gains = - awbAlgo_->gainsFromColourTemperature(kDefaultColourTemperature); + auto gains = awbAlgo_->gainsFromColourTemperature(kDefaultColourTemperature); + if (gains) + context.activeState.awb.automatic.gains = *gains; + else + context.activeState.awb.automatic.gains = RGB{ 1.0 }; + context.activeState.awb.autoEnabled = true; context.activeState.awb.manual.temperatureK = kDefaultColourTemperature; context.activeState.awb.automatic.temperatureK = kDefaultColourTemperature; @@ -184,10 +188,12 @@ void Awb::queueRequest(IPAContext &context, update = true; } else if (colourTemperature) { const auto &gains = awbAlgo_->gainsFromColourTemperature(*colourTemperature); - awb.manual.gains.r() = gains.r(); - awb.manual.gains.b() = gains.b(); + if (gains) { + awb.manual.gains.r() = gains->r(); + awb.manual.gains.b() = gains->b(); + update = true; + } awb.manual.temperatureK = *colourTemperature; - update = true; } if (update) From patchwork Wed Mar 19 16:11:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22991 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 17412C32FE for ; Wed, 19 Mar 2025 16:12:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CD5C66896A; Wed, 19 Mar 2025 17:12:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HfdBn6p9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6933668954 for ; Wed, 19 Mar 2025 17:12:35 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 833708FA; Wed, 19 Mar 2025 17:10:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400652; bh=d8mMAhwL3xlqhrLmdBVsg08EjU9FZJx2JwYHYc8fttI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HfdBn6p9RW/Mvk7FzoYqzH687g3iuWz9kC557RYYFChpeUnsYMzm05kUEJatGDVSx jIT95CgLHZQvb0eibGHgC52/zbBXNtEja3qyzr8YSSjTs9G5bzV2TXIjHwDxnBoMt8 W4vTdZKrPu7MnhIewnZRte+p4wZpXPcA1SGLV56g= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 13/17] ipa: rkisp1: awb: Use gains from color temperature curve Date: Wed, 19 Mar 2025 17:11:18 +0100 Message-ID: <20250319161152.63625-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the color gains for white balance if calibrated light sources are contained in the tuning file. The results are generally better especially when large uniformly colored objects are in the scene. Pure grey world model is still available by removing the color gains from the tuning file. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/awb.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 58b8370d2c61..286d9e3e2018 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -218,6 +218,16 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, auto &awb = context.activeState.awb; frameContext.awb.gains = awb.automatic.gains; frameContext.awb.temperatureK = awb.automatic.temperatureK; + + const auto &gains = awbAlgo_->gainsFromColourTemperature( + awb.automatic.temperatureK); + if (gains) { + frameContext.awb.gains.r() = gains->r(); + frameContext.awb.gains.g() = 1.0; + frameContext.awb.gains.b() = gains->b(); + } else { + frameContext.awb.gains = awb.automatic.gains; + } } auto gainConfig = params->block(); From patchwork Wed Mar 19 16:11:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22992 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 01CF3C32FE for ; Wed, 19 Mar 2025 16:12:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A28C368966; Wed, 19 Mar 2025 17:12:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b1OkpFGy"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1061168954 for ; Wed, 19 Mar 2025 17:12:39 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 243E48FA; Wed, 19 Mar 2025 17:10:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400656; bh=V+XkW0IfwfRCMKPanyqHk8nA3AJV88hvfnsN9o3LBxQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b1OkpFGyW/wz754mc8z+7dSIoS38gNpfn9O5hY6Qp8ajPyCGGjrWpdtDACZ/pwZiW 50KJlAmCzU3de2xhJenoudQko0xl4+DcG3riSbZ8gN4JIMfeI5loICveu9dlceElLm Ba0TQgR+P28Q+rlyspIom90OqZdOtjhYIJJs9X+A= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v2 14/17] ipa: rkisp1: Damp color temperature regulation Date: Wed, 19 Mar 2025 17:11:19 +0100 Message-ID: <20250319161152.63625-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Damp the regulation of the color temperature with the same factor as the gains. Not damping the color temperature leads to visible flicker, as the CCM changes too much. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v2: - Collected tags --- src/ipa/rkisp1/algorithms/awb.cpp | 3 +++ src/ipa/rkisp1/algorithms/ccm.cpp | 4 ---- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 286d9e3e2018..149c6aa59c8d 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -342,9 +342,12 @@ void Awb::process(IPAContext &context, /* Filter the values to avoid oscillations. */ double speed = 0.2; + double ct = estimateCCT(rgbMeans); + ct = ct * speed + activeState.awb.automatic.temperatureK * (1 - speed); awbResult.gains = awbResult.gains * speed + activeState.awb.automatic.gains * (1 - speed); + activeState.awb.automatic.temperatureK = static_cast(ct); activeState.awb.automatic.gains = awbResult.gains; LOG(RkISP1Awb, Debug) diff --git a/src/ipa/rkisp1/algorithms/ccm.cpp b/src/ipa/rkisp1/algorithms/ccm.cpp index 303ac3dd2fe2..b7f32f0d5d8a 100644 --- a/src/ipa/rkisp1/algorithms/ccm.cpp +++ b/src/ipa/rkisp1/algorithms/ccm.cpp @@ -139,10 +139,6 @@ void Ccm::prepare(IPAContext &context, const uint32_t frame, } uint32_t ct = frameContext.awb.temperatureK; - /* - * \todo The colour temperature will likely be noisy, add filtering to - * avoid updating the CCM matrix all the time. - */ if (frame > 0 && ct == ct_) { frameContext.ccm.ccm = context.activeState.ccm.automatic; return; From patchwork Wed Mar 19 16:11:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22993 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93E39C32FE for ; Wed, 19 Mar 2025 16:12:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 49A0368970; Wed, 19 Mar 2025 17:12:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oEoiL6Rs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 673836895B for ; Wed, 19 Mar 2025 17:12:41 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7BB2655A; Wed, 19 Mar 2025 17:10:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400658; bh=NxJ8FBTXkqXAEwUUlhcK1v+TxRRXWgFK/YjME/mdJFA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oEoiL6RseziHCvvxVDqPORS6BncQSePkienjIQa1hKVsbUz2+SJdGuP6GUNrJZBEs UeArT7PlSAnMMI4e5crlDsmvnttcIuWgmBMu1u9Dvw9xrNlOEN1iHx/lDo4MtvToTG BUITHFBGSIEsSp8Zfi56laoa9GmnquZRxanyIfTc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 15/17] ipa: rkisp1: awb: Set rgb mean limits to 180 Date: Wed, 19 Mar 2025 17:11:20 +0100 Message-ID: <20250319161152.63625-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Set the mean limits to 180 to get reasonably good results from the rgb estimations. This is only for internal development as there is no code path to enable rgb estimations at runtime. Signed-off-by: Stefan Klug --- Changes in v2: - None --- src/ipa/rkisp1/algorithms/awb.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 149c6aa59c8d..324bc14b42a0 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -261,9 +261,9 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, * awb_ref_cr, awb_min_y and awb_ref_cb respectively. The other * values are not used, set them to 0. */ - awbConfig->awb_ref_cr = 250; - awbConfig->min_y = 250; - awbConfig->awb_ref_cb = 250; + awbConfig->awb_ref_cr = 180; + awbConfig->min_y = 180; + awbConfig->awb_ref_cb = 180; awbConfig->max_y = 0; awbConfig->min_c = 0; From patchwork Wed Mar 19 16:11:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22994 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 12DACC32FE for ; Wed, 19 Mar 2025 16:12:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C454368976; Wed, 19 Mar 2025 17:12:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VuLcKH+w"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 42FFA68974 for ; Wed, 19 Mar 2025 17:12:44 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5195A8FA; Wed, 19 Mar 2025 17:11:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400661; bh=SYgo4zWmdHpgF5xlz5lNmREp87ksn1IFDvh0ws3M3aE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VuLcKH+wBQDHwfx4i1X2zeJdTgiDsZL7TjPij1jebuyuG/1J00xvk2oBxWDd7BeNY t0KCiKPjY2MvtxykwDeOV75JjPtqOpHqrEXHUx4Mnonkymbvw6bgeRfen/7IAaxcTp dsR67dRUvCxW4ttmnpANN3nBMsN2wigLsTa35DEw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v2 16/17] ipa: rkisp1: awb: Take the CCM into account for the AWB gains calculation Date: Wed, 19 Mar 2025 17:11:21 +0100 Message-ID: <20250319161152.63625-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB measurements are taken after the CCM. This can be seen by enabling debug logging on AWB, disabling AWB (stats will still be processed) and manually chaning the CCM. This means that the estimated colour temperature and the corresponding CCM also lead to changed rgbMeans which in turn leads to oscillations. Fix that by applying the inverse transform on the rgbMeans. Signed-off-by: Stefan Klug --- Changes in v2: - Improved commit message - Added comment in the code --- src/ipa/rkisp1/algorithms/awb.cpp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 324bc14b42a0..e8b04d03748d 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -412,6 +412,12 @@ RGB Awb::calculateRgbMeans(const IPAFrameContext &frameContext, const rk rgbMeans = rgbMeans.max(0.0); } + /* + * The ISP computes the AWB means after applying the ccm. Apply the + * inverse as we want to get the raw means before the colour gains. + */ + rgbMeans = frameContext.ccm.ccm.inverse() * rgbMeans; + /* * The ISP computes the AWB means after applying the colour gains, * divide by the gains that were used to get the raw means from the From patchwork Wed Mar 19 16:11:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22995 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8FE72C32FE for ; Wed, 19 Mar 2025 16:12:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D2A968967; Wed, 19 Mar 2025 17:12:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JQ+bZB6q"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6BAEC68967 for ; Wed, 19 Mar 2025 17:12:47 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:760:e5ca:4814:99c7]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 840408FA; Wed, 19 Mar 2025 17:11:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1742400664; bh=NKZRysVlMDp+4JBntlGKrY42fUrM7otHIIeC2HGtXyc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JQ+bZB6qAlIX9lsMjjg36YoJpDnny6MTF7SAiaJE0VHS+XwE/QVaM2nxvnGJIPrxu Yu51q6runL0pQXQvOwUVNcEgVO7CR05CWGNccKuOAq3UyJAediHlUt5VhOoVqZm0Qi +QqlN9UJmJPXqe75o2QI1RCn+7PUaxRBYYJgStVQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v2 17/17] ipa: rkisp1: awb: Avoid division by zero Date: Wed, 19 Mar 2025 17:11:22 +0100 Message-ID: <20250319161152.63625-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250319161152.63625-1-stefan.klug@ideasonboard.com> References: <20250319161152.63625-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the gains can also be specified manually, the regulation can run into numeric instabilities by dividing by near zero. Mitigate that by applying a small minium value. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v2: - Collected tag --- src/ipa/rkisp1/algorithms/awb.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index e8b04d03748d..eb4545439ae3 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -421,9 +421,9 @@ RGB Awb::calculateRgbMeans(const IPAFrameContext &frameContext, const rk /* * The ISP computes the AWB means after applying the colour gains, * divide by the gains that were used to get the raw means from the - * sensor. + * sensor. Apply a minimum value to avoid divisions by near-zero. */ - rgbMeans /= frameContext.awb.gains; + rgbMeans /= frameContext.awb.gains.max(0.01); return rgbMeans; }