From patchwork Fri Jun 6 16:41:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23482 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BB028C31E9 for ; Fri, 6 Jun 2025 16:42:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1FE6868DCB; Fri, 6 Jun 2025 18:42:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sQmfM8WH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B36868DB3 for ; Fri, 6 Jun 2025 18:42:15 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D96F98DB for ; Fri, 6 Jun 2025 18:42:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228131; bh=9cwBxtvGzC19ypd9Rx9AYGA52GWM6KIvShU8pEJ4NG8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=sQmfM8WHaXBSAj1Txei4PIvMsFBzQqZPJuxgHH4nD3VNb3Jphq0yZR5hRLIvjTE6s g6IUIXH0W9mYCfz58NKjYTmLX6coMsyPKo77mmhhx7M3Vr/9lnXJAZEEhvGwKmp8eT a78O1hFN/S1HxnZ/oJcg3Oj0F0JbNiHCoy3WxKn8= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 01/23] libcamera: controls: Strings are arrays Date: Fri, 6 Jun 2025 18:41:34 +0200 Message-ID: <20250606164156.1442682-2-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" `ControlId::isArray()` and `ControlValue::isArray()` disagree in the case of strings. Fix it by setting the static size of a string to `libcamera::dynamic_extent` to denote a dynamically sized array-like value. One unfortunate side effect of this change is that if there were string controls (there are none at the moment), then `cam` would display them with an extra `Size: n` annotation. Bug: https://bugs.libcamera.org/show_bug.cgi?id=255 Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- include/libcamera/controls.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index 2ae4ec3d4..b170e30cb 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -98,7 +98,7 @@ struct control_type { template<> struct control_type { static constexpr ControlType value = ControlTypeString; - static constexpr std::size_t size = 0; + static constexpr std::size_t size = libcamera::dynamic_extent; }; template<> From patchwork Fri Jun 6 16:41:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23483 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DD99C3292 for ; Fri, 6 Jun 2025 16:42:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3D4C668DE1; Fri, 6 Jun 2025 18:42:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WkNKx8d9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A62DD68DBB for ; Fri, 6 Jun 2025 18:42:15 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2346E11DD for ; Fri, 6 Jun 2025 18:42:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228131; bh=q5/OHTt+0T8GnO91fr1ydch2o9tKBy3WwyDrit4TU/U=; h=From:To:Subject:Date:In-Reply-To:References:From; b=WkNKx8d9EisUVhDhNUsfTgy4tKBqmU2joA1zBjLhPLQM+XtewtqMJdkEsQD1MfRgT q6NVD4jyPAUGTu8qkiqu2b8n8mvYzZ+T5Xlmj8WZDFZKl8uoiA9mwYPG3eje5SyEYj /oM4kouU4lSSAdVmU5ADZKDMTzMZX2JhL/+AUH1U= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 02/23] libcamera: controls: Add `ControlValueView` Date: Fri, 6 Jun 2025 18:41:35 +0200 Message-ID: <20250606164156.1442682-3-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add `ControlValueView`, which is a non-owning read-only handle to a control value with a very similar interface. Signed-off-by: Barnabás Pőcze --- include/libcamera/controls.h | 69 +++++++++++++ src/libcamera/controls.cpp | 194 +++++++++++++++++++++++++++++++++++ 2 files changed, 263 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index b170e30cb..8734479cf 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include #include @@ -25,6 +26,7 @@ namespace libcamera { class ControlValidator; +class ControlValueView; enum ControlType { ControlTypeNone, @@ -160,6 +162,8 @@ public: value.data(), value.size(), sizeof(typename T::value_type)); } + explicit ControlValue(const ControlValueView &cvv); + ~ControlValue(); ControlValue(const ControlValue &other); @@ -247,6 +251,71 @@ private: std::size_t numElements, std::size_t elementSize); }; +class ControlValueView +{ +public: + constexpr ControlValueView() noexcept + : type_(ControlTypeNone) + { + } + + ControlValueView(const ControlValue &cv) noexcept + : ControlValueView(cv.type(), cv.isArray(), cv.numElements(), + reinterpret_cast(cv.data().data())) + { + } + +#ifndef __DOXYGEN__ + // TODO: should have restricted access? + ControlValueView(ControlType type, bool isArray, std::size_t numElements, const std::byte *data) noexcept + : type_(type), + isArray_(isArray), + numElements_(numElements), + data_(data) + { + assert(isArray || numElements == 1); + } +#endif + + [[nodiscard]] explicit operator bool() const { return type_ != ControlTypeNone; } + [[nodiscard]] ControlType type() const { return type_; } + [[nodiscard]] bool isNone() const { return type_ == ControlTypeNone; } + [[nodiscard]] bool isArray() const { return isArray_; } + [[nodiscard]] std::size_t numElements() const { return numElements_; } + [[nodiscard]] Span data() const; + + [[nodiscard]] bool operator==(const ControlValueView &other) const; + + [[nodiscard]] bool operator!=(const ControlValueView &other) const + { + return !(*this == other); + } + + template + [[nodiscard]] auto get() const + { + using TypeInfo = details::control_type>; + + assert(type_ == TypeInfo::value); + assert(isArray_ == (TypeInfo::size > 0)); + + if constexpr (TypeInfo::size > 0) { + return T(reinterpret_cast(data().data()), numElements_); + } else { + assert(numElements_ == 1); + return *reinterpret_cast(data().data()); + } + } + +private: + ControlType type_ : 8; + bool isArray_ = false; + uint32_t numElements_ = 0; + const std::byte *data_ = nullptr; +}; + +std::ostream &operator<<(std::ostream &s, const ControlValueView &v); + class ControlId { public: diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index 98fa7583d..e7e2781e8 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -106,6 +106,16 @@ ControlValue::ControlValue() { } +/** + * \brief Construct a ControlValue from a ControlValueView + */ +ControlValue::ControlValue(const ControlValueView &cvv) + : ControlValue() +{ + set(cvv.type(), cvv.isArray(), cvv.data().data(), + cvv.numElements(), ControlValueSize[cvv.type()]); +} + /** * \fn template T ControlValue::ControlValue(const T &value) * \brief Construct a ControlValue of type T @@ -395,6 +405,190 @@ void ControlValue::reserve(ControlType type, bool isArray, std::size_t numElemen storage_ = reinterpret_cast(new uint8_t[newSize]); } +/** + * \class ControlValueView + * \brief A non-owning view-like type to the value of a control + */ + +/** + * \fn ControlValueView::ControlValueView() + * \brief Construct an empty view + * \sa ControlValue::ControlValue() + */ + +/** + * \fn ControlValueView::ControlValueView(const ControlValue &v) + * \brief Construct a view referring to \a v + * + * The constructed view will refer to the value stored by \a v, and + * thus \a v must not be modified or destroyed before the view. + * + * \sa ControlValue::ControlValue() + */ + +/** + * \fn ControlValueView::operator bool() const + * \brief Determine if the referred value is valid + * \sa ControlValueView::isNone() + */ + +/** + * \fn ControlType ControlValueView::type() const + * \copydoc ControlValue::type() + * \sa ControlValue::type() + */ + +/** + * \fn ControlValueView::isNone() const + * \copydoc ControlValue::isNone() + * \sa ControlValue::isNone() + */ + +/** + * \fn ControlValueView::isArray() const + * \copydoc ControlValue::isArray() + * \sa ControlValue::isArray() + */ + +/** + * \fn ControlValueView::numElements() const + * \copydoc ControlValue::numElements() + * \sa ControlValue::numElements() + */ + +/** + * \copydoc ControlValue::data() + * \sa ControlValue::data() + */ +Span ControlValueView::data() const +{ + return { data_, numElements_ * ControlValueSize[type_] }; +} + +/** + * \copydoc ControlValue::operator==() + * \sa ControlValue::operator==() + * \sa ControlValueView::operator!=() + */ +bool ControlValueView::operator==(const ControlValueView &other) const +{ + if (type_ != other.type_) + return false; + + if (numElements_ != other.numElements_) + return false; + + if (isArray_ != other.isArray_) + return false; + + const auto d = data(); + + return memcmp(d.data(), other.data_, d.size_bytes()) == 0; +} + +/** + * \fn ControlValueView::operator!=() const + * \copydoc ControlValue::operator!=() + * \sa ControlValue::operator!=() + * \sa ControlValueView::operator==() + */ + +/** + * \fn template T ControlValueView::get() const + * \copydoc ControlValue::get() + * \sa ControlValue::get() + */ + +/** + * \brief Insert a text representation of a value into an output stream + * \sa ControlValue::toString() + */ +std::ostream &operator<<(std::ostream &s, const ControlValueView &v) +{ + const auto type = v.type(); + if (type == ControlTypeNone) + return s << "None"; + + const auto *data = v.data().data(); + const auto numElements = v.numElements(); + + if (type == ControlTypeString) + return s << std::string_view(reinterpret_cast(data), + numElements); + + const bool isArray = v.isArray(); + if (isArray) + s << "[ "; + + for (std::size_t i = 0; i < numElements; ++i) { + if (i > 0) + s << ", "; + + switch (type) { + case ControlTypeBool: { + const bool *value = reinterpret_cast(data); + s << (*value ? "true" : "false"); + break; + } + case ControlTypeByte: { + const auto *value = reinterpret_cast(data); + s << static_cast(*value); + break; + } + case ControlTypeUnsigned16: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypeUnsigned32: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypeInteger32: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypeInteger64: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypeFloat: { + const auto *value = reinterpret_cast(data); + s << std::fixed << *value; + break; + } + case ControlTypeRectangle: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypeSize: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypePoint: { + const auto *value = reinterpret_cast(data); + s << *value; + break; + } + case ControlTypeNone: + case ControlTypeString: + break; + } + + data += ControlValueSize[type]; + } + + if (isArray) + s << " ]"; + + return s; +} + /** * \class ControlId * \brief Control static metadata From patchwork Fri Jun 6 16:41:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23484 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E42B8C31E9 for ; Fri, 6 Jun 2025 16:42:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 07B9F68DDB; Fri, 6 Jun 2025 18:42:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Z4IoEY6v"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D29EE68DBE for ; Fri, 6 Jun 2025 18:42:15 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 622836DC for ; Fri, 6 Jun 2025 18:42:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228131; bh=WY6KFl2rgYpr40FhHWFsbmNFUR8W4o5Hraxny1j+k6I=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Z4IoEY6vDH8n54obIheVvyDI2KcMmYGKMlPOv8/Ku3nZHGjEth/zx1zdNjBsoU8bg Ajs/z+xfhAPRX410YIkwO308lshbAxgSiq3y++UPCymtYbFMxVUyJNd6/IOBTqLQiK A10fClcfuAgE4RH7vfuTrKIXm2XDlL+MBBJ7kfFE= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 03/23] libcamera: base: Add file for C++20 polyfills Date: Fri, 6 Jun 2025 18:41:36 +0200 Message-ID: <20250606164156.1442682-4-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add `cxx20.h` that will contain (possibly limited) C++17 implementations of some C++20 standard library features. Signed-off-by: Barnabás Pőcze --- include/libcamera/base/details/cxx20.h | 12 ++++++++++++ include/libcamera/base/meson.build | 1 + 2 files changed, 13 insertions(+) create mode 100644 include/libcamera/base/details/cxx20.h diff --git a/include/libcamera/base/details/cxx20.h b/include/libcamera/base/details/cxx20.h new file mode 100644 index 000000000..0d6173d1b --- /dev/null +++ b/include/libcamera/base/details/cxx20.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * C++20 polyfills + */ + +#pragma once + +namespace libcamera::details::cxx20 { + +} /* namespace libcamera::details::cxx20 */ diff --git a/include/libcamera/base/meson.build b/include/libcamera/base/meson.build index f28ae4d42..9b93b89c3 100644 --- a/include/libcamera/base/meson.build +++ b/include/libcamera/base/meson.build @@ -5,6 +5,7 @@ libcamera_base_include_dir = libcamera_include_dir / 'base' libcamera_base_public_headers = files([ 'bound_method.h', 'class.h', + 'details/cxx20.h', 'flags.h', 'object.h', 'shared_fd.h', From patchwork Fri Jun 6 16:41:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23485 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6C872C3324 for ; Fri, 6 Jun 2025 16:42:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 96F5268DD2; Fri, 6 Jun 2025 18:42:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DFDA754l"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1DF0768DB0 for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9F0D68DB for ; Fri, 6 Jun 2025 18:42:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228131; bh=2fSH/OQqLg58+3C1YZ6eqgo7YNf9r85SKfkKGtI1OpU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=DFDA754l+P+XyRQISlQC8Zf0k0M8FBMh2hHzqZ+BvLwHi66+VR2KSheqGRaHhZwxu ubz/baKrc1o3KicOiqZfnHi1/9RMST1LS6GR7uSXMH6DNSXuJ3Kye+hLfdBd6GnuYV nKKqZXUx/KyjVC3z/yV39uiSO5Menbcd/Wb/32mA= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 04/23] libcamera: base: cxx20: Add `type_identity{, _t}` Date: Fri, 6 Jun 2025 18:41:37 +0200 Message-ID: <20250606164156.1442682-5-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" `type_identity_t` can be used to force non-deduced contexts in templates, such as: void f(T x, type_identity_t y) when calling `f(1u, 2)`, without `type_identity_t`, the compiler could not unambiguously deduce `T` (unsigned int vs int). However, with `type_identity_t`, the type of the argument passed to `y` will not be used to deduce `T`, only the argument passed to `x`. See https://en.cppreference.com/w/cpp/types/type_identity Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- include/libcamera/base/details/cxx20.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/libcamera/base/details/cxx20.h b/include/libcamera/base/details/cxx20.h index 0d6173d1b..2d26db53e 100644 --- a/include/libcamera/base/details/cxx20.h +++ b/include/libcamera/base/details/cxx20.h @@ -9,4 +9,7 @@ namespace libcamera::details::cxx20 { +template struct type_identity { using type = T; }; +template using type_identity_t = typename type_identity::type; + } /* namespace libcamera::details::cxx20 */ From patchwork Fri Jun 6 16:41:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23486 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E179CC3325 for ; Fri, 6 Jun 2025 16:42:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2D9C668DF2; Fri, 6 Jun 2025 18:42:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GYQzkkxQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A04668DBD for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD3126DC for ; Fri, 6 Jun 2025 18:42:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228132; bh=I154HYs0GO304Y6yaX6MDVD3TR7uKqgqftaAYmVUJcU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=GYQzkkxQG1OTmKCdnb8xQqivNQFrBboJN8Ho59lXcDFr8Nvfu0l37lf8eHQPZRk/R BOxEro7jrIJlixO92EJ3sRgKBmkbfApZkDb3KsniojfDEMEAj2T9HyNVbxvXeZyISZ AR1481VjoFpcoxvIf78c0XwyNX/KEuXPo9iEBpoY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 05/23] libcamera: base: cxx20: Add `has_single_bit()` Date: Fri, 6 Jun 2025 18:41:38 +0200 Message-ID: <20250606164156.1442682-6-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" `has_single_bit()` checks if the given unsigned integer has a single bit set, that is, whether the number is a power of two or not. Not all requirements of the C++20 standard are implemented. See https://en.cppreference.com/w/cpp/numeric/has_single_bit Signed-off-by: Barnabás Pőcze --- include/libcamera/base/details/cxx20.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/include/libcamera/base/details/cxx20.h b/include/libcamera/base/details/cxx20.h index 2d26db53e..8bf45ff11 100644 --- a/include/libcamera/base/details/cxx20.h +++ b/include/libcamera/base/details/cxx20.h @@ -7,9 +7,19 @@ #pragma once +#include + namespace libcamera::details::cxx20 { template struct type_identity { using type = T; }; template using type_identity_t = typename type_identity::type; +template +constexpr bool has_single_bit(T x) noexcept +{ + static_assert(std::is_unsigned_v); + + return x != 0 && (x & (x - 1)) == 0; +} + } /* namespace libcamera::details::cxx20 */ From patchwork Fri Jun 6 16:41:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23487 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 343EEC3326 for ; Fri, 6 Jun 2025 16:42:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EAC2B68DB0; Fri, 6 Jun 2025 18:42:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cd6uaOdL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 95F6968DC1 for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 265238DB for ; Fri, 6 Jun 2025 18:42:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228132; bh=iIDSatRfy0cQy+iqy8owgNdQ6BhWbHCC8Hz44BO/7ws=; h=From:To:Subject:Date:In-Reply-To:References:From; b=cd6uaOdLn6JPFVaUnFs6GULk87r//6kq7Ft53pG5VqgDyd+D7nnt5LqPfxt9mDScS QQBKgX13GRHupeioGlD/FCcbs369jTNAKVT/DZzCWx97aiflR8AhAtJxB1XKlqplc6 GYafglcV1LlcAv0XcnCz25oDbmPRR2/gWi6+ruiI= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 06/23] libcamera: base: Add alignment utility functions Date: Fri, 6 Jun 2025 18:41:39 +0200 Message-ID: <20250606164156.1442682-7-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a couple internal functions to check alignment, and to align integers, pointer up to a given alignment. Signed-off-by: Barnabás Pőcze --- include/libcamera/base/details/align.h | 72 ++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 include/libcamera/base/details/align.h diff --git a/include/libcamera/base/details/align.h b/include/libcamera/base/details/align.h new file mode 100644 index 000000000..e1c24c6e1 --- /dev/null +++ b/include/libcamera/base/details/align.h @@ -0,0 +1,72 @@ +#pragma once + +#include +#include +#include + +#include + +namespace libcamera::details::align { + +template +bool is(T *p, std::uintptr_t alignment) +{ + assert(alignment > 0); + + return reinterpret_cast(p) % alignment == 0; +} + +template +constexpr T up(T x, cxx20::type_identity_t alignment) +{ + static_assert(std::is_unsigned_v); + assert(alignment > 0); + + const auto padding = (alignment - (x % alignment)) % alignment; + assert(x + padding >= x); + + return x + padding; +} + +template +auto *up(T *p, std::uintptr_t alignment) +{ + using U = std::conditional_t< + std::is_const_v, + const std::byte, + std::byte + >; + + return reinterpret_cast(up(reinterpret_cast(p), alignment)); +} + +template +T *up(std::size_t size, std::size_t alignment, T *&ptr, std::size_t *avail = nullptr) +{ + assert(alignment > 0); + + auto p = reinterpret_cast(ptr); + const auto padding = (alignment - (p % alignment)) % alignment; + + if (avail) { + if (size > *avail || padding > *avail - size) + return nullptr; + + *avail -= size + padding; + } + + p += padding; + ptr = reinterpret_cast(p + size); + + return reinterpret_cast(p); +} + +template +U *up(T *&ptr, std::size_t *avail = nullptr) +{ + return reinterpret_cast, const U, U> *>( + up(sizeof(U), alignof(U), ptr, avail) + ); +} + +} /* namespace libcamera::details::align */ From patchwork Fri Jun 6 16:41:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23488 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F5B2C3292 for ; Fri, 6 Jun 2025 16:42:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4486F68DB3; Fri, 6 Jun 2025 18:42:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BvRtlSyz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB89A68DC5 for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 664DB6DC for ; Fri, 6 Jun 2025 18:42:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228132; bh=Reg56XeorKAGdSyu8t632fgzuDSyiABZV58MYlpP+jc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=BvRtlSyzJO7bFrYNqTLwBUx/H1u4aqU/3c2Jvg6kg2CJGR+MHW3rnTZLC3knL/0g+ bf9ivYaMkK1rKBKFJ42PbAf4NisyxlJc3S445zI0n62yIBg8VKnny9KP/q4Rs6YQYi HWO0C2bXXvnuImUxmNTEDv3W38W0LCnM0aR4R8jQ= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 07/23] libcamera: Add `MetadataList` Date: Fri, 6 Jun 2025 18:41:40 +0200 Message-ID: <20250606164156.1442682-8-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a dedicated `MetadataList` type, whose purpose is to store the metadata reported by a camera for a given request. Previously, a `ControlList` was used for this purpose. The reason for introducing a separate type is to simplify the access to the returned metadata during the entire lifetime of a request. Specifically, for early metadata completion to be easily usable it should be guaranteed that any completed metadata item can be accessed and looked up at least until the associated requested is reused with `Request::reuse()`. However, when a metadata item is completed early, the pipeline handler might still work on the request in the `CameraManager`'s private thread, therefore there is an inherent synchronization issue when an application accesses early metadata. Restricting the user to only access the metadata items of a not yet completed request in the early metadata availability signal handler by ways of documenting or enforcing it at runtime could be an option, but it is not too convenient for the user. The current `ControlList` implementation employs an `std::unordered_map`, so pointers remain stable when the container is modified, so an application could keep accessing particular metadata items outside the signal handler, but this fact is far from obvious, and the user would still not be able to make a copy of all metadata or do lookups based on the numeric ids or the usual `libcamera::Control<>` objects, thus some type safety is lost. The above also requires that each metadata item is only completed once for a given request, but this does not appear to be serious limitation, and in fact, this restriction is enforced by `MetadataList`. The introduced `MetadataList` supports single writer - multiple reader scenarios, and it can be set, looked-up, and copied in a wait-free fashion without introducing data races or other synchronization issues. This is achieved by requiring the possible set of metadata items to be known (such set is stored in a `MetadataListPlan` object). Based on the this plan, a single contiguous allocation is made to accommodate all potential metadata items. Due to this single contiguous allocation that is not modified during the lifetime of a `MetadataList` and atomic modifications, it is possible to easily gaurantee thread-safe set, lookup, and copy; assuming there is only ever a single writer. Signed-off-by: Barnabás Pőcze --- include/libcamera/meson.build | 2 + include/libcamera/metadata_list.h | 619 +++++++++++++++++++++++++ include/libcamera/metadata_list_plan.h | 109 +++++ src/libcamera/meson.build | 1 + src/libcamera/metadata_list.cpp | 315 +++++++++++++ test/controls/meson.build | 1 + test/controls/metadata_list.cpp | 171 +++++++ 7 files changed, 1218 insertions(+) create mode 100644 include/libcamera/metadata_list.h create mode 100644 include/libcamera/metadata_list_plan.h create mode 100644 src/libcamera/metadata_list.cpp create mode 100644 test/controls/metadata_list.cpp diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 30ea76f94..410b548dd 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -12,6 +12,8 @@ libcamera_public_headers = files([ 'framebuffer_allocator.h', 'geometry.h', 'logging.h', + 'metadata_list.h', + 'metadata_list_plan.h', 'orientation.h', 'pixel_format.h', 'request.h', diff --git a/include/libcamera/metadata_list.h b/include/libcamera/metadata_list.h new file mode 100644 index 000000000..7514bd2ad --- /dev/null +++ b/include/libcamera/metadata_list.h @@ -0,0 +1,619 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + * + * Metadata list + */ + +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include + +// TODO: want this? +#if __has_include() +#if __SANITIZE_ADDRESS__ /* gcc */ +#include +#define HAS_ASAN 1 +#elif defined(__has_feature) +#if __has_feature(address_sanitizer) /* clang */ +#include +#define HAS_ASAN 1 +#endif +#endif +#endif + +namespace libcamera { + +class MetadataList +{ +private: + struct ValueParams { + ControlType type; + bool isArray; + std::uint32_t numElements; + }; + + struct Entry { + const std::uint32_t tag; + const std::uint32_t capacity; + const std::uint32_t alignment; + const ControlType type; + bool isArray; + + static constexpr std::uint32_t invalidOffset = -1; + /* + * Offset from the beginning of the allocation, and + * and _not_ relative to `contentOffset_`. + */ + std::atomic_uint32_t headerOffset = invalidOffset; + + [[nodiscard]] std::optional hasValue() const + { + auto offset = headerOffset.load(std::memory_order_relaxed); + if (offset == invalidOffset) + return {}; + + return offset; + } + + [[nodiscard]] std::optional acquireData() const + { + auto offset = hasValue(); + if (offset) { + /* sync with release-store on `headerOffset` in `MetadataList::set()` */ + std::atomic_thread_fence(std::memory_order_acquire); + } + + return offset; + } + }; + + struct ValueHeader { + std::uint32_t tag; + std::uint32_t size; + std::uint32_t alignment; + ValueParams params; + }; + + struct State { + std::uint32_t count; + std::uint32_t fill; + }; + +public: + explicit MetadataList(const MetadataListPlan &plan) + : capacity_(plan.size()), + contentOffset_(MetadataList::contentOffset(capacity_)), + alloc_(contentOffset_) + { + for (const auto &[tag, e] : plan) { + assert(details::cxx20::has_single_bit(e.alignment)); + + alloc_ += sizeof(ValueHeader); + alloc_ += e.alignment - 1; // XXX: this is the maximum + alloc_ += e.size; + alloc_ += alignof(ValueHeader) - 1; // XXX: this is the maximum + } + + p_ = static_cast(::operator new(alloc_)); + + auto *entries = reinterpret_cast(p_ + entriesOffset()); + auto it = plan.begin(); + + for (std::size_t i = 0; i < capacity_; i++, ++it) { + const auto &[tag, e] = *it; + + new (&entries[i]) Entry{ + .tag = tag, + .capacity = e.size, + .alignment = e.alignment, + .type = e.type, + .isArray = e.isArray, + }; + } + +#if HAS_ASAN + ::__sanitizer_annotate_contiguous_container( + p_ + contentOffset_, p_ + alloc_, + p_ + alloc_, p_ + contentOffset_ + ); +#endif + } + + MetadataList(const MetadataList &other) + : capacity_(other.capacity_), + contentOffset_(other.contentOffset_), + alloc_(other.alloc_), + p_(static_cast(::operator new(alloc_))) + { + auto *entries = reinterpret_cast(p_ + entriesOffset()); + const auto otherEntries = other.entries(); + + for (std::size_t i = 0; i < capacity_; i++) { + auto *e = new (&entries[i]) Entry{ + .tag = otherEntries[i].tag, + .capacity = otherEntries[i].capacity, + .alignment = otherEntries[i].alignment, + .type = otherEntries[i].type, + .isArray = otherEntries[i].isArray, + }; + + auto v = other.data_of(otherEntries[i]); + if (!v) + continue; + + [[maybe_unused]] auto r = set(*e, v); + assert(r == SetError()); + } + +#if HAS_ASAN + ::__sanitizer_annotate_contiguous_container( + p_ + contentOffset_, p_ + alloc_, + p_ + alloc_, p_ + contentOffset_ + ); +#endif + } + + MetadataList(MetadataList &&) = delete; + + MetadataList &operator=(const MetadataList &) = delete; + MetadataList &operator=(MetadataList &&) = delete; + + ~MetadataList() + { +#if HAS_ASAN + /* + * The documentation says the range apparently has to be + * restored to its initial state before it is deallocated. + */ + ::__sanitizer_annotate_contiguous_container( + p_ + contentOffset_, p_ + alloc_, + p_ + contentOffset_ + state_.load(std::memory_order_relaxed).fill, p_ + alloc_ + ); +#endif + + ::operator delete(p_, alloc_); + } + + // TODO: want these? + [[nodiscard]] std::size_t size() const { return state_.load(std::memory_order_relaxed).count; } + [[nodiscard]] bool empty() const { return size() == 0; } + + enum class SetError { + UnknownTag = 1, + AlreadySet, + DataTooLarge, + TypeMismatch, + }; + + [[nodiscard]] SetError set(std::uint32_t tag, ControlValueView v) + { + auto *e = find(tag); + if (!e) + return SetError::UnknownTag; + + return set(*e, v); + } + + template + /* TODO: [[nodiscard]] */ SetError set(const Control &ctrl, const details::cxx20::type_identity_t &value) + { + using TypeInfo = libcamera::details::control_type; + + if constexpr (TypeInfo::size > 0) { + static_assert(std::is_trivially_copyable_v); + + return set(ctrl.id(), { + TypeInfo::value, + true, + value.size(), + reinterpret_cast(value.data()), + }); + } else { + static_assert(std::is_trivially_copyable_v); + + return set(ctrl.id(), { + TypeInfo::value, + false, + 1, + reinterpret_cast(&value), + }); + } + } + + template + [[nodiscard]] decltype(auto) get(const Control &ctrl) const + { + ControlValueView v = get(ctrl.id()); + + return v ? std::optional(v.get()) : std::nullopt; + } + + // TODO: operator ControlListView() const ? + // TODO: explicit operator ControlList() const ? + + [[nodiscard]] ControlValueView get(std::uint32_t tag) const + { + const auto *e = find(tag); + if (!e) + return {}; + + return data_of(*e); + } + + void clear() + { + for (auto &e : entries()) + e.headerOffset.store(Entry::invalidOffset, std::memory_order_relaxed); + + [[maybe_unused]] auto s = state_.exchange({}, std::memory_order_relaxed); + +#if HAS_ASAN + ::__sanitizer_annotate_contiguous_container( + p_ + contentOffset_, p_ + alloc_, + p_ + contentOffset_ + s.fill, p_ + contentOffset_ + ); +#endif + } + + class iterator + { + public: + using difference_type = std::ptrdiff_t; + using value_type = std::pair; + using pointer = void; + using reference = value_type; + using iterator_category = std::forward_iterator_tag; + + iterator() = default; + + iterator& operator++() + { + const auto &h = header(); + + p_ += sizeof(h); + p_ = details::align::up(p_, h.alignment); + p_ += h.size; + p_ = details::align::up(p_, alignof(decltype(h))); + + return *this; + } + + iterator operator++(int) + { + auto copy = *this; + ++*this; + return copy; + } + + [[nodiscard]] reference operator*() const + { + const auto &h = header(); + const auto *data = details::align::up(p_ + sizeof(h), h.alignment); + + return { h.tag, { h.params.type, h.params.isArray, h.params.numElements, data } }; + } + + [[nodiscard]] bool operator==(const iterator &other) const + { + return p_ == other.p_; + } + + [[nodiscard]] bool operator!=(const iterator &other) const + { + return !(*this == other); + } + + private: + iterator(const std::byte *p) + : p_(p) + { + } + + [[nodiscard]] const ValueHeader &header() const + { + return *reinterpret_cast(p_); + } + + friend MetadataList; + + const std::byte *p_ = nullptr; + }; + + [[nodiscard]] iterator begin() const + { + return { p_ + contentOffset_ }; + } + + [[nodiscard]] iterator end() const + { + return { p_ + contentOffset_ + state_.load(std::memory_order_acquire).fill }; + } + + class Diff + { + public: + // TODO: want these? + [[nodiscard]] explicit operator bool() const { return !empty(); } + [[nodiscard]] bool empty() const { return start_ == stop_; } + [[nodiscard]] std::size_t size() const { return changed_; } + [[nodiscard]] const MetadataList &list() const { return *l_; } + + [[nodiscard]] ControlValueView get(std::uint32_t tag) const + { + const auto *e = l_->find(tag); + if (!e) + return {}; + + auto o = e->acquireData(); + if (!o) + return {}; + + if (!(start_ <= *o && *o < stop_)) + return {}; + + return l_->data_of(*o); + } + + template + [[nodiscard]] decltype(auto) get(const Control &ctrl) const + { + ControlValueView v = get(ctrl.id()); + + return v ? std::optional(v.get()) : std::nullopt; + } + + [[nodiscard]] iterator begin() const + { + return { l_->p_ + start_ }; + } + + [[nodiscard]] iterator end() const + { + return { l_->p_ + stop_ }; + } + + private: + Diff(const MetadataList &l, std::size_t changed, std::size_t oldFill, std::size_t newFill) + : l_(&l), + changed_(changed), + start_(l.contentOffset_ + oldFill), + stop_(l.contentOffset_ + newFill) + { + } + + friend MetadataList; + friend struct Checkpoint; + + const MetadataList *l_ = nullptr; + std::size_t changed_; + std::size_t start_; + std::size_t stop_; + }; + + Diff merge(const MetadataList &other) + { + const auto entries = this->entries(); + const auto otherEntries = other.entries(); + const auto c = checkpoint(); + + for (std::size_t i = 0, j = 0; i < entries.size() && j < otherEntries.size(); ) { + if (entries[i].tag < otherEntries[j].tag) { + i += 1; + continue; + } + + if (entries[i].tag > otherEntries[j].tag) { + j += 1; + continue; + } + + assert(entries[i].alignment >= otherEntries[j].alignment); + assert(entries[i].capacity >= otherEntries[j].capacity); + + if (!entries[i].hasValue()) { + auto v = other.data_of(otherEntries[j]); + if (v) { + [[maybe_unused]] auto r = set(entries[i], v); + assert(r == SetError()); + } + } + + i += 1; + j += 1; + } + + return c.diffSince(); + } + + Diff merge(const ControlList &other) + { + // TODO: check id map of `other`? + + const auto c = checkpoint(); + + for (auto &&[tag, value] : other) { + auto *e = find(tag); + if (e) { + [[maybe_unused]] auto r = set(*e, value); + assert(r == SetError() || r == SetError::AlreadySet); // TODO: ? + } + } + + return c.diffSince(); + } + + class Checkpoint + { + public: + // TODO: want this? + [[nodiscard]] const MetadataList &list() const { return *l_; } + + [[nodiscard]] Diff diffSince() const + { + /* sync with release-store on `state_` in `set()` */ + const auto curr = l_->state_.load(std::memory_order_acquire); + + assert(s_.count <= curr.count); + assert(s_.fill <= curr.fill); + + return { + *l_, + curr.count - s_.count, + s_.fill, + curr.fill, + }; + } + + private: + Checkpoint(const MetadataList &l) + : l_(&l), + s_(l.state_.load(std::memory_order_relaxed)) + { + } + + friend MetadataList; + + const MetadataList *l_ = nullptr; + State s_ = {}; + }; + + [[nodiscard]] Checkpoint checkpoint() const + { + return { *this }; + } + +private: + [[nodiscard]] static constexpr std::size_t entriesOffset() + { + return 0; + } + + [[nodiscard]] static constexpr std::size_t contentOffset(std::size_t entries) + { + return details::align::up(entriesOffset() + entries * sizeof(Entry), alignof(ValueHeader)); + } + + [[nodiscard]] Span entries() const + { + return { reinterpret_cast(p_ + entriesOffset()), capacity_ }; + } + + [[nodiscard]] Entry *find(std::uint32_t tag) const + { + const auto entries = this->entries(); + auto it = std::lower_bound(entries.begin(), entries.end(), tag, [](const auto &e, const auto &t) { + return e.tag < t; + }); + + if (it == entries.end() || it->tag != tag) + return nullptr; + + return &*it; + } + + [[nodiscard]] ControlValueView data_of(const Entry &e) const + { + const auto o = e.acquireData(); + return o ? data_of(*o) : ControlValueView{ }; + } + + [[nodiscard]] ControlValueView data_of(std::size_t headerOffset) const + { + assert(headerOffset <= alloc_ - sizeof(ValueHeader)); + assert(details::align::is(p_ + headerOffset, alignof(ValueHeader))); + + const auto *vh = reinterpret_cast(p_ + headerOffset); + const auto *p = reinterpret_cast(vh) + sizeof(*vh); + std::size_t avail = p_ + alloc_ - p; + + const auto *data = details::align::up(vh->size, vh->alignment, p, &avail); + assert(data); + + return { vh->params.type, vh->params.isArray, vh->params.numElements, data }; + } + + [[nodiscard]] SetError set(Entry &e, ControlValueView v) + { + if (e.hasValue()) + return SetError::AlreadySet; + if (e.type != v.type() || e.isArray != v.isArray()) + return SetError::TypeMismatch; + + const auto src = v.data(); + if (e.isArray) { + if (src.size_bytes() > e.capacity) + return SetError::DataTooLarge; + } else { + assert(src.size_bytes() == e.capacity); + } + + auto s = state_.load(std::memory_order_relaxed); + std::byte *oldEnd = p_ + contentOffset_ + s.fill; + std::byte *p = oldEnd; + + auto *headerPtr = details::align::up(p); + auto *dataPtr = details::align::up(src.size_bytes(), e.alignment, p); + details::align::up(0, alignof(ValueHeader), p); + +#if HAS_ASAN + ::__sanitizer_annotate_contiguous_container( + p_ + contentOffset_, p_ + alloc_, + oldEnd, p + ); +#endif + + new (headerPtr) ValueHeader{ + .tag = e.tag, + .size = std::uint32_t(src.size_bytes()), + .alignment = e.alignment, + .params = { + .type = v.type(), + .isArray = v.isArray(), + .numElements = std::uint32_t(v.numElements()), + }, + }; + std::memcpy(dataPtr, src.data(), src.size_bytes()); + e.headerOffset.store(reinterpret_cast(headerPtr) - p_, std::memory_order_release); + + s.fill += p - oldEnd; + s.count += 1; + + state_.store(s, std::memory_order_release); + + return {}; + } + + std::size_t capacity_ = 0; + std::size_t contentOffset_ = -1; + std::size_t alloc_ = 0; + std::atomic state_ = State{}; + std::byte *p_ = nullptr; + // TODO: ControlIdMap in any way shape or form? + + /* + * If this is problematic on a 32-bit architecture, then + * `count` can be stored in a separate atomic variable + * but then `Diff::changed_` must be removed since the fill + * level and item count cannot be retrieved atomically. + */ + static_assert(decltype(state_)::is_always_lock_free); +}; + +} /* namespace libcamera */ + +#undef HAS_ASAN diff --git a/include/libcamera/metadata_list_plan.h b/include/libcamera/metadata_list_plan.h new file mode 100644 index 000000000..9ad4ae87e --- /dev/null +++ b/include/libcamera/metadata_list_plan.h @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + */ + +#pragma once + +#include +#include +#include +#include +#include +#include + +#include + +#include + +namespace libcamera { + +class MetadataListPlan +{ +public: + [[nodiscard]] bool empty() const { return items_.empty(); } + [[nodiscard]] std::size_t size() const { return items_.size(); } + [[nodiscard]] decltype(auto) begin() const { return items_.begin(); } + [[nodiscard]] decltype(auto) end() const { return items_.end(); } + void clear() { items_.clear(); } + + template< + typename T, + std::enable_if_t::size != libcamera::dynamic_extent> * = nullptr + > + decltype(auto) add(const Control &ctrl) + { + if constexpr (libcamera::details::control_type::size > 0) { + static_assert(libcamera::details::control_type::size != libcamera::dynamic_extent); + + return add( + ctrl.id(), + libcamera::details::control_type::size, + true + ); + } else { + return add(ctrl.id(), 1, false); + } + } + + template< + typename T, + std::enable_if_t::size == libcamera::dynamic_extent> * = nullptr + > + decltype(auto) add(const Control &ctrl, std::size_t count) + { + return add(ctrl.id(), count, true); + } + +#ifndef __DOXYGEN__ + MetadataListPlan &add(std::uint32_t tag, + std::size_t size, std::size_t count, std::size_t alignment, + ControlType type, bool isArray) + { + assert(size > 0 && size <= std::numeric_limits::max()); + assert(count <= std::numeric_limits::max() / size); + assert(alignment <= std::numeric_limits::max()); + assert(details::cxx20::has_single_bit(alignment)); + assert(isArray || count == 1); + + items_.insert_or_assign(tag, Entry{ + .size = std::uint32_t(size * count), + .alignment = std::uint32_t(alignment), + .type = type, + .isArray = isArray, + }); + + return *this; + } +#endif + + bool remove(std::uint32_t tag) + { + return items_.erase(tag); + } + + bool remove(const ControlId &ctrl) + { + return remove(ctrl.id()); + } + +private: + struct Entry { + std::uint32_t size; + std::uint32_t alignment; // TODO: is this necessary? + ControlType type; + bool isArray; + }; + + std::map items_; + + template + decltype(auto) add(std::uint32_t tag, std::size_t count, bool isArray) + { + static_assert(std::is_trivially_copyable_v); + + return add(tag, sizeof(T), count, alignof(T), details::control_type::value, isArray); + } +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 202db1efe..d7a850907 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -9,6 +9,7 @@ libcamera_public_sources = files([ 'framebuffer.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', + 'metadata_list.cpp', 'orientation.cpp', 'pixel_format.cpp', 'request.cpp', diff --git a/src/libcamera/metadata_list.cpp b/src/libcamera/metadata_list.cpp new file mode 100644 index 000000000..ccfe37318 --- /dev/null +++ b/src/libcamera/metadata_list.cpp @@ -0,0 +1,315 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + */ + +#include + +namespace libcamera { + +/** + * \class MetadataListPlan + * \brief Class to hold the possible set of metadata items for a \ref MetadataList + */ + +/** + * \fn MetadataListPlan::begin() const + * \brief Retrieve the begin iterator + */ + +/** + * \fn MetadataListPlan::end() const + * \brief Retrieve the end iterator + */ + +/** + * \fn MetadataListPlan::size() const + * \brief Retrieve the number of controls + */ + +/** + * \fn MetadataListPlan::empty() const + * \brief Check if empty + */ + +/** + * \fn MetadataListPlan::clear() + * \brief Remove all controls + */ + +/** + * \fn MetadataListPlan::add(const Control &ctrl) + * \brief Add a control to the metadata list plan + */ + +/** + * \fn MetadataListPlan::add(const Control &ctrl, std::size_t count) + * \brief Add a dynamically-sized control to the metadata list plan + * \param[in] ctrl The control + * \param[in] count The maximum number of elements + * + * Add the dynamically-sized control \a ctrl to the metadata list plan with a maximum + * capacity of \a count elements. + */ + +/** + * \fn MetadataListPlan::remove(std::uint32_t tag) + * \brief Remove the entry with given identifier from the plan + */ + +/** + * \fn MetadataListPlan::remove(const ControlId &ctrl) + * \brief Remove \a ctrl from the plan + */ + +/** + * \class MetadataList + * \brief Class to hold metadata items + */ + +/** + * \fn MetadataList::MetadataList(const MetadataListPlan &plan) + * \brief Construct a metadata list according to \a plan + * + * Construct a metadata list according to the provided \a plan. + */ + +/** + * \fn MetadataList::MetadataList(const MetadataList &other) + * \brief Copy constructor + * \context This function is \threadsafe wrt. \a other. + */ + +/** + * \fn MetadataList::size() const + * \brief Retrieve the number of controls + * \context This function is \threadsafe. + * \note If the list is being modified, the return value may be out of + * date by the time the function returns + */ + +/** + * \fn MetadataList::empty() const + * \brief Check if empty + * \context This function is \threadsafe. + * \note If the list is being modified, the return value may be out of + * date by the time the function returns + */ + +/** + * \internal + * \fn MetadataList::clear() + * \brief Remove all items from the list + * \note This function in effect resets the list to its original state. As a consequence it invalidates - among others - + * all iterators, Checkpoint, and Diff objects that are associated with the list. No readers must exist + * when this function is called. + */ + +/** + * \fn MetadataList::begin() const + * \brief Retrieve begin iterator + * \context This function is \threadsafe. + */ + +/** + * \fn MetadataList::end() const + * \brief Retrieve end iterator + * \context This function is \threadsafe. + */ + +/** + * \fn MetadataList::get(const Control &ctrl) const + * \brief Get the value of control \a ctrl + * \return A std::optional containing the control value, or std::nullopt if + * the control \a ctrl is not present in the list + * \context This function is \threadsafe. + */ + +/** + * \fn MetadataList::get(std::uint32_t tag) const + * \brief Get the value of pertaining to the numeric identifier \a tag + * \return A std::optional containing the control value, or std::nullopt if + * the control is not present in the list + * \context This function is \threadsafe. + */ + +/** + * \fn MetadataList::set(const Control &ctrl, const details::cxx20::type_identity_t &value) + * \brief Set the value of control \a ctrl to \a value + */ + +/** + * \fn MetadataList::set(std::uint32_t tag, ControlValueView v) + * \brief Set the value of pertaining to the numeric identifier \a tag to \a v + */ + +/** + * \internal + * \fn MetadataList::merge(const MetadataList &other) + * \brief Add all missing items from \a other + * + * Add all items from \a other that are not present in \a this. If an item + * has a numeric identifier that was not present in the MetadataListPlan + * used to construct \a this, then the item is ignored. + * + * \context This function is \threadsafe wrt. \a other. + */ + +/** + * \internal + * \fn MetadataList::merge(const ControlList &other) + * \copydoc MetadataList::merge(const MetadataList &other) + */ + +/** + * \enum MetadataList::SetError + * \brief TODO + * + * \var MetadataList::SetError::UnknownTag + * \brief The tag is not supported by the metadata list + * \var MetadataList::SetError::AlreadySet + * \brief A value has already been added with the given tag + * \var MetadataList::SetError::DataTooLarge + * \brief The data is too large for the given tag + * \var MetadataList::SetError::TypeMismatch + * \brief The type of the value does not match the expected type + */ + +/** + * \internal + * \fn MetadataList::checkpoint() const + * \brief Create a checkpoint + * \context This function is \threadsafe. + */ + +/** + * \class MetadataList::iterator + * \brief Iterator + */ + +/** + * \typedef MetadataList::iterator::difference_type + * \brief iterator's difference type + */ + +/** + * \typedef MetadataList::iterator::value_type + * \brief iterator's value type + */ + +/** + * \typedef MetadataList::iterator::pointer + * \brief iterator's pointer type + */ + +/** + * \typedef MetadataList::iterator::reference + * \brief iterator's reference type + */ + +/** + * \typedef MetadataList::iterator::iterator_category + * \brief iterator's category + */ + +/** + * \fn MetadataList::iterator::operator*() + * \brief Retrieve value at iterator + * \return A \a ControlListView representing the value + */ + +/** + * \fn MetadataList::iterator::operator==(const iterator &other) const + * \brief Check if two iterators are equal + */ + +/** + * \fn MetadataList::iterator::operator!=(const iterator &other) const + * \brief Check if two iterators are not equal + */ + +/** + * \fn MetadataList::iterator::operator++(int) + * \brief Advance the iterator + */ + +/** + * \fn MetadataList::iterator::operator++() + * \brief Advance the iterator + */ + +/** + * \class MetadataList::Diff + * \brief Designates a set of consecutively added metadata items from a particular MetadataList + * \sa Camera::metadataAvailable + * \internal + * \sa MetadataList::Checkpoint::diffSince() + * \endinternal + */ + +/** + * \fn MetadataList::Diff::list() const + * \brief Retrieve the associated MetadataList + */ + +/** + * \fn MetadataList::Diff::size() const + * \brief Retrieve the number of metadata items designated + */ + +/** + * \fn MetadataList::Diff::empty() const + * \brief Check if no metadata items are designated + */ + +/** + * \fn MetadataList::Diff::operator bool() const + * \copydoc MetadataList::Diff::empty() const + */ + +/** + * \fn MetadataList::Diff::get(const Control &ctrl) const + * \copydoc MetadataList::get(const Control &ctrl) const + * \note The value pertaining to \a ctrl will only be returned if it is part of Diff, + * meaning that even if \a ctrl is part of the backing MetadataList, it will not + * be returned if \a ctrl is not in the set of controls designated by this Diff object. + */ + +/** + * \fn MetadataList::Diff::get(std::uint32_t tag) const + * \copydoc MetadataList::Diff::get(const Control&ctrl) const + */ + +/** + * \fn MetadataList::Diff::begin() const + * \brief Retrieve the begin iterator + */ + +/** + * \fn MetadataList::Diff::end() const + * \brief Retrieve the end iterator + */ + +/** + * \internal + * \class MetadataList::Checkpoint + * \brief Designates a point in the stream of metadata items + * + * A Checkpoint object designates a point in the stream of metadata items in the associated + * MetadataList. Its main use to be able to retrieve the set of metadata items that were + * added to the list after the designated point using diffSince(). + */ + +/** + * \internal + * \fn MetadataList::Checkpoint::list() const + * \brief Retrieve the associated \ref MetadataList + */ + +/** + * \internal + * \fn MetadataList::Checkpoint::diffSince() const + * \brief Retrieve the set of metadata items added since the checkpoint was created + */ + +} /* namespace libcamera */ diff --git a/test/controls/meson.build b/test/controls/meson.build index 763f8905e..b68a4fc53 100644 --- a/test/controls/meson.build +++ b/test/controls/meson.build @@ -5,6 +5,7 @@ control_tests = [ {'name': 'control_info_map', 'sources': ['control_info_map.cpp']}, {'name': 'control_list', 'sources': ['control_list.cpp']}, {'name': 'control_value', 'sources': ['control_value.cpp']}, + {'name': 'metadata_list', 'sources': ['metadata_list.cpp']}, ] foreach test : control_tests diff --git a/test/controls/metadata_list.cpp b/test/controls/metadata_list.cpp new file mode 100644 index 000000000..e02b4e28e --- /dev/null +++ b/test/controls/metadata_list.cpp @@ -0,0 +1,171 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + * + * MetadataList tests + */ + +#include +#include +#include + +#include +#include +#include + +#include "test.h" + +using namespace std; +using namespace libcamera; + +#define ASSERT(x) do { \ + if (!static_cast(x)) { \ + std::cerr << '`' << #x << "` failed" << std::endl; \ + return TestFail; \ + } \ +} while (false) + +class MetadataListTest : public Test +{ +public: + MetadataListTest() = default; + +protected: + int run() override + { + MetadataListPlan mlp; + mlp.add(controls::ExposureTime); + mlp.add(controls::ExposureValue); + mlp.add(controls::ColourGains); + mlp.add(controls::AfWindows, 10); + mlp.add(controls::AeEnable); + mlp.add(controls::SensorTimestamp); + + MetadataList ml(mlp); + + static_assert(static_cast(properties::LOCATION) == controls::AE_ENABLE); + ASSERT(ml.set(properties::Location, properties::CameraLocationFront) == MetadataList::SetError::TypeMismatch); + + ASSERT(ml.set(controls::AfWindows, std::array{}) == MetadataList::SetError::DataTooLarge); + ASSERT(ml.set(controls::ColourTemperature, 123) == MetadataList::SetError::UnknownTag); + + auto f1 = std::async(std::launch::async, [&] { + using namespace std::chrono_literals; + + std::this_thread::sleep_for(500ms); + ASSERT(ml.set(controls::ExposureTime, 0x1111) == MetadataList::SetError()); + + std::this_thread::sleep_for(500ms); + ASSERT(ml.set(controls::ExposureValue, 1) == MetadataList::SetError()); + + std::this_thread::sleep_for(500ms); + ASSERT(ml.set(controls::ColourGains, std::array{ + 123.f, + 456.f + }) == MetadataList::SetError()); + + std::this_thread::sleep_for(500ms); + ASSERT(ml.set(controls::AfWindows, std::array{ + Rectangle(), + Rectangle(1, 2, 3, 4), + Rectangle(0x1111, 0x2222, 0x3333, 0x4444), + }) == MetadataList::SetError()); + + return TestPass; + }); + + auto f2 = std::async(std::launch::async, [&] { + for (;;) { + const auto x = ml.get(controls::ExposureTime); + const auto y = ml.get(controls::ExposureValue); + const auto z = ml.get(controls::ColourGains); + const auto w = ml.get(controls::AfWindows); + + if (x) + ASSERT(*x == 0x1111); + + if (y) + ASSERT(*y == 1.0f); + + if (z) { + ASSERT(z->size() == 2); + ASSERT((*z)[0] == 123.f); + ASSERT((*z)[1] == 456.f); + } + + if (w) { + ASSERT(w->size() == 3); + ASSERT((*w)[0].isNull()); + ASSERT((*w)[1] == Rectangle(1, 2, 3, 4)); + ASSERT((*w)[2] == Rectangle(0x1111, 0x2222, 0x3333, 0x4444)); + } + + if (x && y && z && w) + break; + } + + return TestPass; + }); + + ASSERT(f1.get() == TestPass); + ASSERT(f2.get() == TestPass); + + ASSERT(ml.set(controls::ExposureTime, 0x2222) == MetadataList::SetError::AlreadySet); + ASSERT(ml.set(controls::ExposureValue, 2) == MetadataList::SetError::AlreadySet); + + ASSERT(ml.get(controls::ExposureTime) == 0x1111); + ASSERT(ml.get(controls::ExposureValue) == 1); + + for (auto &&[tag, v] : ml) + std::cout << "[" << tag << "] -> " << v << '\n'; + + std::cout << std::endl; + + ml.clear(); + ASSERT(ml.empty()); + ASSERT(ml.size() == 0); + + ASSERT(ml.set(controls::ExposureTime, 0x2222) == MetadataList::SetError()); + ASSERT(ml.get(controls::ExposureTime) == 0x2222); + + auto c = ml.checkpoint(); + ASSERT(&c.list() == &ml); + + ASSERT(ml.set(controls::ExposureValue, 2) == MetadataList::SetError()); + ASSERT(ml.set(controls::SensorTimestamp, 0x99999999) == MetadataList::SetError()); + + auto d = c.diffSince(); + ASSERT(&d.list() == &ml); + + ASSERT(ml.set(controls::ColourGains, std::array{ 1.f, 2.f }) == MetadataList::SetError()); + + ASSERT(d); + ASSERT(!d.empty()); + ASSERT(d.size() == 2); + ASSERT(!d.get(controls::ExposureTime)); + ASSERT(!d.get(controls::ColourGains)); + ASSERT(!d.get(controls::AfWindows)); + ASSERT(d.get(controls::ExposureValue) == 2); + ASSERT(d.get(controls::SensorTimestamp) == 0x99999999); + + for (auto &&[tag, v] : d) + std::cout << "[" << tag << "] -> " << v << '\n'; + + /* Test if iterators work with algorithms. */ + std::ignore = std::find_if(d.begin(), d.end(), [](const auto &) { + return false; + }); + +#if 0 + { + auto it = ml.begin(); + ml.clear(); + std::ignore = *it; /* Trigger ASAN. */ + } +#endif + + return TestPass; + } +}; + +TEST_REGISTER(MetadataListTest) From patchwork Fri Jun 6 16:41:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23489 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E53F2C3327 for ; Fri, 6 Jun 2025 16:42:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5A27D68DC3; Fri, 6 Jun 2025 18:42:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gWhzA+lJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4B81768DBF for ; Fri, 6 Jun 2025 18:42:17 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AD8B98DB for ; Fri, 6 Jun 2025 18:42:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228132; bh=1kY2O66QbkDnWvTOQisEpM4nUeC4GAZmxwqjPJ7dfHk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=gWhzA+lJgExJjMc86OEg5mLvWthCuoTlEENMPwkfG7mYtZuUkZWbhHbYppAlGklgr byjqbma35QyNwdeaiiV4y2GQoat/9sM3HMXUvIcbFndOnbXQuiVIDuA4xOjuedk3NT /ggiUbxe2zl9WvghUwAQSaCr4Yc+h1jqq8PQAqEA= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 08/23] Documentation: design: Document `MetadataList` Date: Fri, 6 Jun 2025 18:41:41 +0200 Message-ID: <20250606164156.1442682-9-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a document describing the problem, the choices, and the design of the separate metadata list data structure. Signed-off-by: Barnabás Pőcze --- Documentation/design/metadata-list.rst | 234 +++++++++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 236 insertions(+) create mode 100644 Documentation/design/metadata-list.rst diff --git a/Documentation/design/metadata-list.rst b/Documentation/design/metadata-list.rst new file mode 100644 index 000000000..a42f94bdf --- /dev/null +++ b/Documentation/design/metadata-list.rst @@ -0,0 +1,234 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Design of the metadata list +=========================== + +This document explains the design and rationale of the metadata list. + + +Description of the problem +-------------------------- + +Early metadata +^^^^^^^^^^^^^^ + +A pipeline handler might report numerous metadata items to the application about +a single request. It is likely that different metadata items become available at +different points in time while a request is being processed. + +Simultaneously, an application might desire to carry out potentially non-trivial +extra processing one the image, etc. using certain metadata items. For such an +application it is likely best if the final value of each metadata item is reported +as soon as possible, thus allowing it to start processing as soon as possible. + +For this reason, libcamera provides the `metadataAvailable` signal on each `Camera` object. +This signal is dispatched whenever new metadata items become available for a queued request. +This mechanism is completely optional, only interested applications need to subscribe, +others are free to ignore it completely. `Request::metadata()` will contain the sum of +all early metadata items at request completion. + +Thread safety +^^^^^^^^^^^^^ + +At the moment, event handlers of the application are always dispatched in a private +thread of libcamera. This requires that applications process the various events in a +thread-safe manner wrt. themselves. The burden of correct synchronization falls +upon the applications. + +Previously, a `ControlList` was used to store the metadata pertaining to a particular +request. A `ControlList` is implemented using an `std::unordered_map`, meaning that +its thread-safety is limited. This hints at a need for a separate data structure +or at least some kind of thread-safe wrapper. + + +Requirements +------------ + +We wish to provide a simple, easy-to-use, and hard-to-misuse interface for applications. +Notably, applications should be able to delegate early metadata processing to their +own separate threads safely wrt. the metadata list. Consider the following scenario: +the pipeline handler send early metadata items to the application, the application +delegates it to a separate thread. After that, the private libcamera thread is no +longer blocked, thus the pipeline handler can continue working on the request: e.g. +add more metadata items. Simultaneously, the application might be reading the metadata +items on a separate thread. This situation should be safe and work correctly, ideally +with any number of threads reading the completed metadata items. Until the request +is destroyed or reused, whichever happens first. + +Secondarily, efficiency should be considered: copies, locks, reference counting, etc. +should be avoided if possible. + +Preferably, it should be possible to refer to a contiguous (in insertion order) subset +of values reasonably efficiently (i.e. avoiding having to store a separate list of +numeric identifiers, etc.). + + +Options +------- + +Keep using `ControlList` +^^^^^^^^^^^^^^^^^^^^^^^^ + +Using a `ControlList` (and hence `std::unordered_map`) with early metadata completion would +be possible, but it would place a number of potentially non-intuitive and easy to violate +restrictions on applications, making it harder to use safely. Specifically, the application +would have to retrieve a pointer to the `ControlValue` object in the metadata `ControlList`, +and then access it only through that pointer. It wouldn't be able to do lookups on the metadata +list outside the event handler. As a consequence, the usual way of retrieving metadata using +the pre-defined `Control` objects would no longer be possible, losing type-safety. + +Send a copy +^^^^^^^^^^^ + +Passing a separate `ControlList` containing the just completed metadata, and disallowing access +to the request's metadata list until completion works fine, and avoids the synchronization issues +on the libcamera side. Nonetheless, it has two significant drawbacks: + +1. It moves the issue of synchronization from libcamera to the application: the application still has + to access its own data in a thread-safe manner and/or transfer the partial metadata list to its + *main* thread of execution. +2. Early metadata can be reported multiple times for each request, thus making copies can have negative + performance implications. + + +Design +------ + +A separate data structure is introduced to contain the metadata items pertaining to a given request. +It is referred to as "metadata list" from now on. + +The current design of the metadata list places a number of restrictions on request metadata. +A metadata list is backed by a pre-allocated (at construction time) contiguous block of +memory sized appropriately to contain all possible metadata items. This means that the +number and size of metadata items that a camera can report must be known in advance. The +newly introduced `MetadataListPlan` type is used for that purpose. At the time of writing +this does not appear to be a significant limitation since most metadata has a fixed size, +and each pipeline handler (and IPA) has a fixed set of metadata that it can report. There +are, however, metadata items that have a variably-sized array type. In those cases an upper +bound on the number of elements must be provided. + +`MetadataListPlan` +^^^^^^^^^^^^^^^^^^ + +A `MetadataListPlan` collects the set of possible metadata items. It maps the numeric id +of the control to a collection of static information (size, etc.). This is most importantly +used to calculate the size required to store all possible metadata item. + +Each camera has its own `MetadataListPlan` object similarly to its `ControlInfoMap`. It is +used to create requests for the the camera with an appropriately sized `MetadataList`. +Pipeline handlers should fill it during camera initialization or configuration, and they +are allowed to modify it as long as they camera is not configured and during configuration. + +`MetadataList` +^^^^^^^^^^^^^^ + +The current metadata list implementation is a single-writer multiple-readers thread-safe +data structure that provides lock-free lookup and access for any number of threads, while +allowing a single thread at a time to add metadata items. + +The implemented metadata list has two main parts. The first part essentially contains +a copy of the `MetadataListPlan` used to construct the `MetadataList`. In addition to +the static information about the metadata item, it contains dynamic information such +as whether the metadata item has been added to the list or not. + +The second part of a metadata list is a completely self-contained serialized list +of metadata items. The number of bytes used for actually storing metadata items in +this second part will be referred to as the "fill level" from now on. The self-contained +nature of the second part leads to a certain level of data duplication between the two +parts, however, the end goal is to have a serialized version of `ControlList` with the +same serialized format. This would allow a `MetadataList` to be "trivially" reinterpreted +as a control list at any point of its lifetime, simplifying the interoperability between the two. +TODO: do we really want that? + +A metadata list, at construction time, calculates the number of bytes necessary to store +all possible metadata items according to the supplied `MetadataListPlan`. Storage, for +all possible metadata items and the necessary auxiliary structures is then allocated. +This allocation remains fixed for the entire lifetime of a `MetadataList`, which is +crucial to satisfy the earlier requirements. + +Each metadata item can only be added to a metadata list once. This constraint does not pose +a significant limitation, instead, it simplifies the interface and implementation; it is +essentially an append-only list. + +Serialization +''''''''''''' + +The actual values are encoded in the "second part" of the metadata list in a fairly +simple fashion. Each control value is encoded as header + data bytes + padding. Each +value has a header, which contains information such as the size, alignment, type, etc. +of the value. The data bytes are aligned to the alignment specified in the header, +and padding may be inserted after the last data byte to guarantee proper alignment +for the next header. Padding is present even after the last entry. + +The minimum amount of state needed to describe such a serialized list of values is +merely the number of bytes used, which can reasonably be limited to 4 GiB, meaning +that a 32-bit unsigned integer is sufficient to store the fill level. This makes it +possible to easily update the state in a wait-free fashion. + +Lookup +'''''' + +Lookup in a metadata list is done using the metadata entries in the "first part". +These entries are sorted by their numeric identifiers, hence binary search is used to +find the appropriate entry. Then, it is checked whether the given control id has already +been added, and if it has, then its data can be returned in a `ControlValueView` object. + +Insertion +''''''''' + +Similarly to lookup, insertion also starts with binary searching the entry belonging +to the given numeric identifier. If an entry is present for the given id and no value +has already been stored with that id, then insertion can proceed. The value is appended +to the serialized list of control values according to the format described earlier. +Then the fill level is atomically incremented, and the entry is marked as set. After +that the new value is available for readers to consume. + +Having a single writer is an essential requirement to be able to carry out insertion in +a reasonably efficient, and thread-safe manner. + +Iteration +''''''''' + +Iteration of a `MetadataList` is carried out only using the serialized list of controls +in the "second part" of the data structure. An iterator can be implemented as a single +pointer, pointing to the header of the current entry. The begin iterator simply points +to location of the header of the first value. The end iterator is simply the end of the +serialized list of values, which can be calculated from the begin iterator and the fill +level of the serialized list. + +The above iterator can model a `C++ forward iterator`_, that is, only increments of 1 are +possible in constant time, and going backwards is not possible. Advancing to the next value +can be simply implemented by reading the size and alignment from the header, and adjusting +the iterator's pointer by the necessary amount. + +TODO: is a forward iterator enough? is a bidirectional iterator needed? + +.. _C++ forward iterator: https://en.cppreference.com/w/cpp/iterator/forward_iterator.html + +Clearing +'''''''' + +Removing a single value is not supported, but clearing the entire metadata list is. +This should only be done when there are no readers, otherwise readers might run into +data races if they keep reading the metadata when new entries are being added after +clearing it. + +Clearing is implemented by resetting each metadata entry in the "first part", as well +as resetting the stored fill level of the serialized buffer to 0. + +Partial view +'''''''''''' + +When multiple metadata items are completed early, it is important to provide a way +for the application to know exactly which metadata items have just been added. The +serialized values in the data structure are encoded such that a simple byte range +is capable of representing any number of items that have been added in succession. + +The `MetadataList::Checkpoint` type is used to store that state of the serialized +list (number of bytes and number of items) at a given point in time. From such a +checkpoint object a `MetadataList::Diff` object can be constructed, which represents +all values added since the checkpoint. This *diff* object is reasonably small, and +trivially copyable, making it easy to provide to the application. It has much of +the same features as a `MetadataList`, e.g. it can be iterated and one can do lookups. +Naturally, both iteration and lookups only consider the values added after the checkpoint +and before the creation of the `MetadataList::Diff` object. diff --git a/Documentation/index.rst b/Documentation/index.rst index 251112fbd..60cb77702 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -24,6 +24,7 @@ Tracing guide Design document: AE + Design document: Metadata list .. toctree:: :hidden: diff --git a/Documentation/meson.build b/Documentation/meson.build index 0fc5909d0..79e687953 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -127,6 +127,7 @@ if sphinx.found() 'conf.py', 'contributing.rst', 'design/ae.rst', + 'design/metadata-list.rst', 'documentation-contents.rst', 'environment_variables.rst', 'feature_requirements.rst', From patchwork Fri Jun 6 16:41:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23490 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9A221C3328 for ; Fri, 6 Jun 2025 16:42:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C572A68DC5; Fri, 6 Jun 2025 18:42:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pCXhTnsI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8BFD368DB3 for ; Fri, 6 Jun 2025 18:42:17 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EC5BA6DC for ; Fri, 6 Jun 2025 18:42:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228133; bh=NBWnkHiQNaNt4gPUtAlgyBGwTlS7OK0WUY5YX86mWYs=; h=From:To:Subject:Date:In-Reply-To:References:From; b=pCXhTnsIUHPy8I2vV9kKNO539FP+w51WVOJxHpg40egrIfA3JBF4ouJ1iqhNPWFPs +Wjvc3pDHvvTpEQ8lXHtmeJuXCEeS9eyERZyecRWuEgEoHo23pbgou9Z0dd9YLZ7Lu UiatgMDHv2oISdtqxaeE2NdKjSfLqovvYdwwHNy8= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 09/23] libcamera: ipa_data_serializer: Support `MetadataListPlan` Date: Fri, 6 Jun 2025 18:41:42 +0200 Message-ID: <20250606164156.1442682-10-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Define the type in `core.mojom` with external (de)serialization, and add the necessary `IPADataSerializer` template specialization. Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- include/libcamera/ipa/core.mojom | 1 + src/libcamera/ipa_data_serializer.cpp | 84 +++++++++++++++++++ .../core_ipa_interface.h.tmpl | 1 + 3 files changed, 86 insertions(+) diff --git a/include/libcamera/ipa/core.mojom b/include/libcamera/ipa/core.mojom index bce797245..754e4065c 100644 --- a/include/libcamera/ipa/core.mojom +++ b/include/libcamera/ipa/core.mojom @@ -83,6 +83,7 @@ module libcamera; [skipSerdes, skipHeader] struct ControlInfoMap {}; [skipSerdes, skipHeader] struct ControlList {}; [skipSerdes, skipHeader] struct SharedFD {}; +[skipSerdes, skipHeader] struct MetadataListPlan {}; [skipHeader] struct Point { int32 x; diff --git a/src/libcamera/ipa_data_serializer.cpp b/src/libcamera/ipa_data_serializer.cpp index 0537f785b..4646f4aa9 100644 --- a/src/libcamera/ipa_data_serializer.cpp +++ b/src/libcamera/ipa_data_serializer.cpp @@ -11,6 +11,8 @@ #include +#include + #include "libcamera/internal/byte_stream_buffer.h" /** @@ -620,6 +622,88 @@ IPADataSerializer::deserialize(const std::vector &d return deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs); } +template<> +std::tuple, std::vector> +IPADataSerializer::serialize([[maybe_unused]] const MetadataListPlan &data, + [[maybe_unused]] ControlSerializer *cs) +{ + std::vector dataVec; + + appendPOD(dataVec, data.size()); + + for (auto &&[tag, e] : data) { + appendPOD(dataVec, tag); + appendPOD(dataVec, e.size); + appendPOD(dataVec, e.alignment); + appendPOD(dataVec, e.type); + appendPOD(dataVec, e.isArray); + } + + return { dataVec, {} }; +} + +template<> +MetadataListPlan +IPADataSerializer::deserialize([[maybe_unused]] std::vector::const_iterator dataBegin, + [[maybe_unused]] std::vector::const_iterator dataEnd, + [[maybe_unused]] std::vector::const_iterator fdsBegin, + [[maybe_unused]] std::vector::const_iterator fdsEnd, + [[maybe_unused]] ControlSerializer *cs) +{ + MetadataListPlan ret; + std::size_t offset = 0; + + auto count = readPOD(dataBegin, 0, dataEnd); + offset += sizeof(count); + + while (count--) { + auto tag = readPOD(dataBegin, offset, dataEnd); + offset += sizeof(tag); + + auto size = readPOD(dataBegin, offset, dataEnd); + offset += sizeof(size); + + auto alignment = readPOD(dataBegin, offset, dataEnd); + offset += sizeof(alignment); + + auto type = readPOD(dataBegin, offset, dataEnd); + offset += sizeof(type); + + auto isArray = readPOD(dataBegin, offset, dataEnd); + offset += sizeof(isArray); + + ret.add(tag, size, 1, alignment, static_cast(type), isArray); + } + + return ret; +} + +template<> +MetadataListPlan +IPADataSerializer::deserialize(std::vector::const_iterator dataBegin, + std::vector::const_iterator dataEnd, + ControlSerializer *cs) +{ + return deserialize(dataBegin, dataEnd, {}, {}, cs); +} + +template<> +MetadataListPlan +IPADataSerializer::deserialize(const std::vector &data, + ControlSerializer *cs) +{ + return deserialize(data.cbegin(), data.end(), cs); +} + +template<> +MetadataListPlan +IPADataSerializer::deserialize(const std::vector &data, + const std::vector &fds, + ControlSerializer *cs) +{ + return deserialize(data.cbegin(), data.end(), fds.cbegin(), fds.end(), cs); +} + #endif /* __DOXYGEN__ */ } /* namespace libcamera */ diff --git a/utils/codegen/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl b/utils/codegen/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl index 3942e5708..b3774cd64 100644 --- a/utils/codegen/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl +++ b/utils/codegen/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl @@ -21,6 +21,7 @@ #include #include #include +#include #include From patchwork Fri Jun 6 16:41:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23491 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A4E24C31E9 for ; Fri, 6 Jun 2025 16:42:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1027368DCA; Fri, 6 Jun 2025 18:42:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HalIKhkm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE02668DC9 for ; Fri, 6 Jun 2025 18:42:17 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 34F9B11DD for ; Fri, 6 Jun 2025 18:42:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228133; bh=M1CNsryuizApYzNyJSEgga9OUl+tNty6V9mro8Qlmb8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=HalIKhkmAyhAiZsryUT2alG/N+9G3ymo1RISnYj7jRsODK5pIKszIOWzsuU1AjsRn s3kaVOLgcZImzxWCaBrkBXhnXoOyaLqiduQTAsbJICK6i1YD4GC++P34SJqd+SUdpu JJHoU1Vw2fSLd2mjwXOhyEHTaLmZV8itk0QLxh/I= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 10/23] libcamera: camera: Store `MetadataListPlan` in `Camera::Private` Date: Fri, 6 Jun 2025 18:41:43 +0200 Message-ID: <20250606164156.1442682-11-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Just like `ControlInfoMap controlInfo_` is a public member of the private camera data for pipeline handlers to populate, add a `MetadataListPlan` as well for the pipeline handlers to fill. This will be used to initialize the `MetadataList` of each request created by the camera. Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- include/libcamera/internal/camera.h | 3 +++ src/libcamera/camera.cpp | 8 ++++++++ 2 files changed, 11 insertions(+) diff --git a/include/libcamera/internal/camera.h b/include/libcamera/internal/camera.h index 18f5c32a1..d5d869887 100644 --- a/include/libcamera/internal/camera.h +++ b/include/libcamera/internal/camera.h @@ -17,6 +17,7 @@ #include #include +#include namespace libcamera { @@ -41,6 +42,8 @@ public: uint32_t requestSequence_; + MetadataListPlan metadataPlan_; + const CameraControlValidator *validator() const { return validator_.get(); } private: diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index c180a5fdd..31f5ad94b 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -660,6 +660,14 @@ Camera::Private::~Private() * over a single capture session. */ +/** + * \var Camera::Private::metadataPlan_ + * \brief The set of metadata supported by the camera + * + * The metadata information shall be initialised by the pipeline handler when + * creating the camera. + */ + static const char *const camera_state_names[] = { "Available", "Acquired", From patchwork Fri Jun 6 16:41:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23492 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C5CECC3329 for ; Fri, 6 Jun 2025 16:42:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3AD3E68DC9; Fri, 6 Jun 2025 18:42:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BBZ/ndM8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D8A568DCA for ; Fri, 6 Jun 2025 18:42:18 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 735018DB for ; Fri, 6 Jun 2025 18:42:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228133; bh=unLwYMysziuAD650a3MXzhC8IyXlSvsliMIvZqJ2L4w=; h=From:To:Subject:Date:In-Reply-To:References:From; b=BBZ/ndM8f9gdhUUKNyxW4J/QiUayRXMf0kd9RDFppnu3Rhasgb6VOTRiiv4e645pI uoL5NqEa7sBY0j+9xUoBC/KzxzqOdI0aaxwI11kUgRDKErikVFaToDpDDj8hEV/m30 6HnrhOF3LXV06ndMmBgcL9GhrXH8QqUVK+mZ9C2s= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 11/23] libcamera: request: Store `MetadataList` Date: Fri, 6 Jun 2025 18:41:44 +0200 Message-ID: <20250606164156.1442682-12-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a `MetadataList` to the `Request` object to store the metadata items returned by the camera. The metadata list is initialized using the camera's `MetadataListPlan` object, which is supposed to be filled by the pipeline handlers. Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- include/libcamera/request.h | 5 +++++ src/libcamera/request.cpp | 2 ++ 2 files changed, 7 insertions(+) diff --git a/include/libcamera/request.h b/include/libcamera/request.h index e214a9d13..171a40903 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -18,6 +18,7 @@ #include #include +#include namespace libcamera { @@ -51,6 +52,9 @@ public: ControlList &controls() { return *controls_; } ControlList &metadata() { return *metadata_; } +#ifndef __DOXYGEN__ + [[nodiscard]] auto &metadata2() { return metadata2_; } +#endif const BufferMap &buffers() const { return bufferMap_; } int addBuffer(const Stream *stream, FrameBuffer *buffer, std::unique_ptr fence = nullptr); @@ -69,6 +73,7 @@ private: ControlList *controls_; ControlList *metadata_; + MetadataList metadata2_; BufferMap bufferMap_; const uint64_t cookie_; diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 7f1e11e8f..17ca67d08 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -354,6 +354,7 @@ void Request::Private::timeout() */ Request::Request(Camera *camera, uint64_t cookie) : Extensible(std::make_unique(camera)), + metadata2_(camera->_d()->metadataPlan_), cookie_(cookie), status_(RequestPending) { controls_ = new ControlList(controls::controls, @@ -407,6 +408,7 @@ void Request::reuse(ReuseFlag flags) controls_->clear(); metadata_->clear(); + metadata2_.clear(); } /** From patchwork Fri Jun 6 16:41:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23493 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD3EDC332A for ; Fri, 6 Jun 2025 16:42:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 74C9668DD0; Fri, 6 Jun 2025 18:42:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KpNH4fI+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4CE3868DCC for ; Fri, 6 Jun 2025 18:42:18 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CB4246DC for ; Fri, 6 Jun 2025 18:42:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228133; bh=QG9CvMx9CE2zEBWnL9bIIerDbEtEB9yOXsiQeAi1mOE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=KpNH4fI+EtmHu2nfQQnb8DTGEm29ticXZa8hGFhnUtvrr434zLCuRBg4cUI0GlUuS 4MyQNXjG2n1xhh/7f8xgtOER5jqR8zNjy+jGAiHpFk7wBvfBPcjyXeotfWXyqgZFdU f58AHLhbQpelqs/Ij+33jfezhLrDxBGS3pBonkxA= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 12/23] [DNI] apps: cam: Print `MetadataList` of `Request` as well Date: Fri, 6 Jun 2025 18:41:45 +0200 Message-ID: <20250606164156.1442682-13-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the "--metadata" option is enabled, print the dedicated `MetadataList` of the request as well. --- src/apps/cam/camera_session.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/apps/cam/camera_session.cpp b/src/apps/cam/camera_session.cpp index f63fcb228..f6894340b 100644 --- a/src/apps/cam/camera_session.cpp +++ b/src/apps/cam/camera_session.cpp @@ -510,11 +510,19 @@ void CameraSession::processRequest(Request *request) if (printMetadata_) { const ControlList &requestMetadata = request->metadata(); + std::cout << "Metadata (" << requestMetadata.size() << " entries):\n"; for (const auto &[key, value] : requestMetadata) { const ControlId *id = controls::controls.at(key); std::cout << "\t" << id->name() << " = " << value.toString() << std::endl; } + + const auto &requestMetadata2 = request->metadata2(); + std::cout << "Metadata2 (" << requestMetadata2.size() << " entries):\n"; + for (auto &&[tag, v] : requestMetadata2) { + const ControlId *id = controls::controls.at(tag); + std::cout << '\t' << id->name() << " = " << v << std::endl; + } } /* From patchwork Fri Jun 6 16:41:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23494 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F2315C332B for ; Fri, 6 Jun 2025 16:42:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 709D468DD3; Fri, 6 Jun 2025 18:42:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rNex3+cf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 907C268DCD for ; Fri, 6 Jun 2025 18:42:18 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 13B208DB; Fri, 6 Jun 2025 18:42:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228134; bh=SW2O+QHlgvwrk3GucZFzQSATnJnlcKEwg8tmUIUhFmk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rNex3+cf1iBoa4vspHhSvEcnoFETBz5lt79zH5uC6oih5dICfC4hIsgcF0yGNndko bB9SqhSROqUr5pJcDm1pj/7KcFcR6/xPByEBqvalb0maMHmoCSCMLFPIIGhGRCCgwB h2/0xHE/ngBwulw6BU0dAG0af2ASfw8xeoR3kkTk= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [RFC PATCH v1 13/23] libcamera: camera: Introduce metadataAvailable signal Date: Fri, 6 Jun 2025 18:41:46 +0200 Message-ID: <20250606164156.1442682-14-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add a new signal to the Camera class that allows applications to receive notifications for early completion of metadata results. To avoid expensive copies of the metadata results the signal transports the ids of the metadata keys that are ready. Applications can use these ids to access the metadata results from Request::metadata(). The signal is an opt-in feature for applications and the sum of all metadata results notified through this signal is available in Request::metadata() at request completion time. Signed-off-by: Jacopo Mondi [Use `MetadataList::Diff`, change documentation accordingly.] Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- Original: https://patchwork.libcamera.org/patch/22227/ --- include/libcamera/camera.h | 1 + src/libcamera/camera.cpp | 54 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 94cee7bd8..bf5623ddf 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -122,6 +122,7 @@ public: const std::string &id() const; + Signal metadataAvailable; Signal bufferCompleted; Signal requestCompleted; Signal<> disconnected; diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 31f5ad94b..2c87c3274 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -902,6 +902,60 @@ const std::string &Camera::id() const return _d()->id_; } +/** + * \var Camera::metadataAvailable + * \brief Signal emitted when metadata for a request is available + * + * The metadataAvailable signal notifies applications about the availability + * of metadata for a request before the request completes. + * + * As metadata results could be large in size, the signal transports only a view + * object via which the newly completed metadata items can be accessed. Similarly + * to the metadata list itself, this object is thread-safe, and can be sent to other + * threads for deferred processing. The view object is valid until the request is + * destroyed or reused, whichever happens first. + * + * Applications can access the value of the newly available metadata results as follows: + * + * \code + + void metadataAvailableHandler(Request *request, MetadataList::Diff update) + { + // The object can be iterated... + for (auto &&[id, data] : update) { + // `id` is the numeric identifier + // `data` is a `ControlValueView` object + } + + // ...or individual items can be looked up. + if (auto x = update.get(controls::SensorTimestamp)) { + // `SensorTimestamp` will only be found if it is part of this + // particular update; metadata items completed earlier will + // not be found. + } + } + \endcode + * + * This signal is emitted multiple times for the same request, it is in fact + * emitted by the framework every time a new metadata list is made available + * by the Camera to the application. + * + * The sum of all metadata lists reported through this signal is equal to + * Request::metadata() list when the Request completes. + * + * Application can opt-in to handle this signal to receive fast notifications + * of metadata availability or can equally access the full metadata list + * at Request complete time through Request::metadata() if they have no interest + * in early metadata notification. + * + * \note The received MetadataList::Diff object is merely a view, it is only valid until + * the associated Request is destroyed or \ref Request::reuse() "reused". However, + * during its valid lifetime, an application is free to create copies and access the + * completed metadata items from separate thread by iteration or MetadataList::Diff::get(). + * + * \sa ControlValueView + */ + /** * \var Camera::bufferCompleted * \brief Signal emitted when a buffer for a request queued to the camera has From patchwork Fri Jun 6 16:41:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23495 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 02FF4C3324 for ; Fri, 6 Jun 2025 16:42:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 87F8C68DD2; Fri, 6 Jun 2025 18:42:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="g01j8VDP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D4EA568DCE for ; Fri, 6 Jun 2025 18:42:18 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 59EC76DC; Fri, 6 Jun 2025 18:42:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228134; bh=L8vegDVV/VxgKR16hc7hqlqqDyd87dQzWT7piV86QbQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g01j8VDPtBX1lw01WQgXDBNg7qNe6O6LFP4q3lkDvAiyaXBW7Uec0Uu1OTbTiMIup zppaySxqiKo3kdtoG2CGYx4uNepdSnbM8vROujO9XjimYxvGAUNReQmuaL/xqwmpzk 7PDKuh9J1KblHgctYcZhigyIEPci/XZEYEkYRdiY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [RFC PATCH v1 14/23] guides: application: Document Camera::metadataAvailable Date: Fri, 6 Jun 2025 18:41:47 +0200 Message-ID: <20250606164156.1442682-15-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Document the Camera::metadataAvailable signal in the application developer guide as an opt-in feature to receive early metadata notifications. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- Original: https://patchwork.libcamera.org/patch/22228/ --- Documentation/guides/application-developer.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/Documentation/guides/application-developer.rst b/Documentation/guides/application-developer.rst index 6501345a9..2d7c29010 100644 --- a/Documentation/guides/application-developer.rst +++ b/Documentation/guides/application-developer.rst @@ -357,6 +357,14 @@ Signals and Slots`_) to connect events with callbacks to handle them. The ``Camera`` device emits two signals that applications can connect to in order to execute callbacks on frame completion events. +The ``Camera::metadataAvailable`` signal notifies applications of the +availability of metadata results before a request completes. Receiving +notification about metadata availability allows application to fast-track +handling of metadata results before all the image buffers in a request are +ready. The full list of metadata results associated with a Request is anyway +available at request complete time, and receiving notifications for early +metadata availability is an optional feature for applications. + The ``Camera::bufferCompleted`` signal notifies applications that a buffer with image data is available. Receiving notifications about the single buffer completion event allows applications to implement partial request completion From patchwork Fri Jun 6 16:41:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23496 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B63AC332C for ; Fri, 6 Jun 2025 16:42:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C1C5B68DD7; Fri, 6 Jun 2025 18:42:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FRmvPFIQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2A08968DC8 for ; Fri, 6 Jun 2025 18:42:19 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A03FB8DB; Fri, 6 Jun 2025 18:42:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228134; bh=KEBykHu4ZK0hxfanIyPCBBbc35h6VOsKusO/SB43KhA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FRmvPFIQWK/ZwqcQdIEFfjZYH3EdO1tO1KnYBnWuD2zOqTP/25ooLhJlbKpYEwfHk d7KliCAlB+yc1Gxuzqw6Rg8Z4Qt+iVI299ZcXkA/g/MpD33dUF3OlMINx3GAgltWlq bajPE7m4uveEZ3/xGw0qappG/nAR5ExpZIBdetQ4= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [RFC PATCH v1 15/23] libcamera: pipeline_handler: Add metadataAvailable() function Date: Fri, 6 Jun 2025 18:41:48 +0200 Message-ID: <20250606164156.1442682-16-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Currently the way pipeline handlers have to store metadata results for a Request is to access the Request::metadata_ list directly and either merge a ControlList or set a single control value there. Direct access to Request::metadata_ is however problematic as even if metadata would be available earlier, pipeline handlers can only accumulate the results in Request::metadata_ and they're only available for applications at Request complete time. Instead of letting pipeline handlers access Request::metadata_ directly provide two helper functions, similar in spirit to PipelineHandler::completeBuffer() and PipelineHandler::completeRequest(), to allow pipeline handlers to notify early availability of metadata. Provide two overloads, one that accepts a ControlList and merges it into Request::metadata_ and one that allows to set a single metadata result there without going through an intermediate copy. The newly provided helpers trigger the Camera::availableMetadata signal from where applications can retrieve the list of controls that have just been made available by the pipeline handler. Signed-off-by: Jacopo Mondi [Fill both metadat lists, use `type_identity`, adjust commit message.] Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- Original: https://patchwork.libcamera.org/patch/22229/ --- include/libcamera/internal/pipeline_handler.h | 21 +++++++ src/libcamera/pipeline_handler.cpp | 61 +++++++++++++++++++ 2 files changed, 82 insertions(+) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index 972a2fa65..bdec80589 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -13,11 +13,15 @@ #include #include +#include #include +#include #include #include +#include "libcamera/internal/request.h" + namespace libcamera { class Camera; @@ -58,6 +62,23 @@ public: void registerRequest(Request *request); void queueRequest(Request *request); + void metadataAvailable(Request *request, const ControlList &metadata); + + template + void metadataAvailable(Request *request, const Control &ctrl, + const details::cxx20::type_identity_t &value) + { + auto &m = request->metadata2(); + const auto c = m.checkpoint(); + + m.set(ctrl, value); + request->metadata().set(ctrl, value); + + const auto d = c.diffSince(); + if (d) + request->_d()->camera()->metadataAvailable.emit(request, d); + } + bool completeBuffer(Request *request, FrameBuffer *buffer); void completeRequest(Request *request); void cancelRequest(Request *request); diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index d84dff3c9..6a9d11241 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -507,6 +507,67 @@ void PipelineHandler::doQueueRequests() * \return 0 on success or a negative error code otherwise */ +/** + * \brief Notify the availability of a list of metadata for \a request + * \param[in] request The request the metadata belongs to + * \param[in] metadata The metadata list + * + * This function should be called multiple times by pipeline handlers to signal + * the availability of a list of metadata results. It notifies applications + * by triggering the Camera::availableMetadata signal and accumulates the + * metadata results in Request::metadata(). + * + * Early metadata completion allows pipeline handlers to fast track delivery of + * metadata results as soon as they are available before the completion of \a + * request. The full list of metadata results of a Request is available at + * Request completion time in Request::metadata(). + * + * A metadata key is expected to be notified at most once. Metadata keys + * notified multiple times are ignored. + * + * This overload allows to signal the availability of a list of metadata and + * merges them in the Request::metadata() list. This operations is expensive + * as controls are copied from \a metadata to Request::metadata(). + * + * \context This function shall be called from the CameraManager thread. + */ +void PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata) +{ + request->metadata().merge(metadata); + + const auto d = request->metadata2().merge(metadata); + if (d) + request->_d()->camera()->metadataAvailable.emit(request, d); +} + +/** + * \fn void PipelineHandler::metadataAvailable(Request *request, const Control &ctrl, + * const details::cxx20::type_identity_t &value) + * \brief Notify the availability of a metadata result for \a request + * \param[in] request The request the metadata belongs to + * \param[in] ctrl The control id to notify + * \param[in] value the control value + * + * This function should be called multiple times by pipeline handlers to signal + * the availability of a metadata result. It notifies applications + * by triggering the Camera::availableMetadata signal and accumulates the + * metadata result in Request::metadata(). + * + * Early metadata completion allows pipeline handlers to fast track delivery of + * metadata results as soon as they are available before the completion of \a + * request. The full list of metadata results of a Request is available at + * Request completion time in Request::metadata(). + * + * A metadata key is expected to be notified at most once. Metadata keys + * notified multiple times are ignored. + * + * This overload allows to signal the availability of a single metadata and + * merge \a value in the Request::metadata() list. This operations copies \a + * value in the Request::metadata() list without creating intermediate copies. + * + * \context This function shall be called from the CameraManager thread. + */ + /** * \brief Complete a buffer for a request * \param[in] request The request the buffer belongs to From patchwork Fri Jun 6 16:41:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23497 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 60C7BC332D for ; Fri, 6 Jun 2025 16:42:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB4D168DC9; Fri, 6 Jun 2025 18:42:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O3tCDzf2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6CD3D68DD0 for ; Fri, 6 Jun 2025 18:42:19 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E79496DC; Fri, 6 Jun 2025 18:42:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228135; bh=5tsQOQ4pRrQ2k22MlYaRs2MsmppIelQld7OjJgwPHsg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O3tCDzf2D1C3ch1+vZw5N87WnbQ0GSwZyqkdbihdNy+piXcVDl5E4fFeOzpFOqUJb suXzrZ/iEwggThcJoVRAps6h0HzSUnoqWEwJAPymKPJKOmUSLpq94uVaKujK5SwURY k4vhN60NzVclwdeewenRDMRwBzVOTpylXNvvMTiI= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [RFC PATCH v1 16/23] guides: pipeline_handler: Document PipelineHandler::metadataAvailable Date: Fri, 6 Jun 2025 18:41:49 +0200 Message-ID: <20250606164156.1442682-17-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Document the new available helpers to accumulate metadata results in Request::metadata and notify applications about metadata availability. Signed-off-by: Jacopo Mondi Signed-off-by: Barnabás Pőcze --- Original: https://patchwork.libcamera.org/patch/22230/ --- Documentation/guides/pipeline-handler.rst | 26 ++++++++++++++++------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/Documentation/guides/pipeline-handler.rst b/Documentation/guides/pipeline-handler.rst index 0d5df7269..2b5e8414d 100644 --- a/Documentation/guides/pipeline-handler.rst +++ b/Documentation/guides/pipeline-handler.rst @@ -1425,18 +1425,28 @@ classes documentation. .. _libcamera Signal and Slot: https://libcamera.org/api-html/classlibcamera_1_1Signal.html#details In order to notify applications about the availability of new frames and data, -the ``Camera`` device exposes two ``Signals`` to which applications can connect -to be notified of frame completion events. The ``bufferComplete`` signal serves -to report to applications the completion event of a single ``Stream`` part of a -``Request``, while the ``requestComplete`` signal notifies the completion of all -the ``Streams`` and data submitted as part of a request. This mechanism allows +the ``Camera`` device exposes three ``Signals`` to which applications can +connect to be notified of frame completion and metadata availability events. + +The ``metadataAvailable`` signal serves to notify about the availability of +metadata and accumulates metadata results in the list of metadata associated +with a ``Request``. The ``bufferComplete`` signal serves to report to +applications the completion event of a single ``Stream`` part of a ``Request``, +while the ``requestComplete`` signal notifies the completion of all the +``Streams`` and data submitted as part of a request. This mechanism allows implementation of partial request completion, which allows an application to inspect completed buffers associated with the single streams without waiting for all of them to be ready. -The ``bufferComplete`` and ``requestComplete`` signals are emitted by the -``Camera`` device upon notifications received from the pipeline handler, which -tracks the buffers and request completion status. +The ``metadataAvailable``, ``bufferComplete`` and ``requestComplete`` signals +are emitted by the ``Camera`` device upon notifications received from the +pipeline handler, which tracks the metadata, buffers and request completion +status. + +Metadata availability is signalled by the pipeline handlers by calling the +PipelineHandler base class ``metadataAvailable`` function. This function +notifies applications about metadata availability and accumulates metadata +results in the ``Request::metadata()`` list. The single buffer completion notification is implemented by pipeline handlers by `connecting`_ the ``bufferReady`` signal of the capture devices they have queued From patchwork Fri Jun 6 16:41:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23498 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 79026C3325 for ; Fri, 6 Jun 2025 16:42:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F358068DDB; Fri, 6 Jun 2025 18:42:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="epNGjGQm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BAC2B68DD1 for ; Fri, 6 Jun 2025 18:42:19 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3AE068DB; Fri, 6 Jun 2025 18:42:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228135; bh=lvSUfnI63UmjzaOOoai8j6Ns3gn4AujTEvGIQVxbfUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=epNGjGQm7DGKFkzaGvHZPh7dGhZEdfMhYFzvk2F/lIiOZymn0WthoL/JhQxFtUJ4b vUuW/dkFViyxPUv3wg5gWlNOk1FdrRvZF8e8iwbQ4PjTqUGdYh2D7i5iT6OUaCXASa HVUH/6s+Y45Zdp3jTun9BudSrbnCMSzTKsYrRJos= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [RFC PATCH v1 17/23] [DNI] apps: cam: Use Camera::metadataAvailable signal Date: Fri, 6 Jun 2025 18:41:50 +0200 Message-ID: <20250606164156.1442682-18-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Handle the Camera::metadataAvailable signal and print the metadata list. Use the --metadata option of cam to validate that the metadata list in Request::metadata() matches the accumulated results. --- Original: https://patchwork.libcamera.org/patch/22234/ --- src/apps/cam/camera_session.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/apps/cam/camera_session.cpp b/src/apps/cam/camera_session.cpp index f6894340b..e50b26a5a 100644 --- a/src/apps/cam/camera_session.cpp +++ b/src/apps/cam/camera_session.cpp @@ -285,6 +285,17 @@ int CameraSession::start() camera_->requestCompleted.connect(this, &CameraSession::requestComplete); + if (printMetadata_) { + camera_->metadataAvailable.connect(this, [](Request *r, MetadataList::Diff update) { + std::cout << ">> early metadata for " << r->sequence() << " with " << update.size() << " entries {\n"; + for (auto &&[tag, v] : update) { + const auto *id = controls::controls.at(tag); + std::cout << '\t' << id->name() << " = " << v << '\n'; + } + std::cout << "}" << std::endl; + }); + } + #ifdef HAVE_KMS if (options_.isSet(OptDisplay)) sink_ = std::make_unique(options_[OptDisplay].toString()); From patchwork Fri Jun 6 16:41:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23500 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EC19CC332F for ; Fri, 6 Jun 2025 16:42:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E3F168DDF; Fri, 6 Jun 2025 18:42:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XsaOqk/8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0EF4368DBE for ; Fri, 6 Jun 2025 18:42:20 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 85A1C6DC for ; Fri, 6 Jun 2025 18:42:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228135; bh=It+Q59xjSMgFvh2ZWYUB5X0W02YEhlkpD+do6j4AZWg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=XsaOqk/8uqgc9zVp3qGOp81d4mnZ+vl+PDX1VPUXGwrdZvwlST1j6qmTA4lpYGmQv kG+6R10aKXItGLrInRa/4xOKWQatRQ3PfgZQp2S+24dN3/xfuzlYKUg3VeYIi2Hz49 TdKya/1944rfxeRTnZNlN/UiL0zxz5+CswfNj7QU= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 18/23] libcamera: pipeline: Fill `MetadataListPlan` of cameras Date: Fri, 6 Jun 2025 18:41:51 +0200 Message-ID: <20250606164156.1442682-19-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fill the newly introduced `MetadataListPlan` member of the camera's private data during initializations, similarly to the camera's `ControlInfoMap`. Signed-off-by: Barnabás Pőcze Reviewed-by: Jacopo Mondi --- .../internal/software_isp/software_isp.h | 3 +- include/libcamera/ipa/ipu3.mojom | 3 +- include/libcamera/ipa/mali-c55.mojom | 3 +- include/libcamera/ipa/raspberrypi.mojom | 1 + include/libcamera/ipa/rkisp1.mojom | 2 +- include/libcamera/ipa/soft.mojom | 3 +- src/ipa/ipu3/algorithms/agc.cpp | 4 +++ src/ipa/ipu3/algorithms/awb.cpp | 12 +++++++ src/ipa/ipu3/algorithms/awb.h | 1 + src/ipa/ipu3/ipa_context.cpp | 3 ++ src/ipa/ipu3/ipa_context.h | 3 ++ src/ipa/ipu3/ipu3.cpp | 8 +++-- src/ipa/mali-c55/algorithms/agc.cpp | 5 +++ src/ipa/mali-c55/algorithms/awb.cpp | 7 ++++ src/ipa/mali-c55/algorithms/awb.h | 1 + src/ipa/mali-c55/algorithms/blc.cpp | 2 ++ src/ipa/mali-c55/ipa_context.h | 3 ++ src/ipa/mali-c55/mali-c55.cpp | 6 ++-- src/ipa/rkisp1/algorithms/agc.cpp | 10 ++++++ src/ipa/rkisp1/algorithms/awb.cpp | 4 +++ src/ipa/rkisp1/algorithms/blc.cpp | 2 ++ src/ipa/rkisp1/ipa_context.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 8 +++-- src/ipa/rpi/common/ipa_base.cpp | 34 +++++++++++++++++++ src/ipa/rpi/pisp/pisp.cpp | 5 +-- src/ipa/rpi/vc4/vc4.cpp | 4 ++- src/ipa/simple/algorithms/agc.cpp | 8 +++++ src/ipa/simple/algorithms/agc.h | 1 + src/ipa/simple/algorithms/awb.cpp | 8 +++++ src/ipa/simple/algorithms/awb.h | 1 + src/ipa/simple/algorithms/blc.cpp | 3 ++ src/ipa/simple/algorithms/ccm.cpp | 3 ++ src/ipa/simple/algorithms/lut.cpp | 3 ++ src/ipa/simple/ipa_context.h | 2 ++ src/ipa/simple/soft_simple.cpp | 8 +++-- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 ++ src/libcamera/pipeline/ipu3/ipu3.cpp | 7 +++- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++- .../pipeline/rpi/common/pipeline_base.cpp | 7 ++++ src/libcamera/pipeline/simple/simple.cpp | 2 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 2 ++ src/libcamera/pipeline/vimc/vimc.cpp | 2 ++ .../pipeline/virtual/config_parser.cpp | 3 ++ src/libcamera/software_isp/software_isp.cpp | 6 ++-- 45 files changed, 190 insertions(+), 22 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 786246592..bd85f6b4b 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -41,6 +41,7 @@ class DebayerCpu; class FrameBuffer; class PixelFormat; class Stream; +class MetadataListPlan; struct StreamConfiguration; LOG_DECLARE_CATEGORY(SoftwareIsp) @@ -49,7 +50,7 @@ class SoftwareIsp : public Object { public: SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, - ControlInfoMap *ipaControls); + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan); ~SoftwareIsp(); int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d9a50b01d..f49b77797 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -20,7 +20,8 @@ interface IPAIPU3Interface { init(libcamera.IPASettings settings, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, + libcamera.MetadataListPlan metadata); start() => (int32 ret); stop(); diff --git a/include/libcamera/ipa/mali-c55.mojom b/include/libcamera/ipa/mali-c55.mojom index 5d7eb4eef..7229c53e1 100644 --- a/include/libcamera/ipa/mali-c55.mojom +++ b/include/libcamera/ipa/mali-c55.mojom @@ -11,7 +11,8 @@ struct IPAConfigInfo { interface IPAMaliC55Interface { init(libcamera.IPASettings settings, IPAConfigInfo configInfo) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, + libcamera.MetadataListPlan metadataPlan); start() => (int32 ret); stop(); diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index e30c70bde..69160e133 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -26,6 +26,7 @@ struct InitParams { struct InitResult { SensorConfig sensorConfig; libcamera.ControlInfoMap controlInfo; + libcamera.MetadataListPlan metadataPlan; }; struct BufferIds { diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index 043ad27ea..440a59977 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -19,7 +19,7 @@ interface IPARkISP1Interface { uint32 hwRevision, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, libcamera.MetadataListPlan metadataPlan); start() => (int32 ret); stop(); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 77328c5fd..360ed668b 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -18,7 +18,8 @@ interface IPASoftInterface { libcamera.SharedFD fdParams, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); + => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled, + libcamera.MetadataListPlan metadataPlan); start() => (int32 ret); stop(); configure(IPAConfigInfo configInfo) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 39d0aebb0..e3b2acbc6 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -82,6 +82,10 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) context.ctrlMap.merge(controls()); + context.metadataPlan.add(controls::AnalogueGain); + context.metadataPlan.add(controls::ExposureTime); + context.metadataPlan.add(controls::FrameDuration); + return 0; } diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 55de05d9e..28e29e29c 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -197,6 +197,18 @@ Awb::Awb() Awb::~Awb() = default; +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.add(controls::AwbEnable); + context.metadataPlan.add(controls::ColourGains); + context.metadataPlan.add(controls::ColourTemperature); + + return 0; +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index dbf69c907..1d47bffa3 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,6 +40,7 @@ public: Awb(); ~Awb(); + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 3b22f7917..f0b8b5dbe 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -54,6 +54,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAContext::ctrlMap * \brief A ControlInfoMap::Map of controls populated by the algorithms + * + * \var IPAContext::metadataPlan + * \brief A MetadataListPlan populated by the algorithms */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 97fcf06cd..f4f45ef4d 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -14,6 +14,7 @@ #include #include +#include #include @@ -95,6 +96,8 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1cae08bf2..8d8811f71 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -143,7 +143,8 @@ public: int init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) override; int start() override; void stop() override; @@ -299,7 +300,8 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, int IPAIPU3::init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (camHelper_ == nullptr) { @@ -348,6 +350,8 @@ int IPAIPU3::init(const IPASettings &settings, /* Initialize controls. */ updateControls(sensorInfo, sensorControls, ipaControls); + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 70667db34..bfca8f5a5 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -145,6 +145,11 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) ); context.ctrlMap.merge(controls()); + context.metadataPlan.add(controls::ExposureTime); + context.metadataPlan.add(controls::AnalogueGain); + context.metadataPlan.add(controls::DigitalGain); + context.metadataPlan.add(controls::ColourTemperature); + return 0; } diff --git a/src/ipa/mali-c55/algorithms/awb.cpp b/src/ipa/mali-c55/algorithms/awb.cpp index 050b191b7..b5acd4390 100644 --- a/src/ipa/mali-c55/algorithms/awb.cpp +++ b/src/ipa/mali-c55/algorithms/awb.cpp @@ -29,6 +29,13 @@ Awb::Awb() { } +int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.add(controls::ColourGains); + + return 0; +} + int Awb::configure([[maybe_unused]] IPAContext &context, [[maybe_unused]] const IPACameraSensorInfo &configInfo) { diff --git a/src/ipa/mali-c55/algorithms/awb.h b/src/ipa/mali-c55/algorithms/awb.h index 800c2e834..db59e8d79 100644 --- a/src/ipa/mali-c55/algorithms/awb.h +++ b/src/ipa/mali-c55/algorithms/awb.h @@ -18,6 +18,7 @@ public: Awb(); ~Awb() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, diff --git a/src/ipa/mali-c55/algorithms/blc.cpp b/src/ipa/mali-c55/algorithms/blc.cpp index 2a54c86a9..0aaaae7dd 100644 --- a/src/ipa/mali-c55/algorithms/blc.cpp +++ b/src/ipa/mali-c55/algorithms/blc.cpp @@ -51,6 +51,8 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, tuningParameters_ = true; + context.metadataPlan.add(controls::SensorBlackLevels); + LOG(MaliC55Blc, Debug) << "Black levels: 00 " << offset00 << ", 01 " << offset01 << ", 10 " << offset10 << ", 11 " << offset11; diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index 5e3e2fbde..d76fdcb2b 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -9,6 +9,7 @@ #include #include +#include #include "libcamera/internal/bayer_format.h" @@ -83,6 +84,8 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? }; } /* namespace ipa::mali_c55 */ diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index c6941a950..7006ae55c 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -46,7 +46,7 @@ public: IPAMaliC55(); int init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan) override; int start() override; void stop() override; int configure(const IPAConfigInfo &ipaConfig, uint8_t bayerOrder, @@ -96,7 +96,7 @@ std::string IPAMaliC55::logPrefix() const } int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (!camHelper_) { @@ -131,6 +131,8 @@ int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig updateControls(ipaConfig.sensorInfo, ipaConfig.sensorControls, ipaControls); + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 137a07500..22db11b24 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -159,6 +159,16 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true, true); context.ctrlMap.merge(controls()); + context.metadataPlan.add(controls::AnalogueGain); + context.metadataPlan.add(controls::ExposureTime); + context.metadataPlan.add(controls::FrameDuration); + context.metadataPlan.add(controls::FrameDuration); + context.metadataPlan.add(controls::ExposureTimeMode); + context.metadataPlan.add(controls::AnalogueGainMode); + context.metadataPlan.add(controls::AeMeteringMode); + context.metadataPlan.add(controls::AeExposureMode); + context.metadataPlan.add(controls::AeConstraintMode); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 399fb51be..aaaa3cced 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -117,6 +117,10 @@ int Awb::init(IPAContext &context, const YamlObject &tuningData) const auto &src = awbAlgo_->controls(); cmap.insert(src.begin(), src.end()); + context.metadataPlan.add(controls::AwbEnable); + context.metadataPlan.add(controls::ColourGains); + context.metadataPlan.add(controls::ColourTemperature); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index 98cb7145e..3288f24e3 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -103,6 +103,8 @@ int BlackLevelCorrection::init(IPAContext &context, const YamlObject &tuningData << ", green (blue) " << blackLevelGreenB_ << ", blue " << blackLevelBlue_; + context.metadataPlan.add(controls::SensorBlackLevels); + return 0; } diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f0d504215..cd4b98921 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -200,6 +200,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? DebugMetadata debugMetadata; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 1ed7d7d92..88c454ea1 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -54,7 +54,8 @@ public: int init(const IPASettings &settings, unsigned int hwRevision, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) override; int start() override; void stop() override; @@ -135,7 +136,8 @@ std::string IPARkISP1::logPrefix() const int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) { /* \todo Add support for other revisions */ switch (hwRevision) { @@ -204,6 +206,8 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, /* Initialize controls. */ updateControls(sensorInfo, sensorControls, ipaControls); + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index e0f8b7e78..032d6c43b 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -180,6 +180,40 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + // TODO: only set those that can be reported by configured algorithms? + // TODO: move this somewhere else? + result->metadataPlan.add(controls::AnalogueGainMode); + result->metadataPlan.add(controls::ExposureTimeMode); + result->metadataPlan.add(controls::ExposureTime); + result->metadataPlan.add(controls::AnalogueGain); + result->metadataPlan.add(controls::AeMeteringMode); + result->metadataPlan.add(controls::AeConstraintMode); + result->metadataPlan.add(controls::AeExposureMode); + result->metadataPlan.add(controls::ExposureValue); + result->metadataPlan.add(controls::AwbEnable); + result->metadataPlan.add(controls::AwbMode); + result->metadataPlan.add(controls::ColourGains); + result->metadataPlan.add(controls::Brightness); + result->metadataPlan.add(controls::Contrast); + result->metadataPlan.add(controls::Saturation); + result->metadataPlan.add(controls::Sharpness); + result->metadataPlan.add(controls::draft::NoiseReductionMode); + result->metadataPlan.add(controls::FrameDuration); + result->metadataPlan.add(controls::SensorTemperature); + result->metadataPlan.add(controls::LensPosition); + result->metadataPlan.add(controls::DigitalGain); + result->metadataPlan.add(controls::AeState); + result->metadataPlan.add(controls::Lux); + result->metadataPlan.add(controls::ColourTemperature); + result->metadataPlan.add(controls::SensorBlackLevels); + result->metadataPlan.add(controls::FocusFoM); + result->metadataPlan.add(controls::ColourCorrectionMatrix); + result->metadataPlan.add(controls::AfState); + result->metadataPlan.add(controls::AfPauseState); + result->metadataPlan.add(controls::HdrMode); + result->metadataPlan.add(controls::HdrChannel); + result->metadataPlan.add(controls::FrameDurationLimits); + return platformInit(params, result); } diff --git a/src/ipa/rpi/pisp/pisp.cpp b/src/ipa/rpi/pisp/pisp.cpp index bb50a9e05..a640bec32 100644 --- a/src/ipa/rpi/pisp/pisp.cpp +++ b/src/ipa/rpi/pisp/pisp.cpp @@ -267,8 +267,7 @@ private: HdrStatus lastStitchHdrStatus_; }; -int32_t IpaPiSP::platformInit(const InitParams ¶ms, - [[maybe_unused]] InitResult *result) +int32_t IpaPiSP::platformInit(const InitParams ¶ms, InitResult *result) { const std::string &target = controller_.getTarget(); if (target != "pisp") { @@ -301,6 +300,8 @@ int32_t IpaPiSP::platformInit(const InitParams ¶ms, setDefaultConfig(); + result->metadataPlan.add(controls::rpi::PispStatsOutput, sizeof(pisp_statistics)); + return 0; } diff --git a/src/ipa/rpi/vc4/vc4.cpp b/src/ipa/rpi/vc4/vc4.cpp index ba43e4741..d02e06589 100644 --- a/src/ipa/rpi/vc4/vc4.cpp +++ b/src/ipa/rpi/vc4/vc4.cpp @@ -83,7 +83,7 @@ private: void *lsTable_; }; -int32_t IpaVc4::platformInit([[maybe_unused]] const InitParams ¶ms, [[maybe_unused]] InitResult *result) +int32_t IpaVc4::platformInit([[maybe_unused]] const InitParams ¶ms, InitResult *result) { const std::string &target = controller_.getTarget(); @@ -94,6 +94,8 @@ int32_t IpaVc4::platformInit([[maybe_unused]] const InitParams ¶ms, [[maybe_ return -EINVAL; } + result->metadataPlan.add(controls::rpi::Bcm2835StatsOutput, sizeof(bcm2835_isp_stats)); + return 0; } diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp index c46bb0ebe..47a499be6 100644 --- a/src/ipa/simple/algorithms/agc.cpp +++ b/src/ipa/simple/algorithms/agc.cpp @@ -41,6 +41,14 @@ Agc::Agc() { } +int Agc::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.add(controls::ExposureTime); + context.metadataPlan.add(controls::AnalogueGain); + + return 0; +} + void Agc::updateExposure(IPAContext &context, IPAFrameContext &frameContext, double exposureMSV) { /* diff --git a/src/ipa/simple/algorithms/agc.h b/src/ipa/simple/algorithms/agc.h index 112d9f5a1..00e70ea70 100644 --- a/src/ipa/simple/algorithms/agc.h +++ b/src/ipa/simple/algorithms/agc.h @@ -19,6 +19,7 @@ public: Agc(); ~Agc() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index cf567e894..cce4585f9 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -25,6 +25,14 @@ LOG_DEFINE_CATEGORY(IPASoftAwb) namespace ipa::soft::algorithms { +int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.add(controls::ColourGains); + context.metadataPlan.add(controls::ColourTemperature); + + return 0; +} + int Awb::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h index ad993f39c..b8ae63dcb 100644 --- a/src/ipa/simple/algorithms/awb.h +++ b/src/ipa/simple/algorithms/awb.h @@ -19,6 +19,7 @@ public: Awb() = default; ~Awb() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 8c1e9ed08..4b9284e54 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -34,6 +34,9 @@ int BlackLevel::init([[maybe_unused]] IPAContext &context, */ definedLevel_ = blackLevel.value() >> 8; } + + context.metadataPlan.add(controls::SensorBlackLevels); + return 0; } diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index 0a98406c1..3cab8eaa4 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -39,6 +39,9 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData context.ccmEnabled = true; context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); + context.metadataPlan.add(controls::ColourCorrectionMatrix); + context.metadataPlan.add(controls::Saturation); + return 0; } diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index d1d5f7271..0f99be544 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -28,6 +28,9 @@ int Lut::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) { context.ctrlMap[&controls::Contrast] = ControlInfo(0.0f, 2.0f, 1.0f); + + context.metadataPlan.add(controls::Contrast); + return 0; } diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index a471b80ae..f4ecf43db 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -12,6 +12,7 @@ #include #include +#include #include "libcamera/internal/matrix.h" #include "libcamera/internal/vector.h" @@ -97,6 +98,7 @@ struct IPAContext { IPAActiveState activeState; FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? bool ccmEnabled = false; }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index c94c4cd55..db721199d 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -56,7 +56,8 @@ public: const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, - bool *ccmEnabled) override; + bool *ccmEnabled, + MetadataListPlan *metadataPlan) override; int configure(const IPAConfigInfo &configInfo) override; int start() override; @@ -96,7 +97,8 @@ int IPASoftSimple::init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, - bool *ccmEnabled) + bool *ccmEnabled, + MetadataListPlan *metadataPlan) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (!camHelper_) { @@ -190,6 +192,8 @@ int IPASoftSimple::init(const IPASettings &settings, return -EINVAL; } + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index ecda426a6..1c6da2006 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -166,6 +166,8 @@ int ISICameraData::init() properties_ = sensor_->properties(); + metadataPlan_.add(controls::SensorTimestamp); + return 0; } diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e31e3879d..b4a4283f4 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1078,6 +1078,11 @@ int PipelineHandlerIPU3::registerCameras() if (ret) continue; + data->metadataPlan_.add(controls::draft::PipelineDepth); + data->metadataPlan_.add(controls::draft::TestPatternMode); + data->metadataPlan_.add(controls::ScalerCrop); + data->metadataPlan_.add(controls::SensorTimestamp); + const CameraSensorProperties::SensorDelays &delays = cio2->sensor()->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1187,7 +1192,7 @@ int IPU3CameraData::loadIPA() ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - sensorInfo, sensor->controls(), &ipaControls_); + sensorInfo, sensor->controls(), &ipaControls_, &metadataPlan_); if (ret) { LOG(IPU3, Error) << "Failed to initialise the IPU3 IPA"; return ret; diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 4acc091bd..19980f6d2 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -402,7 +402,7 @@ int MaliC55CameraData::loadIPA() ControlInfoMap ipaControls; ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, - &ipaControls); + &ipaControls, &metadataPlan_); if (ret) { LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; return ret; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 675f0a749..34be87087 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -394,7 +394,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) } ret = ipa_->init({ ipaTuningFile, sensor_->model() }, hwRevision, - sensorInfo, sensor_->controls(), &ipaControls_); + sensorInfo, sensor_->controls(), &ipaControls_, &metadataPlan_); if (ret < 0) { LOG(RkISP1, Error) << "IPA initialization failure"; return ret; @@ -1333,6 +1333,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) updateControls(data.get()); + data->metadataPlan_.add(controls::SensorTimestamp); + std::set streams{ &data->mainPathStream_, &data->selfPathStream_, diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 1f13e5230..98507a152 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -589,6 +589,9 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) data->controlInfo_ = ControlInfoMap(std::move(ctrlMap), result.controlInfo.idmap()); + /* Update `rpi::ScalerCrops` size for the corrent configuration. */ + data->metadataPlan_.add(controls::rpi::ScalerCrops, config->size()); + /* Setup the Video Mux/Bridge entities. */ for (auto &[device, link] : data->bridgeDevices_) { /* @@ -832,6 +835,10 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); + data->metadataPlan_ = std::move(result.metadataPlan); + data->metadataPlan_.add(controls::SensorTimestamp); + data->metadataPlan_.add(controls::ScalerCrop); + /* * The V4L2_CID_NOTIFY_GAINS control, if present, is used to inform the * sensor of the colour gains. It is defined to be a linear gain where diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb07051b..05387ca7c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -592,7 +592,7 @@ int SimpleCameraData::init() * Instantiate Soft ISP if this is enabled for the given driver and no converter is used. */ if (!converter_ && pipe->swIspEnabled()) { - swIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_); + swIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_, &metadataPlan_); if (!swIsp_->isValid()) { LOG(SimplePipeline, Warning) << "Failed to create software ISP, disabling software debayering"; diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 58aa0eb4c..e0036e3b5 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -603,6 +603,8 @@ int UVCCameraData::init(MediaDevice *media) controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + metadataPlan_.add(controls::SensorTimestamp); + /* * Close to allow camera to go into runtime-suspend, video_ will be * re-opened from acquireDevice() and validate(). diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 07273bd2b..f8a29da41 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -590,6 +590,8 @@ int VimcCameraData::init() controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + metadataPlan_.add(controls::SensorTimestamp); + /* Initialize the camera properties. */ properties_ = sensor_->properties(); diff --git a/src/libcamera/pipeline/virtual/config_parser.cpp b/src/libcamera/pipeline/virtual/config_parser.cpp index 1d3d9ba87..f10aea7bf 100644 --- a/src/libcamera/pipeline/virtual/config_parser.cpp +++ b/src/libcamera/pipeline/virtual/config_parser.cpp @@ -65,6 +65,9 @@ ConfigParser::parseConfigFile(File &file, PipelineHandler *pipe) controls[&controls::draft::FaceDetectMode] = ControlInfo(supportedFaceDetectModes); data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + + data->metadataPlan_.add(controls::SensorTimestamp); + configurations.push_back(std::move(data)); } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 28e2a360e..6da69daf9 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -71,10 +71,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \param[in] pipe The pipeline handler in use * \param[in] sensor Pointer to the CameraSensor instance owned by the pipeline * \param[out] ipaControls The IPA controls to update + * \param[out] metadataPlan The metadata plan to update * handler */ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan) : dmaHeap_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) @@ -146,7 +147,8 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, sensorInfo, sensor->controls(), ipaControls, - &ccmEnabled_); + &ccmEnabled_, + metadataPlan); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset(); From patchwork Fri Jun 6 16:41:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23504 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58973C3332 for ; Fri, 6 Jun 2025 16:42:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CACE968DE8; Fri, 6 Jun 2025 18:42:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="a2xtG28F"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5DE9F68DD2 for ; Fri, 6 Jun 2025 18:42:20 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CC9468DB; Fri, 6 Jun 2025 18:42:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228135; bh=Qxjl4B3Hqig3nKlEdVu8Edjg8HHs7ry8jruyMLKQ0TQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a2xtG28FYAEqT1ls6Uph+f1CBexUlNyZWHQf94n1CGDfn4CukWDT5UiFY1zRjl77I MZ3jirjozme889PCf06U322omY7Fnp147Zi8HlKyFRa+ZoQdLdUdvO+8cHRkMTrIht qSZpMHGWlvHdl6Es9lIVjXMtQFbMaOsW3d1gfSqY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi Subject: [RFC PATCH v1 19/23] libcamera: pipeline: Use `metadataAvailable()` Date: Fri, 6 Jun 2025 18:41:52 +0200 Message-ID: <20250606164156.1442682-20-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Use the newly introduced `metadataAvailable()` function to send metadata items to the application. Signed-off-by: Jacopo Mondi [Adjust commit message, split rpi changes.] Signed-off-by: Barnabás Pőcze --- Original: https://patchwork.libcamera.org/patch/22232/ --- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 5 +---- src/libcamera/pipeline/ipu3/ipu3.cpp | 14 +++++++------- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 7 +++---- src/libcamera/pipeline/simple/simple.cpp | 5 +++-- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 3 +-- src/libcamera/pipeline/vimc/vimc.cpp | 3 +-- src/libcamera/pipeline/virtual/virtual.cpp | 2 +- 8 files changed, 18 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index 1c6da2006..f07df2d59 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -1100,10 +1100,7 @@ void PipelineHandlerISI::bufferReady(FrameBuffer *buffer) Request *request = buffer->request(); /* Record the sensor's timestamp in the request metadata. */ - ControlList &metadata = request->metadata(); - if (!metadata.contains(controls::SensorTimestamp.id())) - metadata.set(controls::SensorTimestamp, - buffer->metadata().timestamp); + metadataAvailable(request, controls::SensorTimestamp, buffer->metadata().timestamp); completeBuffer(request, buffer); if (request->hasPendingBuffers()) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index b4a4283f4..0f59d0727 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1250,7 +1250,7 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) return; Request *request = info->request; - request->metadata().merge(metadata); + pipe()->metadataAvailable(request, metadata); info->metadataProcessed = true; if (frameInfos_.tryComplete(info)) @@ -1277,12 +1277,14 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) pipe()->completeBuffer(request, buffer); - request->metadata().set(controls::draft::PipelineDepth, 3); + pipe()->metadataAvailable(request, controls::draft::PipelineDepth, 3); + /* \todo Actually apply the scaler crop region to the ImgU. */ const auto &scalerCrop = request->controls().get(controls::ScalerCrop); if (scalerCrop) cropRegion_ = *scalerCrop; - request->metadata().set(controls::ScalerCrop, cropRegion_); + + pipe()->metadataAvailable(request, controls::ScalerCrop, cropRegion_); if (frameInfos_.tryComplete(info)) pipe()->completeRequest(request); @@ -1322,8 +1324,7 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); + pipe()->metadataAvailable(request, controls::SensorTimestamp, buffer->metadata().timestamp); info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); @@ -1417,8 +1418,7 @@ void IPU3CameraData::frameStart(uint32_t sequence) return; } - request->metadata().set(controls::draft::TestPatternMode, - *testPatternMode); + pipe()->metadataAvailable(request, controls::draft::TestPatternMode, *testPatternMode); } REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3, "ipu3") diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 19980f6d2..d1a107629 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -1523,7 +1523,7 @@ void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; frameInfo.statsDone = true; - frameInfo.request->metadata().merge(metadata); + metadataAvailable(frameInfo.request, metadata); tryComplete(&frameInfo); } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 34be87087..aa5ea8799 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -433,7 +433,7 @@ void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &meta if (!info) return; - info->request->metadata().merge(metadata); + pipe()->metadataAvailable(info->request, metadata); info->metadataProcessed = true; pipe()->tryCompleteRequest(info); @@ -1476,8 +1476,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - metadata.timestamp); + metadataAvailable(request, controls::SensorTimestamp, metadata.timestamp); if (isRaw_) { const ControlList &ctrls = @@ -1560,7 +1559,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) LOG(RkISP1, Error) << "Cannot queue buffers to dewarper: " << strerror(-ret); - request->metadata().set(controls::ScalerCrop, activeCrop_.value()); + metadataAvailable(request, controls::ScalerCrop, activeCrop_.value()); } void PipelineHandlerRkISP1::dewarpBufferReady(FrameBuffer *buffer) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 05387ca7c..937c781c6 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -909,7 +909,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) } if (request) - request->metadata().set(controls::SensorTimestamp, + // TODO: is this correct? should it be set later? + pipe->metadataAvailable(request, controls::SensorTimestamp, buffer->metadata().timestamp); /* @@ -997,7 +998,7 @@ void SimpleCameraData::metadataReady(uint32_t frame, const ControlList &metadata if (!info) return; - info->request->metadata().merge(metadata); + pipe()->metadataAvailable(info->request, metadata); info->metadataProcessed = true; tryCompleteRequest(info->request); } diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index e0036e3b5..d1bdc1c24 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -883,8 +883,7 @@ void UVCCameraData::imageBufferReady(FrameBuffer *buffer) Request *request = buffer->request(); /* \todo Use the UVC metadata to calculate a more precise timestamp */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); + pipe()->metadataAvailable(request, controls::SensorTimestamp, buffer->metadata().timestamp); pipe()->completeBuffer(request, buffer); pipe()->completeRequest(request); diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index f8a29da41..59445627b 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -617,8 +617,7 @@ void VimcCameraData::imageBufferReady(FrameBuffer *buffer) } /* Record the sensor's timestamp in the request metadata. */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); + pipe->metadataAvailable(request, controls::SensorTimestamp, buffer->metadata().timestamp); pipe->completeBuffer(request, buffer); pipe->completeRequest(request); diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index 049ebcba5..93c1733d6 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -331,7 +331,7 @@ int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, ASSERT(found); } - request->metadata().set(controls::SensorTimestamp, timestamp); + metadataAvailable(request, controls::SensorTimestamp, timestamp); completeRequest(request); return 0; From patchwork Fri Jun 6 16:41:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23499 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A3E11C332E for ; Fri, 6 Jun 2025 16:42:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E19568DD2; Fri, 6 Jun 2025 18:42:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pp3FBgoL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96C1C68DD3 for ; Fri, 6 Jun 2025 18:42:20 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 224156DC for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228136; bh=v4haTZL+iX8qFxPedc94HVWa6Y3xwi6lSmxDSlZEqSQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=pp3FBgoLpfhjhH84HlnN0IIip6dIhlIxbUaOmTyENc70CBLS/iAa23Etk3eIwM1rU /OL5rFqM/yuMTwkteOCUXGgFU9c89O2zLjqo+halTydr+FlEtGchkfNJqu0b1k786m zpYQF9j1roZ0kpWquQJ13VOGolOFYYlsQ0dkJxFY= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 20/23] libcamera: pipeline: rpi: Queue metadata until completion Date: Fri, 6 Jun 2025 18:41:53 +0200 Message-ID: <20250606164156.1442682-21-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The rpi pipeline drops a certain number of initial frames. It uses `ControlList::clear()` to remove the metadata items of a request that was populated while processing a frame that is ultimately dropped. Since the decision to drop a frame only occurs at the very end of the processing, this stop-gap measure is introduced to delay metadata completion until the request is actually completed. Obsoleted by https://patchwork.libcamera.org/cover/23474/. Signed-off-by: Barnabás Pőcze --- .../pipeline/rpi/common/pipeline_base.cpp | 29 +++++++++++-------- .../pipeline/rpi/common/pipeline_base.h | 4 +-- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 8 ++--- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 8 ++--- 4 files changed, 27 insertions(+), 22 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 98507a152..d432cfb51 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -780,7 +780,7 @@ int PipelineHandlerBase::queueRequestDevice(Camera *camera, Request *request) } /* Push the request to the back of the queue. */ - data->requestQueue_.push(request); + data->requestQueue_.push({ request, {} }); data->handleState(); return 0; @@ -1232,8 +1232,8 @@ void CameraData::metadataReady(const ControlList &metadata) /* Add to the Request metadata buffer what the IPA has provided. */ /* Last thing to do is to fill up the request metadata. */ - Request *request = requestQueue_.front(); - request->metadata().merge(metadata); + ControlList &requestMetadata = requestQueue_.front().second; + requestMetadata.merge(metadata); /* * Inform the sensor of the latest colour gains if it has the @@ -1392,7 +1392,7 @@ void CameraData::clearIncompleteRequests() * back to the application. */ while (!requestQueue_.empty()) { - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); for (auto &b : request->buffers()) { FrameBuffer *buffer = b.second; @@ -1406,6 +1406,9 @@ void CameraData::clearIncompleteRequests() } } + // TODO: need this when cancelled? + request->metadata().merge(metadata); + pipe()->completeRequest(request); requestQueue_.pop(); } @@ -1418,7 +1421,7 @@ void CameraData::handleStreamBuffer(FrameBuffer *buffer, RPi::Stream *stream) * that we actually have one to action, otherwise we just return * buffer back to the stream. */ - Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front(); + Request *request = requestQueue_.empty() ? nullptr : requestQueue_.front().first; if (!dropFrameCount_ && request && request->findBuffer(stream) == buffer) { /* * Tag the buffer as completed, returning it to the @@ -1471,7 +1474,7 @@ void CameraData::checkRequestCompleted() * change the state to IDLE when ready. */ if (!dropFrameCount_) { - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); if (request->hasPendingBuffers()) return; @@ -1482,6 +1485,8 @@ void CameraData::checkRequestCompleted() LOG(RPI, Debug) << "Completing request sequence: " << request->sequence(); + request->metadata().merge(metadata); + pipe()->completeRequest(request); requestQueue_.pop(); requestCompleted = true; @@ -1504,10 +1509,10 @@ void CameraData::checkRequestCompleted() } } -void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request *request) +void CameraData::fillRequestMetadata(const ControlList &bufferControls, ControlList &metadata) { - request->metadata().set(controls::SensorTimestamp, - bufferControls.get(controls::SensorTimestamp).value_or(0)); + metadata.set(controls::SensorTimestamp, + bufferControls.get(controls::SensorTimestamp).value_or(0)); if (cropParams_.size()) { std::vector crops; @@ -1515,10 +1520,10 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request for (auto const &[k, v] : cropParams_) crops.push_back(scaleIspCrop(v.ispCrop)); - request->metadata().set(controls::ScalerCrop, crops[0]); + metadata.set(controls::ScalerCrop, crops[0]); if (crops.size() > 1) { - request->metadata().set(controls::rpi::ScalerCrops, - Span(crops.data(), crops.size())); + metadata.set(controls::rpi::ScalerCrops, + Span(crops.data(), crops.size())); } } } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f35..d3a1bd216 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -129,7 +129,7 @@ public: return state_ != State::Stopped && state_ != State::Error; } - std::queue requestQueue_; + std::queue> requestQueue_; /* For handling digital zoom. */ IPACameraSensorInfo sensorInfo_; @@ -179,7 +179,7 @@ public: protected: void fillRequestMetadata(const ControlList &bufferControls, - Request *request); + ControlList &metadata); virtual void tryRunPipeline() = 0; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 91e7f4c94..8436cb4fa 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -2318,7 +2318,7 @@ void PiSPCameraData::tryRunPipeline() CfeJob &job = cfeJobQueue_.front(); /* Take the first request from the queue and action the IPA. */ - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); /* See if a new ScalerCrop value needs to be applied. */ applyScalerCrop(request->controls()); @@ -2328,8 +2328,8 @@ void PiSPCameraData::tryRunPipeline() * related controls. We clear it first because the request metadata * may have been populated if we have dropped the previous frame. */ - request->metadata().clear(); - fillRequestMetadata(job.sensorControls, request); + metadata.clear(); + fillRequestMetadata(job.sensorControls, metadata); /* Set our state to say the pipeline is active. */ state_ = State::Busy; @@ -2347,7 +2347,7 @@ void PiSPCameraData::tryRunPipeline() params.buffers.bayer = RPi::MaskBayerData | bayerId; params.buffers.stats = RPi::MaskStats | statsId; params.buffers.embedded = 0; - params.ipaContext = requestQueue_.front()->sequence(); + params.ipaContext = requestQueue_.front().first->sequence(); params.delayContext = job.delayContext; params.sensorControls = std::move(job.sensorControls); params.requestControls = request->controls(); diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fe910bdf2..a311f43a4 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -836,7 +836,7 @@ void Vc4CameraData::ispOutputDequeue(FrameBuffer *buffer) if (stream == &isp_[Isp::Stats]) { ipa::RPi::ProcessParams params; params.buffers.stats = index | RPi::MaskStats; - params.ipaContext = requestQueue_.front()->sequence(); + params.ipaContext = requestQueue_.front().first->sequence(); ipa_->processStats(params); } else { /* Any other ISP output can be handed back to the application now. */ @@ -935,7 +935,7 @@ void Vc4CameraData::tryRunPipeline() return; /* Take the first request from the queue and action the IPA. */ - Request *request = requestQueue_.front(); + auto &[request, metadata] = requestQueue_.front(); /* See if a new ScalerCrop value needs to be applied. */ applyScalerCrop(request->controls()); @@ -945,8 +945,8 @@ void Vc4CameraData::tryRunPipeline() * related controls. We clear it first because the request metadata * may have been populated if we have dropped the previous frame. */ - request->metadata().clear(); - fillRequestMetadata(bayerFrame.controls, request); + metadata.clear(); + fillRequestMetadata(bayerFrame.controls, metadata); /* Set our state to say the pipeline is active. */ state_ = State::Busy; From patchwork Fri Jun 6 16:41:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23502 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 321A4C3330 for ; Fri, 6 Jun 2025 16:42:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B4B4B68DE3; Fri, 6 Jun 2025 18:42:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UVb/aVDW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D685668DD4 for ; Fri, 6 Jun 2025 18:42:20 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5ED708DB for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228136; bh=GfFwFjJcvkYj36wlL1DD6cCGMcoRu84novp/y0KZMyo=; h=From:To:Subject:Date:In-Reply-To:References:From; b=UVb/aVDWLNZvfnRDVry2FeNrAb+1Y2OG4oiKf26HHoblHiLVJuc/hiyv+nmUcZ1As SCtP5NjP7Dztr2S273tuIaBxwBI0WM3SY5P1Qo15hGWnuRkeV1aWQaL4rkwvNkH10u weItR4RhIK8HL9XQtJdLsmc+fnMp20sFhi+8MAEE= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 21/23] libcamera: pipeline: rpi: Use `metadataAvailable()` Date: Fri, 6 Jun 2025 18:41:54 +0200 Message-ID: <20250606164156.1442682-22-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the newly introduced `metadataAvailable()` function to send metadata items to the application. Obsoleted by https://patchwork.libcamera.org/cover/23474/. Signed-off-by: Barnabás Pőcze --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index d432cfb51..c5dd6fd94 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1407,7 +1407,7 @@ void CameraData::clearIncompleteRequests() } // TODO: need this when cancelled? - request->metadata().merge(metadata); + pipe()->metadataAvailable(request, metadata); pipe()->completeRequest(request); requestQueue_.pop(); @@ -1485,7 +1485,7 @@ void CameraData::checkRequestCompleted() LOG(RPI, Debug) << "Completing request sequence: " << request->sequence(); - request->metadata().merge(metadata); + pipe()->metadataAvailable(request, metadata); pipe()->completeRequest(request); requestQueue_.pop(); From patchwork Fri Jun 6 16:41:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23501 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 066DAC3326 for ; Fri, 6 Jun 2025 16:42:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 824B968DD7; Fri, 6 Jun 2025 18:42:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="max17jzX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 21C7B68DD5 for ; Fri, 6 Jun 2025 18:42:21 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9F0E46DC for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228136; bh=wyBwsGxtmrsWrgi7j2El1Bo9xmYz+7ox7WZqrMf9PW4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=max17jzXICfcxywFkoKJSeNCHsv/Nk3AAd7z8LC6yAPthY+7cR4ImwKXft8vPjz0V +TYLoLeKDZJgbXI5nlw+gvjNY85LKgLhTdnCXrYcHA9u3EjiBjbLcYMIZjzBRSVKRs jnk/vzmzvzLFrIvXIlm3SBwJaRuUv4nsxOYR+D30= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 22/23] py: Use `Request::metadata2()` Date: Fri, 6 Jun 2025 18:41:55 +0200 Message-ID: <20250606164156.1442682-23-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Expose the newly introduced `MetadataList` for any given `Request`. This done similarly to the `Request::metadata()`, using the same `controlValueToPy()` function. However, that function is changed to accept `ControlValueView` as its argument since `MetadataList` does not contain `ControlValue`s directly. Signed-off-by: Barnabás Pőcze --- src/py/libcamera/py_helpers.cpp | 4 ++-- src/py/libcamera/py_helpers.h | 2 +- src/py/libcamera/py_main.cpp | 9 ++++----- 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/src/py/libcamera/py_helpers.cpp b/src/py/libcamera/py_helpers.cpp index 1ad1d4c1a..32f5cdaa6 100644 --- a/src/py/libcamera/py_helpers.cpp +++ b/src/py/libcamera/py_helpers.cpp @@ -16,7 +16,7 @@ namespace py = pybind11; using namespace libcamera; template -static py::object valueOrTuple(const ControlValue &cv) +static py::object valueOrTuple(const ControlValueView &cv) { if (cv.isArray()) { const T *v = reinterpret_cast(cv.data().data()); @@ -31,7 +31,7 @@ static py::object valueOrTuple(const ControlValue &cv) return py::cast(cv.get()); } -py::object controlValueToPy(const ControlValue &cv) +py::object controlValueToPy(const ControlValueView &cv) { switch (cv.type()) { case ControlTypeNone: diff --git a/src/py/libcamera/py_helpers.h b/src/py/libcamera/py_helpers.h index 983969dff..895006d0c 100644 --- a/src/py/libcamera/py_helpers.h +++ b/src/py/libcamera/py_helpers.h @@ -9,5 +9,5 @@ #include -pybind11::object controlValueToPy(const libcamera::ControlValue &cv); +pybind11::object controlValueToPy(const libcamera::ControlValueView &cv); libcamera::ControlValue pyToControlValue(const pybind11::object &ob, libcamera::ControlType type); diff --git a/src/py/libcamera/py_main.cpp b/src/py/libcamera/py_main.cpp index 441a70ab4..126f544e7 100644 --- a/src/py/libcamera/py_main.cpp +++ b/src/py/libcamera/py_main.cpp @@ -460,14 +460,13 @@ PYBIND11_MODULE(_libcamera, m) self.controls().set(id.id(), pyToControlValue(value, id.type())); }) .def_property_readonly("metadata", [](Request &self) { - /* Convert ControlList to std container */ + /* Convert MetadataList to std container */ std::unordered_map ret; - for (const auto &[key, cv] : self.metadata()) { - const ControlId *id = controls::controls.at(key); - py::object ob = controlValueToPy(cv); - ret[id] = ob; + for (const auto &[k, v] : self.metadata2()) { + const ControlId *id = controls::controls.at(k); + ret.try_emplace(id, controlValueToPy(v)); } return ret; From patchwork Fri Jun 6 16:41:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23503 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56C22C3331 for ; Fri, 6 Jun 2025 16:42:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B5DD368DE6; Fri, 6 Jun 2025 18:42:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fzBfkdQ0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C5C768DD6 for ; Fri, 6 Jun 2025 18:42:21 +0200 (CEST) Received: from pb-laptop.local (185.182.215.79.nat.pool.zt.hu [185.182.215.79]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD2A98DB for ; Fri, 6 Jun 2025 18:42:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1749228137; bh=+Lvbk24NSO+srL5gi1SABT81HvDAXG4rQKskNZJQn9Q=; h=From:To:Subject:Date:In-Reply-To:References:From; b=fzBfkdQ0R/MhucPL91czFSIJ1vkibi7ITcM7lsqAI8PZrhWSCYwbv+5BewuzEWS9c iGqJt6rBkmlcBSuh/yrmDloZSMDdInX+1SBIWFMe5n4FE62XL2zxRsoZM1od2Maoin 3O42kFD2Y7gTp1CqdXoeWuDuQaUbE9Q/Gg4ufYNw= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v1 23/23] treewide: Use `Request::metadata2()` Date: Fri, 6 Jun 2025 18:41:56 +0200 Message-ID: <20250606164156.1442682-24-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> References: <20250606164156.1442682-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Apart from a few exceptions, make everything use the new metadata list of a request to access metadata items. After this change the only thing remaining is to remove the exceptional uses, remove `Request::metadata()` altogether, and finally rename `Request::metadata2()`. Signed-off-by: Barnabás Pőcze --- src/android/camera_device.cpp | 4 ++-- src/apps/cam/file_sink.cpp | 4 ++-- src/apps/cam/file_sink.h | 4 ++-- src/apps/common/dng_writer.cpp | 2 +- src/apps/common/dng_writer.h | 4 ++-- src/apps/qcam/main_window.cpp | 4 ++-- src/apps/qcam/main_window.h | 4 ++-- src/gstreamer/gstlibcamera-controls.cpp.in | 4 ++-- src/gstreamer/gstlibcamerasrc.cpp | 2 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 2 +- 10 files changed, 17 insertions(+), 17 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 80ff248c2..b8c6ddf84 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1187,7 +1187,7 @@ void CameraDevice::requestComplete(Request *request) * \todo The shutter event notification should be sent to the framework * as soon as possible, earlier than request completion time. */ - uint64_t sensorTimestamp = static_cast(request->metadata() + uint64_t sensorTimestamp = static_cast(request->metadata2() .get(controls::SensorTimestamp) .value_or(0)); notifyShutter(descriptor->frameNumber_, sensorTimestamp); @@ -1423,7 +1423,7 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream, std::unique_ptr CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) const { - const ControlList &metadata = descriptor.request_->metadata(); + const MetadataList &metadata = descriptor.request_->metadata2(); const CameraMetadata &settings = descriptor.settings_; camera_metadata_ro_entry_t entry; bool found; diff --git a/src/apps/cam/file_sink.cpp b/src/apps/cam/file_sink.cpp index 65794a2f9..1cc1b2b12 100644 --- a/src/apps/cam/file_sink.cpp +++ b/src/apps/cam/file_sink.cpp @@ -96,13 +96,13 @@ void FileSink::mapBuffer(FrameBuffer *buffer) bool FileSink::processRequest(Request *request) { for (auto [stream, buffer] : request->buffers()) - writeBuffer(stream, buffer, request->metadata()); + writeBuffer(stream, buffer, request->metadata2()); return true; } void FileSink::writeBuffer(const Stream *stream, FrameBuffer *buffer, - [[maybe_unused]] const ControlList &metadata) + [[maybe_unused]] const MetadataList &metadata) { std::string filename = pattern_; size_t pos; diff --git a/src/apps/cam/file_sink.h b/src/apps/cam/file_sink.h index 26cd61b36..1f973f2e6 100644 --- a/src/apps/cam/file_sink.h +++ b/src/apps/cam/file_sink.h @@ -11,7 +11,7 @@ #include #include -#include +#include #include #include "frame_sink.h" @@ -44,7 +44,7 @@ private: void writeBuffer(const libcamera::Stream *stream, libcamera::FrameBuffer *buffer, - const libcamera::ControlList &metadata); + const libcamera::MetadataList &metadata); #ifdef HAVE_TIFF const libcamera::Camera *camera_; diff --git a/src/apps/common/dng_writer.cpp b/src/apps/common/dng_writer.cpp index ac4619511..d8884548c 100644 --- a/src/apps/common/dng_writer.cpp +++ b/src/apps/common/dng_writer.cpp @@ -521,7 +521,7 @@ const std::map formatInfo = { int DNGWriter::write(const char *filename, const Camera *camera, const StreamConfiguration &config, - const ControlList &metadata, + const MetadataList &metadata, [[maybe_unused]] const FrameBuffer *buffer, const void *data) { diff --git a/src/apps/common/dng_writer.h b/src/apps/common/dng_writer.h index aaa8a852b..741f78a75 100644 --- a/src/apps/common/dng_writer.h +++ b/src/apps/common/dng_writer.h @@ -10,8 +10,8 @@ #ifdef HAVE_TIFF #include -#include #include +#include #include class DNGWriter @@ -19,7 +19,7 @@ class DNGWriter public: static int write(const char *filename, const libcamera::Camera *camera, const libcamera::StreamConfiguration &config, - const libcamera::ControlList &metadata, + const libcamera::MetadataList &metadata, const libcamera::FrameBuffer *buffer, const void *data); }; diff --git a/src/apps/qcam/main_window.cpp b/src/apps/qcam/main_window.cpp index 7e3f3da60..46dfbb198 100644 --- a/src/apps/qcam/main_window.cpp +++ b/src/apps/qcam/main_window.cpp @@ -643,7 +643,7 @@ void MainWindow::captureRaw() } void MainWindow::processRaw(FrameBuffer *buffer, - [[maybe_unused]] const ControlList &metadata) + [[maybe_unused]] const MetadataList &metadata) { #ifdef HAVE_TIFF QString defaultPath = QStandardPaths::writableLocation(QStandardPaths::PicturesLocation); @@ -707,7 +707,7 @@ void MainWindow::processCapture() processViewfinder(request->buffers().at(vfStream_)); if (request->buffers().count(rawStream_)) - processRaw(request->buffers().at(rawStream_), request->metadata()); + processRaw(request->buffers().at(rawStream_), request->metadata2()); request->reuse(); QMutexLocker locker(&mutex_); diff --git a/src/apps/qcam/main_window.h b/src/apps/qcam/main_window.h index 81fcf915a..eb1e4ce58 100644 --- a/src/apps/qcam/main_window.h +++ b/src/apps/qcam/main_window.h @@ -12,9 +12,9 @@ #include #include -#include #include #include +#include #include #include @@ -66,7 +66,7 @@ private Q_SLOTS: void saveImageAs(); void captureRaw(); void processRaw(libcamera::FrameBuffer *buffer, - const libcamera::ControlList &metadata); + const libcamera::MetadataList &metadata); void renderComplete(libcamera::FrameBuffer *buffer); diff --git a/src/gstreamer/gstlibcamera-controls.cpp.in b/src/gstreamer/gstlibcamera-controls.cpp.in index 89c530da0..f5bd2885c 100644 --- a/src/gstreamer/gstlibcamera-controls.cpp.in +++ b/src/gstreamer/gstlibcamera-controls.cpp.in @@ -322,6 +322,6 @@ void GstCameraControls::applyControls(std::unique_ptr &reque void GstCameraControls::readMetadata(libcamera::Request *request) { - controls_acc_.merge(request->metadata(), - ControlList::MergePolicy::OverwriteExisting); + for (const auto &[k, v] : request->metadata2()) + controls_acc_.set(k, ControlValue(v)); } diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index b34f08977..5b570e86f 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -247,7 +247,7 @@ GstLibcameraSrcState::requestCompleted(Request *request) } if (GST_ELEMENT_CLOCK(src_)) { - int64_t timestamp = request->metadata().get(controls::SensorTimestamp).value_or(0); + int64_t timestamp = request->metadata2().get(controls::SensorTimestamp).value_or(0); GstClockTime gst_base_time = GST_ELEMENT(src_)->base_time; GstClockTime gst_now = gst_clock_get_time(GST_ELEMENT_CLOCK(src_)); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 0f59d0727..5bc8b5ed8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1375,7 +1375,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer) return; } - ipa_->processStats(info->id, request->metadata().get(controls::SensorTimestamp).value_or(0), + ipa_->processStats(info->id, request->metadata2().get(controls::SensorTimestamp).value_or(0), info->statBuffer->cookie(), info->effectiveSensorControls); }