[{"id":36614,"web_url":"https://patchwork.libcamera.org/comment/36614/","msgid":"<176209761170.2973778.3691638889771779739@ping.linuxembedded.co.uk>","date":"2025-11-02T15:33:31","subject":"Re: [RFC PATCH v3 16/22] libcamera: pipeline_handler: Add\n\tmetadataAvailable() function","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Barnabás Pőcze (2025-10-30 16:58:10)\n> From: Jacopo Mondi \n> \n> Currently the way pipeline handlers report metadata for a Request is\n> by accessing the Request::metadata_ list directly, and either merging\n> a ControlList or seting a single control value there.\n> \n> Direct access to Request::metadata_ is, however, problematic as even if\n> metadata would be available earlier, pipeline handlers can only\n> accumulate the results in Request::metadata_ and they're only available\n> for applications at Request complete time.\n> \n> Instead of letting pipeline handlers access Request::metadata_ directly\n> provide two helper functions, similar in spirit to\n> PipelineHandler::completeBuffer() and\n> PipelineHandler::completeRequest(), to allow pipeline handlers to\n> notify early availability of metadata.\n> \n> Provide three overloads, one that accepts a ControlList and merges it into\n> Request::metadata_, one that allows to set a single metadata result there\n> without going through an intermediate copy, and one that runs a callback\n> for conditional metadata reporting without an intermediate ControlList.\n> \n> The newly provided helpers trigger the Camera::availableMetadata signal\n> from where applications can retrieve the list of controls that have\n> just been made available by the pipeline handler.\n> \n> Signed-off-by: Jacopo Mondi \n> [Fill both lists, use `type_identity`, new overload, adjust commit message.]\n> Signed-off-by: Barnabás Pőcze \n> Reviewed-by: Paul Elder \n> ---\n> Original: https://patchwork.libcamera.org/patch/22229/\n> \n> changes in v3:\n> * abort if merging ControlList into metadata fails\n> \n> changes in v2:\n> * add new overload that takes an invocable object for more flexibility\n> ---\n> include/libcamera/internal/pipeline_handler.h | 49 ++++++++++\n> src/libcamera/pipeline_handler.cpp | 90 +++++++++++++++++++\n> 2 files changed, 139 insertions(+)\n> \n> diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h\n> index e89d6a33e3..6c0e546e67 100644\n> --- a/include/libcamera/internal/pipeline_handler.h\n> +++ b/include/libcamera/internal/pipeline_handler.h\n> @@ -7,16 +7,21 @@\n> \n> #pragma once\n> \n> +#include \n> #include \n> #include \n> #include \n> #include \n> \n> +#include \n> #include \n> \n> +#include \n> #include \n> #include \n> \n> +#include \"libcamera/internal/request.h\"\n> +\n> namespace libcamera {\n> \n> class Camera;\n> @@ -58,6 +63,50 @@ public:\n> void registerRequest(Request *request);\n> void queueRequest(Request *request);\n> \n> + void metadataAvailable(Request *request, const ControlList &metadata);\n> +\n> + template\n> + void metadataAvailable(Request *request, const Control &ctrl,\n> + const internal::cxx20::type_identity_t &value)\n> + {\n> + auto &m = request->metadata2();\n> + const auto c = m.checkpoint();\n> +\n> + std::ignore = m.set(ctrl, value);\n> + request->metadata().set(ctrl, value);\n> +\n> + const auto d = c.diffSince();\n> + if (d)\n> + request->_d()->camera()->metadataAvailable.emit(request, d);\n> + }\n> +\n> +#ifndef __DOXYGEN__\n> + struct MetadataSetter {\n> + Request *request;\n> +\n> + template\n> + void operator()(const Control &ctrl,\n> + const internal::cxx20::type_identity_t &value) const\n> + {\n> + request->metadata().set(ctrl, value);\n> + std::ignore = request->metadata2().set(ctrl, value);\n> + }\n> + };\n> +\n> + template> * = nullptr>\n> +#else\n> + template\n> +#endif\n> + void metadataAvailable(Request *request, Func func)\n> + {\n> + const auto c = request->metadata2().checkpoint();\n> +\n> + std::invoke(func, MetadataSetter{ request });\n> +\n> + if (const auto d = c.diffSince())\n> + request->_d()->camera()->metadataAvailable.emit(request, d);\n> + }\n> +\n> bool completeBuffer(Request *request, FrameBuffer *buffer);\n> void completeRequest(Request *request);\n> void cancelRequest(Request *request);\n> diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp\n> index e5f9e55c97..de98a93306 100644\n> --- a/src/libcamera/pipeline_handler.cpp\n> +++ b/src/libcamera/pipeline_handler.cpp\n> @@ -532,6 +532,96 @@ void PipelineHandler::doQueueRequests(Camera *camera)\n> * \\return 0 on success or a negative error code otherwise\n> */\n> \n> +/**\n> + * \\brief Signal the availability of metadata for \\a request\n> + * \\param[in] request The request the metadata belongs to\n> + * \\param[in] metadata The collection of metadata items\n> + *\n> + * This function copies metadata items from \\a metadata to the cumulative metadata\n> + * collection of \\a request. This function may be called multiple times, but metadata\n> + * items already present in Request::metadata() are ignored. Afterwards the function\n\nIgnored? Hrm ... I would expect the behaviour to be to replace? Maybe I\nmisssed a bit somewhere - so if code sets a metadata item multiple times\nits the first that wins, not the last ?\n\n> + * notifies the application by triggering the Camera::availableMetadata signal with\n> + * the just added metadata items.\n> + *\n> + * Early metadata completion allows pipeline handlers to fast track delivery of\n> + * metadata results as soon as they are available before the completion of \\a\n> + * request. The full list of metadata results of a Request is available at\n> + * Request completion time in Request::metadata().\n> + *\n> + * \\context This function shall be called from the CameraManager thread.\n> + *\n> + * \\sa PipelineHandler::metadataAvailable(Request *request, Func func)\n> + * \\sa PipelineHandler::metadataAvailable(Request *request,\n> + * const Control &ctrl,\n> + * const internal::cxx20::type_identity_t &value)\n> + */\n> +void PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> +{\n> + request->metadata().merge(metadata);\n> +\n> + const auto d = request->metadata2().merge(metadata);\n> + if (!d)\n> + LOG(Pipeline, Fatal) << \"Tried to add incompatible metadata items\";\n> +\n> + if (!d->empty())\n> + request->_d()->camera()->metadataAvailable.emit(request, *d);\n> +}\n> +\n> +/**\n> + * \\fn void PipelineHandler::metadataAvailable(Request *request, const Control &ctrl,\n> + * const internal::cxx20::type_identity_t &value)\n> + * \\copybrief PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> + * \\param[in] request The request the metadata belongs to\n> + * \\param[in] ctrl The control id of the metadata item\n> + * \\param[in] value The value of the metadata item\n> + *\n> + * This function servers the same purpose as\n\ns/servers/serves/\n\n> + * PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> + * but it allows a single metadata item to be reported directly,\n> + * without creating a ControlList.\n> + *\n> + * \\context This function shall be called from the CameraManager thread.\n> + * \\sa PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> + */\n> +\n> +/**\n> + * \\fn void PipelineHandler::metadataAvailable(Request *request, Func func)\n> + * \\copybrief PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> + * \\param[in] request The request the metadata belongs to\n> + * \\param[in] func The callback to invoke\n> + *\n> + * This function serves the same purpose as\n> + * PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> + * but it provides more flexibility for pipeline handlers. This function invokes\n> + * \\a func, which receives as its sole argument an object of unspecified type whose\n> + * operator() can be used to add metadata items.\n> + *\n> + * For example, a PipelineHandler might use this function to conditionally report two\n> + * metadata items without creating an intermediate ControlList:\n> + *\n> + * \\code\n> + metadataAvailable(request, [&](auto set) {\n> + if (...) // controls::X is available and ready to be reported\n> + set(controls::X, ...);\n> + if (...) // controls::Y is available and ready to be reported\n> + set(controls::Y, ...);\n> + // ...\n> + });\n> + * \\endcode\n> + *\n> + * The advantage of the above over two calls to\n> + * PipelineHandler::metadataAvailable(Request *request, const Control &ctrl, const internal::cxx20::type_identity_t &value)\n> + * is that the application is only notified once, after \\a func has returned.\n> + *\n> + * \\note Calling any overload of metadataAvailable() inside \\a func is not allowed.\n> + * \\note The object passed to \\a func is only usable while \\a func runs, it must not\n> + * be saved or reused.\n\nOh nice - so we can reduce signals and make sure we group metadata\nsetting in the ipa algorithms.\n\n\nI think the handling of precedence of setting a metadata item that is\nalready set is outside of this patch so aside from the small comments\nabove:\n\nReviewed-by: Kieran Bingham \n\n> + *\n> + * \\context This function shall be called from the CameraManager thread.\n> + *\n> + * \\sa PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n> + */\n> +\n> /**\n> * \\brief Complete a buffer for a request\n> * \\param[in] request The request the buffer belongs to\n> -- \n> 2.51.1\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BAE0ABDE4C\n\tfor ;\n\tSun, 2 Nov 2025 15:33:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EC9AC6096B;\n\tSun, 2 Nov 2025 16:33:35 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8ABAF606E6\n\tfor ;\n\tSun, 2 Nov 2025 16:33:34 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4DD6B10D4;\n\tSun, 2 Nov 2025 16:31:42 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"RMCNH+C/\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1762097502;\n\tbh=nMtxz19jAfn7XpWFvlyoYwrLWeJs9WlaKT+bxKHCM3E=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=RMCNH+C/SdR4bEv+bCK4o4+5IWtad5CTtfV3yVrezokoDRN3gu2Mjc2mlgxkhc++B\n\tdLdljwphg1H9W15xzhnneNiLNiwSsZI1RGZdlfeO9oklXdDlqRsW0Gr47VaZ1nxvOp\n\tX8af3nPWf3CmRxw8W9SDNlVO3c/pe+E8UHbRQu5g=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20251030165816.1095180-17-barnabas.pocze@ideasonboard.com>","References":"<20251030165816.1095180-1-barnabas.pocze@ideasonboard.com>\n\t<20251030165816.1095180-17-barnabas.pocze@ideasonboard.com>","Subject":"Re: [RFC PATCH v3 16/22] libcamera: pipeline_handler: Add\n\tmetadataAvailable() function","From":"Kieran Bingham ","Cc":"Jacopo Mondi ,\n\tPaul Elder ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Sun, 02 Nov 2025 15:33:31 +0000","Message-ID":"<176209761170.2973778.3691638889771779739@ping.linuxembedded.co.uk>","User-Agent":"alot/0.9.1","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":36637,"web_url":"https://patchwork.libcamera.org/comment/36637/","msgid":"","date":"2025-11-03T09:39:42","subject":"Re: [RFC PATCH v3 16/22] libcamera: pipeline_handler: Add\n\tmetadataAvailable() function","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2025. 11. 02. 16:33 keltezéssel, Kieran Bingham írta:\n> Quoting Barnabás Pőcze (2025-10-30 16:58:10)\n>> From: Jacopo Mondi \n>>\n>> Currently the way pipeline handlers report metadata for a Request is\n>> by accessing the Request::metadata_ list directly, and either merging\n>> a ControlList or seting a single control value there.\n>>\n>> Direct access to Request::metadata_ is, however, problematic as even if\n>> metadata would be available earlier, pipeline handlers can only\n>> accumulate the results in Request::metadata_ and they're only available\n>> for applications at Request complete time.\n>>\n>> Instead of letting pipeline handlers access Request::metadata_ directly\n>> provide two helper functions, similar in spirit to\n>> PipelineHandler::completeBuffer() and\n>> PipelineHandler::completeRequest(), to allow pipeline handlers to\n>> notify early availability of metadata.\n>>\n>> Provide three overloads, one that accepts a ControlList and merges it into\n>> Request::metadata_, one that allows to set a single metadata result there\n>> without going through an intermediate copy, and one that runs a callback\n>> for conditional metadata reporting without an intermediate ControlList.\n>>\n>> The newly provided helpers trigger the Camera::availableMetadata signal\n>> from where applications can retrieve the list of controls that have\n>> just been made available by the pipeline handler.\n>>\n>> Signed-off-by: Jacopo Mondi \n>> [Fill both lists, use `type_identity`, new overload, adjust commit message.]\n>> Signed-off-by: Barnabás Pőcze \n>> Reviewed-by: Paul Elder \n>> ---\n>> Original: https://patchwork.libcamera.org/patch/22229/\n>>\n>> changes in v3:\n>> * abort if merging ControlList into metadata fails\n>>\n>> changes in v2:\n>> * add new overload that takes an invocable object for more flexibility\n>> ---\n>> include/libcamera/internal/pipeline_handler.h | 49 ++++++++++\n>> src/libcamera/pipeline_handler.cpp | 90 +++++++++++++++++++\n>> 2 files changed, 139 insertions(+)\n>>\n>> [...]\n>> diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp\n>> index e5f9e55c97..de98a93306 100644\n>> --- a/src/libcamera/pipeline_handler.cpp\n>> +++ b/src/libcamera/pipeline_handler.cpp\n>> @@ -532,6 +532,96 @@ void PipelineHandler::doQueueRequests(Camera *camera)\n>> * \\return 0 on success or a negative error code otherwise\n>> */\n>> \n>> +/**\n>> + * \\brief Signal the availability of metadata for \\a request\n>> + * \\param[in] request The request the metadata belongs to\n>> + * \\param[in] metadata The collection of metadata items\n>> + *\n>> + * This function copies metadata items from \\a metadata to the cumulative metadata\n>> + * collection of \\a request. This function may be called multiple times, but metadata\n>> + * items already present in Request::metadata() are ignored. Afterwards the function\n> \n> Ignored? Hrm ... I would expect the behaviour to be to replace? Maybe I\n> misssed a bit somewhere - so if code sets a metadata item multiple times\n> its the first that wins, not the last ?\n\nThe same item cannot be set multiple times, this is a limitation of this design.\nOtherwise thread-safety cannot be guaranteed. But the description is incorrect,\nexecution will run into the \"Fatal\" log message and not ignore things.\n\n\n> \n>> + * notifies the application by triggering the Camera::availableMetadata signal with\n>> + * the just added metadata items.\n>> + *\n>> + * Early metadata completion allows pipeline handlers to fast track delivery of\n>> + * metadata results as soon as they are available before the completion of \\a\n>> + * request. The full list of metadata results of a Request is available at\n>> + * Request completion time in Request::metadata().\n>> + *\n>> + * \\context This function shall be called from the CameraManager thread.\n>> + *\n>> + * \\sa PipelineHandler::metadataAvailable(Request *request, Func func)\n>> + * \\sa PipelineHandler::metadataAvailable(Request *request,\n>> + * const Control &ctrl,\n>> + * const internal::cxx20::type_identity_t &value)\n>> + */\n>> +void PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)\n>> +{\n>> + request->metadata().merge(metadata);\n>> +\n>> + const auto d = request->metadata2().merge(metadata);\n>> + if (!d)\n>> + LOG(Pipeline, Fatal) << \"Tried to add incompatible metadata items\";\n>> +\n>> + if (!d->empty())\n>> + request->_d()->camera()->metadataAvailable.emit(request, *d);\n>> +}\n>> [...]","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 48209BDE4C\n\tfor ;\n\tMon, 3 Nov 2025 09:39:48 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 60D2260A80;\n\tMon, 3 Nov 2025 10:39:47 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1AAB160805\n\tfor ;\n\tMon, 3 Nov 2025 10:39:46 +0100 (CET)","from [192.168.33.39] (185.221.140.239.nat.pool.zt.hu\n\t[185.221.140.239])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 18D4E99F;\n\tMon, 3 Nov 2025 10:37:53 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"NjihA3AG\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1762162673;\n\tbh=PnRfutGno7QR6wFWzlz3cQeP0m/SKULPXeYGpzcws+0=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=NjihA3AGBT4sIj9THdvNd6cjmJaoVGfmEBP3ynN8pK4HyFJ2nmpt81Ld4oe6m5esp\n\tKCAbr0lxtjLU3Sk2b9TEz87yochHCMt/Zpw7Ol8PWVuz0dw5P6rXdtaKl9aDRraBMR\n\td4E+kYJDVl7kXGhcc79qSZ+t6qudfJPZr5PiTy3U=","Message-ID":"","Date":"Mon, 3 Nov 2025 10:39:42 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [RFC PATCH v3 16/22] libcamera: pipeline_handler: Add\n\tmetadataAvailable() function","To":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Jacopo Mondi ,\n\tPaul Elder ","References":"<20251030165816.1095180-1-barnabas.pocze@ideasonboard.com>\n\t<20251030165816.1095180-17-barnabas.pocze@ideasonboard.com>\n\t<176209761170.2973778.3691638889771779739@ping.linuxembedded.co.uk>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<176209761170.2973778.3691638889771779739@ping.linuxembedded.co.uk>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]