[RFC,v3,16/22] libcamera: pipeline_handler: Add metadataAvailable() function

24918 diff mbox series

From: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Currently the way pipeline handlers report metadata for a Request is
by accessing the Request::metadata_ list directly, and either merging
a ControlList or seting a single control value there.

Direct access to Request::metadata_ is, however, problematic as even if
metadata would be available earlier, pipeline handlers can only
accumulate the results in Request::metadata_ and they're only available
for applications at Request complete time.

Instead of letting pipeline handlers access Request::metadata_ directly
provide two helper functions, similar in spirit to
PipelineHandler::completeBuffer() and
PipelineHandler::completeRequest(), to allow pipeline handlers to
notify early availability of metadata.

Provide three overloads, one that accepts a ControlList and merges it into
Request::metadata_, one that allows to set a single metadata result there
without going through an intermediate copy, and one that runs a callback
for conditional metadata reporting without an intermediate ControlList.

The newly provided helpers trigger the Camera::availableMetadata signal
from where applications can retrieve the list of controls that have
just been made available by the pipeline handler.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
[Fill both lists, use `type_identity`, new overload, adjust commit message.]
Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
---
Original: https://patchwork.libcamera.org/patch/22229/

changes in v3:
  * abort if merging ControlList into metadata fails

changes in v2:
  * add new overload that takes an invocable object for more flexibility
---
 include/libcamera/internal/pipeline_handler.h | 49 ++++++++++
 src/libcamera/pipeline_handler.cpp            | 90 +++++++++++++++++++
 2 files changed, 139 insertions(+)

Quoting Barnabás Pőcze (2025-10-30 16:58:10)
> From: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Currently the way pipeline handlers report metadata for a Request is
> by accessing the Request::metadata_ list directly, and either merging
> a ControlList or seting a single control value there.
> 
> Direct access to Request::metadata_ is, however, problematic as even if
> metadata would be available earlier, pipeline handlers can only
> accumulate the results in Request::metadata_ and they're only available
> for applications at Request complete time.
> 
> Instead of letting pipeline handlers access Request::metadata_ directly
> provide two helper functions, similar in spirit to
> PipelineHandler::completeBuffer() and
> PipelineHandler::completeRequest(), to allow pipeline handlers to
> notify early availability of metadata.
> 
> Provide three overloads, one that accepts a ControlList and merges it into
> Request::metadata_, one that allows to set a single metadata result there
> without going through an intermediate copy, and one that runs a callback
> for conditional metadata reporting without an intermediate ControlList.
> 
> The newly provided helpers trigger the Camera::availableMetadata signal
> from where applications can retrieve the list of controls that have
> just been made available by the pipeline handler.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> [Fill both lists, use `type_identity`, new overload, adjust commit message.]
> Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
> ---
> Original: https://patchwork.libcamera.org/patch/22229/
> 
> changes in v3:
>   * abort if merging ControlList into metadata fails
> 
> changes in v2:
>   * add new overload that takes an invocable object for more flexibility
> ---
>  include/libcamera/internal/pipeline_handler.h | 49 ++++++++++
>  src/libcamera/pipeline_handler.cpp            | 90 +++++++++++++++++++
>  2 files changed, 139 insertions(+)
> 
> diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h
> index e89d6a33e3..6c0e546e67 100644
> --- a/include/libcamera/internal/pipeline_handler.h
> +++ b/include/libcamera/internal/pipeline_handler.h
> @@ -7,16 +7,21 @@
>  
>  #pragma once
>  
> +#include <functional>
>  #include <memory>
>  #include <string>
>  #include <sys/types.h>
>  #include <vector>
>  
> +#include <libcamera/base/internal/cxx20.h>
>  #include <libcamera/base/object.h>
>  
> +#include <libcamera/camera.h>
>  #include <libcamera/controls.h>
>  #include <libcamera/stream.h>
>  
> +#include "libcamera/internal/request.h"
> +
>  namespace libcamera {
>  
>  class Camera;
> @@ -58,6 +63,50 @@ public:
>         void registerRequest(Request *request);
>         void queueRequest(Request *request);
>  
> +       void metadataAvailable(Request *request, const ControlList &metadata);
> +
> +       template<typename T>
> +       void metadataAvailable(Request *request, const Control<T> &ctrl,
> +                              const internal::cxx20::type_identity_t<T> &value)
> +       {
> +               auto &m = request->metadata2();
> +               const auto c = m.checkpoint();
> +
> +               std::ignore = m.set(ctrl, value);
> +               request->metadata().set(ctrl, value);
> +
> +               const auto d = c.diffSince();
> +               if (d)
> +                       request->_d()->camera()->metadataAvailable.emit(request, d);
> +       }
> +
> +#ifndef __DOXYGEN__
> +       struct MetadataSetter {
> +               Request *request;
> +
> +               template<typename T>
> +               void operator()(const Control<T> &ctrl,
> +                               const internal::cxx20::type_identity_t<T> &value) const
> +               {
> +                       request->metadata().set(ctrl, value);
> +                       std::ignore = request->metadata2().set(ctrl, value);
> +               }
> +       };
> +
> +       template<typename Func, std::enable_if_t<std::is_invocable_v<Func&, MetadataSetter>> * = nullptr>
> +#else
> +       template<typename Func>
> +#endif
> +       void metadataAvailable(Request *request, Func func)
> +       {
> +               const auto c = request->metadata2().checkpoint();
> +
> +               std::invoke(func, MetadataSetter{ request });
> +
> +               if (const auto d = c.diffSince())
> +                       request->_d()->camera()->metadataAvailable.emit(request, d);
> +       }
> +
>         bool completeBuffer(Request *request, FrameBuffer *buffer);
>         void completeRequest(Request *request);
>         void cancelRequest(Request *request);
> diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp
> index e5f9e55c97..de98a93306 100644
> --- a/src/libcamera/pipeline_handler.cpp
> +++ b/src/libcamera/pipeline_handler.cpp
> @@ -532,6 +532,96 @@ void PipelineHandler::doQueueRequests(Camera *camera)
>   * \return 0 on success or a negative error code otherwise
>   */
>  
> +/**
> + * \brief Signal the availability of metadata for \a request
> + * \param[in] request The request the metadata belongs to
> + * \param[in] metadata The collection of metadata items
> + *
> + * This function copies metadata items from \a metadata to the cumulative metadata
> + * collection of \a request. This function may be called multiple times, but metadata
> + * items already present in Request::metadata() are ignored. Afterwards the function

Ignored? Hrm ... I would expect the behaviour to be to replace? Maybe I
misssed a bit somewhere - so if code sets a metadata item multiple times
its the first that wins, not the last ?

> + * notifies the application by triggering the Camera::availableMetadata signal with
> + * the just added metadata items.
> + *
> + * Early metadata completion allows pipeline handlers to fast track delivery of
> + * metadata results as soon as they are available before the completion of \a
> + * request. The full list of metadata results of a Request is available at
> + * Request completion time in Request::metadata().
> + *
> + * \context This function shall be called from the CameraManager thread.
> + *
> + * \sa PipelineHandler::metadataAvailable(Request *request, Func func)
> + * \sa PipelineHandler::metadataAvailable(Request *request,
> + *                                        const Control<T> &ctrl,
> + *                                        const internal::cxx20::type_identity_t<T> &value)
> + */
> +void PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> +{
> +       request->metadata().merge(metadata);
> +
> +       const auto d = request->metadata2().merge(metadata);
> +       if (!d)
> +               LOG(Pipeline, Fatal) << "Tried to add incompatible metadata items";
> +
> +       if (!d->empty())
> +               request->_d()->camera()->metadataAvailable.emit(request, *d);
> +}
> +
> +/**
> + * \fn void PipelineHandler::metadataAvailable(Request *request, const Control<T> &ctrl,
> + *                                            const internal::cxx20::type_identity_t<T> &value)
> + * \copybrief PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> + * \param[in] request The request the metadata belongs to
> + * \param[in] ctrl The control id of the metadata item
> + * \param[in] value The value of the metadata item
> + *
> + * This function servers the same purpose as

s/servers/serves/

> + * PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> + * but it allows a single metadata item to be reported directly,
> + * without creating a ControlList.
> + *
> + * \context This function shall be called from the CameraManager thread.
> + * \sa PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> + */
> +
> +/**
> + * \fn void PipelineHandler::metadataAvailable(Request *request, Func func)
> + * \copybrief PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> + * \param[in] request The request the metadata belongs to
> + * \param[in] func The callback to invoke
> + *
> + * This function serves the same purpose as
> + * PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> + * but it provides more flexibility for pipeline handlers. This function invokes
> + * \a func, which receives as its sole argument an object of unspecified type whose
> + * operator() can be used to add metadata items.
> + *
> + * For example, a PipelineHandler might use this function to conditionally report two
> + * metadata items without creating an intermediate ControlList:
> + *
> + * \code
> +       metadataAvailable(request, [&](auto set) {
> +               if (...) // controls::X is available and ready to be reported
> +                       set(controls::X, ...);
> +               if (...) // controls::Y is available and ready to be reported
> +                       set(controls::Y, ...);
> +               // ...
> +       });
> + * \endcode
> + *
> + * The advantage of the above over two calls to
> + * PipelineHandler::metadataAvailable(Request *request, const Control<T> &ctrl, const internal::cxx20::type_identity_t<T> &value)
> + * is that the application is only notified once, after \a func has returned.
> + *
> + * \note Calling any overload of metadataAvailable() inside \a func is not allowed.
> + * \note The object passed to \a func is only usable while \a func runs, it must not
> + *       be saved or reused.

Oh nice - so we can reduce signals and make sure we group metadata
setting in the ipa algorithms.


I think the handling of precedence of setting a metadata item that is
already set is outside of this patch so aside from the small comments
above:

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

> + *
> + * \context This function shall be called from the CameraManager thread.
> + *
> + * \sa PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
> + */
> +
>  /**
>   * \brief Complete a buffer for a request
>   * \param[in] request The request the buffer belongs to
> -- 
> 2.51.1
>

Hi

2025. 11. 02. 16:33 keltezéssel, Kieran Bingham írta:
> Quoting Barnabás Pőcze (2025-10-30 16:58:10)
>> From: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>>
>> Currently the way pipeline handlers report metadata for a Request is
>> by accessing the Request::metadata_ list directly, and either merging
>> a ControlList or seting a single control value there.
>>
>> Direct access to Request::metadata_ is, however, problematic as even if
>> metadata would be available earlier, pipeline handlers can only
>> accumulate the results in Request::metadata_ and they're only available
>> for applications at Request complete time.
>>
>> Instead of letting pipeline handlers access Request::metadata_ directly
>> provide two helper functions, similar in spirit to
>> PipelineHandler::completeBuffer() and
>> PipelineHandler::completeRequest(), to allow pipeline handlers to
>> notify early availability of metadata.
>>
>> Provide three overloads, one that accepts a ControlList and merges it into
>> Request::metadata_, one that allows to set a single metadata result there
>> without going through an intermediate copy, and one that runs a callback
>> for conditional metadata reporting without an intermediate ControlList.
>>
>> The newly provided helpers trigger the Camera::availableMetadata signal
>> from where applications can retrieve the list of controls that have
>> just been made available by the pipeline handler.
>>
>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>> [Fill both lists, use `type_identity`, new overload, adjust commit message.]
>> Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
>> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
>> ---
>> Original: https://patchwork.libcamera.org/patch/22229/
>>
>> changes in v3:
>>    * abort if merging ControlList into metadata fails
>>
>> changes in v2:
>>    * add new overload that takes an invocable object for more flexibility
>> ---
>>   include/libcamera/internal/pipeline_handler.h | 49 ++++++++++
>>   src/libcamera/pipeline_handler.cpp            | 90 +++++++++++++++++++
>>   2 files changed, 139 insertions(+)
>>
>> [...]
>> diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp
>> index e5f9e55c97..de98a93306 100644
>> --- a/src/libcamera/pipeline_handler.cpp
>> +++ b/src/libcamera/pipeline_handler.cpp
>> @@ -532,6 +532,96 @@ void PipelineHandler::doQueueRequests(Camera *camera)
>>    * \return 0 on success or a negative error code otherwise
>>    */
>>   
>> +/**
>> + * \brief Signal the availability of metadata for \a request
>> + * \param[in] request The request the metadata belongs to
>> + * \param[in] metadata The collection of metadata items
>> + *
>> + * This function copies metadata items from \a metadata to the cumulative metadata
>> + * collection of \a request. This function may be called multiple times, but metadata
>> + * items already present in Request::metadata() are ignored. Afterwards the function
> 
> Ignored? Hrm ... I would expect the behaviour to be to replace? Maybe I
> misssed a bit somewhere - so if code sets a metadata item multiple times
> its the first that wins, not the last ?

The same item cannot be set multiple times, this is a limitation of this design.
Otherwise thread-safety cannot be guaranteed. But the description is incorrect,
execution will run into the "Fatal" log message and not ignore things.


> 
>> + * notifies the application by triggering the Camera::availableMetadata signal with
>> + * the just added metadata items.
>> + *
>> + * Early metadata completion allows pipeline handlers to fast track delivery of
>> + * metadata results as soon as they are available before the completion of \a
>> + * request. The full list of metadata results of a Request is available at
>> + * Request completion time in Request::metadata().
>> + *
>> + * \context This function shall be called from the CameraManager thread.
>> + *
>> + * \sa PipelineHandler::metadataAvailable(Request *request, Func func)
>> + * \sa PipelineHandler::metadataAvailable(Request *request,
>> + *                                        const Control<T> &ctrl,
>> + *                                        const internal::cxx20::type_identity_t<T> &value)
>> + */
>> +void PipelineHandler::metadataAvailable(Request *request, const ControlList &metadata)
>> +{
>> +       request->metadata().merge(metadata);
>> +
>> +       const auto d = request->metadata2().merge(metadata);
>> +       if (!d)
>> +               LOG(Pipeline, Fatal) << "Tried to add incompatible metadata items";
>> +
>> +       if (!d->empty())
>> +               request->_d()->camera()->metadataAvailable.emit(request, *d);
>> +}
>> [...]