[libcamera-devel,1/2] android: Document the structures and functions for post-processing

15240 diff mbox series

Specifically document:
 - CameraDevice::sendCaptureResults()
 - CameraDevice::completeDescriptor()
 - CameraDevice::streamProcessingComplete()

 - CameraStream::PostProcessorWorker class
 - Camera3RequestDescriptor::StreamBuffer structure

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 src/android/camera_device.cpp  | 37 +++++++++++++++++++++++++++++++++
 src/android/camera_request.cpp | 38 ++++++++++++++++++++++++++++++++++
 src/android/camera_stream.cpp  | 12 +++++++++++
 3 files changed, 87 insertions(+)

Quoting Umang Jain (2022-01-04 06:52:00)
> Specifically document:
>  - CameraDevice::sendCaptureResults()
>  - CameraDevice::completeDescriptor()
>  - CameraDevice::streamProcessingComplete()
> 
>  - CameraStream::PostProcessorWorker class
>  - Camera3RequestDescriptor::StreamBuffer structure
> 
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  src/android/camera_device.cpp  | 37 +++++++++++++++++++++++++++++++++
>  src/android/camera_request.cpp | 38 ++++++++++++++++++++++++++++++++++
>  src/android/camera_stream.cpp  | 12 +++++++++++
>  3 files changed, 87 insertions(+)
> 
> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp
> index 83825736..00d48471 100644
> --- a/src/android/camera_device.cpp
> +++ b/src/android/camera_device.cpp
> @@ -1164,6 +1164,17 @@ void CameraDevice::requestComplete(Request *request)
>         }
>  }
>  
> +/**
> + * \brief Complete the Camera3RequestDescriptor
> + * \param[in] descriptor The Camera3RequestDescriptor that has completed
> + *
> + * The function marks the Camera3RequestDescriptor as 'complete'. It shall be
> + * called when all the streams in the Camera3RequestDescriptor have completed
> + * capture (or have been generated via post-processing) and the request is ready
> + * to be sent back to the framework.
> + *
> + * \context This function is \threadsafe.
> + */
>  void CameraDevice::completeDescriptor(Camera3RequestDescriptor *descriptor)
>  {
>         MutexLocker lock(descriptorsMutex_);
> @@ -1172,6 +1183,19 @@ void CameraDevice::completeDescriptor(Camera3RequestDescriptor *descriptor)
>         sendCaptureResults();
>  }
>  
> +/**
> + * \brief Sequentially send capture results to the framework
> + *
> + * Iterate over the descriptors queue to send completed descriptors back to the
> + * framework, in the same order as they have been queued. For each complete
> + * descriptor, populate a locally-scoped camera3_capture_result_t from the
> + * descriptor, send the capture result back by calling the
> + * process_capture_result() callback, and remove the descriptor from the queue.
> + * Stop iterating if the descriptor at the front of the queue is not complete.
> + *
> + * This function should never be called directly in the codebase. Use
> + * completeDescriptor() instead.
> + */
>  void CameraDevice::sendCaptureResults()
>  {
>         while (!descriptors_.empty() && !descriptors_.front()->isPending()) {
> @@ -1231,6 +1255,19 @@ void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &strea
>         }
>  }
>  
> +/**
> + * \brief Handle post-processing completion of a stream in a capture request
> + * \param[in] streamBuffer The StreamBuffer for which processing is complete
> + * \param[in] status Stream post-processing status
> + *
> + * This function is called from the post-processor's thread whenever a camera
> + * stream has finished post processing. The corresponding entry is dropped from
> + * the descriptor's pendingStreamsToProcess_ map.
> + *
> + * If the pendingStreamsToProcess_ map is then empty, all streams requiring to
> + * be generated from post-processing have been completed. Mark the descriptor as
> + * complete using completeDescriptor() in that case.
> + */
>  void CameraDevice::streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
>                                             Camera3RequestDescriptor::Status status)
>  {
> diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp
> index 91776907..027b46d3 100644
> --- a/src/android/camera_request.cpp
> +++ b/src/android/camera_request.cpp
> @@ -52,6 +52,44 @@ Camera3RequestDescriptor::Camera3RequestDescriptor(
>  
>  Camera3RequestDescriptor::~Camera3RequestDescriptor() = default;
>  
> +/**
> + * \struct Camera3RequestDescriptor::StreamBuffer
> + * \brief Group information for per-stream buffer of Camera3RequestDescriptor
> + *
> + * A capture request placed to the libcamera HAL can contain multiple streams.
> + * Each stream will have an associated buffer to be filled. StreamBuffer
> + * tracks this buffer with contextual information which aids in the stream's
> + * generation. The generation of the stream will depend on its type (refer to
> + * the CameraStream::Type documentation).
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::stream
> + * \brief Pointer to the corresponding CameraStream
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::camera3Buffer
> + * \brief Native handle to the buffer
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::frameBuffer
> + * \brief Encapsulate the dmabuf handle inside a libcamera::FrameBuffer for
> + * direct streams
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::fence
> + * \brief Acquire fence of the buffer
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::status
> + * \brief Track the status of the buffer
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::internalBuffer
> + * \brief Pointer to a buffer internally handled by CameraStream (if any)
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::srcBuffer
> + * \brief Pointer to the source frame buffer used for post-processing
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::dstBuffer
> + * \brief Pointer to the destination frame buffer used for post-processing
> + *
> + * \var Camera3RequestDescriptor::StreamBuffer::request
> + * \brief Back pointer to Camera3RequestDescriptor to which StreamBuffer belongs

I'd put some extra 'the' in here, but not essential:

'to the Camera3RequestDescriptor'
'which the StreamBuffer'


Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

> + */
>  Camera3RequestDescriptor::StreamBuffer::StreamBuffer(
>         CameraStream *cameraStream, const camera3_stream_buffer_t &buffer,
>         Camera3RequestDescriptor *requestDescriptor)
> diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp
> index c2157450..94f1884c 100644
> --- a/src/android/camera_stream.cpp
> +++ b/src/android/camera_stream.cpp
> @@ -244,6 +244,18 @@ void CameraStream::putBuffer(FrameBuffer *buffer)
>         buffers_.push_back(buffer);
>  }
>  
> +/**
> + * \class CameraStream::PostProcessorWorker
> + * \brief Post-process a CameraStream in an internal thread
> + *
> + * If the association between CameraStream and camera3_stream_t dictated by
> + * CameraStream::Type is internal or mapped, the stream is generated by post
> + * processing of a libcamera stream. Such a request is queued to a
> + * PostProcessorWorker in CameraStream::process(). A queue of post-processing
> + * requests is maintained by the PostProcessorWorker and it will run the
> + * post-processing on an internal thread as soon as any request is available on
> + * its queue.
> + */
>  CameraStream::PostProcessorWorker::PostProcessorWorker(PostProcessor *postProcessor)
>         : postProcessor_(postProcessor)
>  {
> -- 
> 2.31.1
>

Hi Kieran,

On 3/1/22 14:57, Kieran Bingham wrote:
> Quoting Umang Jain (2022-01-04 06:52:00)
>> Specifically document:
>>   - CameraDevice::sendCaptureResults()
>>   - CameraDevice::completeDescriptor()
>>   - CameraDevice::streamProcessingComplete()
>>
>>   - CameraStream::PostProcessorWorker class
>>   - Camera3RequestDescriptor::StreamBuffer structure
>>
>> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> ---
>>   src/android/camera_device.cpp  | 37 +++++++++++++++++++++++++++++++++
>>   src/android/camera_request.cpp | 38 ++++++++++++++++++++++++++++++++++
>>   src/android/camera_stream.cpp  | 12 +++++++++++
>>   3 files changed, 87 insertions(+)
>>
>> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp
>> index 83825736..00d48471 100644
>> --- a/src/android/camera_device.cpp
>> +++ b/src/android/camera_device.cpp
>> @@ -1164,6 +1164,17 @@ void CameraDevice::requestComplete(Request *request)
>>          }
>>   }
>>   
>> +/**
>> + * \brief Complete the Camera3RequestDescriptor
>> + * \param[in] descriptor The Camera3RequestDescriptor that has completed
>> + *
>> + * The function marks the Camera3RequestDescriptor as 'complete'. It shall be
>> + * called when all the streams in the Camera3RequestDescriptor have completed
>> + * capture (or have been generated via post-processing) and the request is ready
>> + * to be sent back to the framework.
>> + *
>> + * \context This function is \threadsafe.
>> + */
>>   void CameraDevice::completeDescriptor(Camera3RequestDescriptor *descriptor)
>>   {
>>          MutexLocker lock(descriptorsMutex_);
>> @@ -1172,6 +1183,19 @@ void CameraDevice::completeDescriptor(Camera3RequestDescriptor *descriptor)
>>          sendCaptureResults();
>>   }
>>   
>> +/**
>> + * \brief Sequentially send capture results to the framework
>> + *
>> + * Iterate over the descriptors queue to send completed descriptors back to the
>> + * framework, in the same order as they have been queued. For each complete
>> + * descriptor, populate a locally-scoped camera3_capture_result_t from the
>> + * descriptor, send the capture result back by calling the
>> + * process_capture_result() callback, and remove the descriptor from the queue.
>> + * Stop iterating if the descriptor at the front of the queue is not complete.
>> + *
>> + * This function should never be called directly in the codebase. Use
>> + * completeDescriptor() instead.
>> + */
>>   void CameraDevice::sendCaptureResults()
>>   {
>>          while (!descriptors_.empty() && !descriptors_.front()->isPending()) {
>> @@ -1231,6 +1255,19 @@ void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &strea
>>          }
>>   }
>>   
>> +/**
>> + * \brief Handle post-processing completion of a stream in a capture request
>> + * \param[in] streamBuffer The StreamBuffer for which processing is complete
>> + * \param[in] status Stream post-processing status
>> + *
>> + * This function is called from the post-processor's thread whenever a camera
>> + * stream has finished post processing. The corresponding entry is dropped from
>> + * the descriptor's pendingStreamsToProcess_ map.
>> + *
>> + * If the pendingStreamsToProcess_ map is then empty, all streams requiring to
>> + * be generated from post-processing have been completed. Mark the descriptor as
>> + * complete using completeDescriptor() in that case.
>> + */
>>   void CameraDevice::streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *streamBuffer,
>>                                              Camera3RequestDescriptor::Status status)
>>   {
>> diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp
>> index 91776907..027b46d3 100644
>> --- a/src/android/camera_request.cpp
>> +++ b/src/android/camera_request.cpp
>> @@ -52,6 +52,44 @@ Camera3RequestDescriptor::Camera3RequestDescriptor(
>>   
>>   Camera3RequestDescriptor::~Camera3RequestDescriptor() = default;
>>   
>> +/**
>> + * \struct Camera3RequestDescriptor::StreamBuffer
>> + * \brief Group information for per-stream buffer of Camera3RequestDescriptor
>> + *
>> + * A capture request placed to the libcamera HAL can contain multiple streams.
>> + * Each stream will have an associated buffer to be filled. StreamBuffer
>> + * tracks this buffer with contextual information which aids in the stream's
>> + * generation. The generation of the stream will depend on its type (refer to
>> + * the CameraStream::Type documentation).
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::stream
>> + * \brief Pointer to the corresponding CameraStream
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::camera3Buffer
>> + * \brief Native handle to the buffer
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::frameBuffer
>> + * \brief Encapsulate the dmabuf handle inside a libcamera::FrameBuffer for
>> + * direct streams
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::fence
>> + * \brief Acquire fence of the buffer
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::status
>> + * \brief Track the status of the buffer
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::internalBuffer
>> + * \brief Pointer to a buffer internally handled by CameraStream (if any)
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::srcBuffer
>> + * \brief Pointer to the source frame buffer used for post-processing
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::dstBuffer
>> + * \brief Pointer to the destination frame buffer used for post-processing
>> + *
>> + * \var Camera3RequestDescriptor::StreamBuffer::request
>> + * \brief Back pointer to Camera3RequestDescriptor to which StreamBuffer belongs
> I'd put some extra 'the' in here, but not essential:
>
> 'to the Camera3RequestDescriptor'
> 'which the StreamBuffer'


Ok, I'll fixup locally.

>
>
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>


Thanks!

>
>> + */
>>   Camera3RequestDescriptor::StreamBuffer::StreamBuffer(
>>          CameraStream *cameraStream, const camera3_stream_buffer_t &buffer,
>>          Camera3RequestDescriptor *requestDescriptor)
>> diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp
>> index c2157450..94f1884c 100644
>> --- a/src/android/camera_stream.cpp
>> +++ b/src/android/camera_stream.cpp
>> @@ -244,6 +244,18 @@ void CameraStream::putBuffer(FrameBuffer *buffer)
>>          buffers_.push_back(buffer);
>>   }
>>   
>> +/**
>> + * \class CameraStream::PostProcessorWorker
>> + * \brief Post-process a CameraStream in an internal thread
>> + *
>> + * If the association between CameraStream and camera3_stream_t dictated by
>> + * CameraStream::Type is internal or mapped, the stream is generated by post
>> + * processing of a libcamera stream. Such a request is queued to a
>> + * PostProcessorWorker in CameraStream::process(). A queue of post-processing
>> + * requests is maintained by the PostProcessorWorker and it will run the
>> + * post-processing on an internal thread as soon as any request is available on
>> + * its queue.
>> + */
>>   CameraStream::PostProcessorWorker::PostProcessorWorker(PostProcessor *postProcessor)
>>          : postProcessor_(postProcessor)
>>   {
>> -- 
>> 2.31.1
>>