From patchwork Tue Jan 4 06:52:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15240 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AF505BE080 for ; Tue, 4 Jan 2022 06:52:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 633B360868; Tue, 4 Jan 2022 07:52:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZS7qRhbT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 92695604F8 for ; Tue, 4 Jan 2022 07:52:15 +0100 (CET) Received: from perceval.ideasonboard.com (unknown [27.57.188.152]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 08FD3501; Tue, 4 Jan 2022 07:52:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1641279135; bh=Ks6nKJAYbpZIrmZMuXMEfBfagqHt24KWdGQvIJ/x5zE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZS7qRhbTLt76MpJmLN07FUhpZeF/uu6Fe4IOyqMpu4nrjv9/7j5xESmJBuQeb17OD Be4wVWVF6MwJMnm8IEFlWNxR31M7HM/8bCNapF5DZ5j7XOb0KvWO/AB55QvGAeKJYZ nMMZLxfikmoxval1i3QLIt5QZcteNfhO2IaJciec= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Jan 2022 12:22:00 +0530 Message-Id: <20220104065201.25744-2-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220104065201.25744-1-umang.jain@ideasonboard.com> References: <20220104065201.25744-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/2] android: Document the structures and functions for post-processing X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Specifically document: - CameraDevice::sendCaptureResults() - CameraDevice::completeDescriptor() - CameraDevice::streamProcessingComplete() - CameraStream::PostProcessorWorker class - Camera3RequestDescriptor::StreamBuffer structure Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/android/camera_device.cpp | 37 +++++++++++++++++++++++++++++++++ src/android/camera_request.cpp | 38 ++++++++++++++++++++++++++++++++++ src/android/camera_stream.cpp | 12 +++++++++++ 3 files changed, 87 insertions(+) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 83825736..00d48471 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1164,6 +1164,17 @@ void CameraDevice::requestComplete(Request *request) } } +/** + * \brief Complete the Camera3RequestDescriptor + * \param[in] descriptor The Camera3RequestDescriptor that has completed + * + * The function marks the Camera3RequestDescriptor as 'complete'. It shall be + * called when all the streams in the Camera3RequestDescriptor have completed + * capture (or have been generated via post-processing) and the request is ready + * to be sent back to the framework. + * + * \context This function is \threadsafe. + */ void CameraDevice::completeDescriptor(Camera3RequestDescriptor *descriptor) { MutexLocker lock(descriptorsMutex_); @@ -1172,6 +1183,19 @@ void CameraDevice::completeDescriptor(Camera3RequestDescriptor *descriptor) sendCaptureResults(); } +/** + * \brief Sequentially send capture results to the framework + * + * Iterate over the descriptors queue to send completed descriptors back to the + * framework, in the same order as they have been queued. For each complete + * descriptor, populate a locally-scoped camera3_capture_result_t from the + * descriptor, send the capture result back by calling the + * process_capture_result() callback, and remove the descriptor from the queue. + * Stop iterating if the descriptor at the front of the queue is not complete. + * + * This function should never be called directly in the codebase. Use + * completeDescriptor() instead. + */ void CameraDevice::sendCaptureResults() { while (!descriptors_.empty() && !descriptors_.front()->isPending()) { @@ -1231,6 +1255,19 @@ void CameraDevice::setBufferStatus(Camera3RequestDescriptor::StreamBuffer &strea } } +/** + * \brief Handle post-processing completion of a stream in a capture request + * \param[in] streamBuffer The StreamBuffer for which processing is complete + * \param[in] status Stream post-processing status + * + * This function is called from the post-processor's thread whenever a camera + * stream has finished post processing. The corresponding entry is dropped from + * the descriptor's pendingStreamsToProcess_ map. + * + * If the pendingStreamsToProcess_ map is then empty, all streams requiring to + * be generated from post-processing have been completed. Mark the descriptor as + * complete using completeDescriptor() in that case. + */ void CameraDevice::streamProcessingComplete(Camera3RequestDescriptor::StreamBuffer *streamBuffer, Camera3RequestDescriptor::Status status) { diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 91776907..027b46d3 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -52,6 +52,44 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; +/** + * \struct Camera3RequestDescriptor::StreamBuffer + * \brief Group information for per-stream buffer of Camera3RequestDescriptor + * + * A capture request placed to the libcamera HAL can contain multiple streams. + * Each stream will have an associated buffer to be filled. StreamBuffer + * tracks this buffer with contextual information which aids in the stream's + * generation. The generation of the stream will depend on its type (refer to + * the CameraStream::Type documentation). + * + * \var Camera3RequestDescriptor::StreamBuffer::stream + * \brief Pointer to the corresponding CameraStream + * + * \var Camera3RequestDescriptor::StreamBuffer::camera3Buffer + * \brief Native handle to the buffer + * + * \var Camera3RequestDescriptor::StreamBuffer::frameBuffer + * \brief Encapsulate the dmabuf handle inside a libcamera::FrameBuffer for + * direct streams + * + * \var Camera3RequestDescriptor::StreamBuffer::fence + * \brief Acquire fence of the buffer + * + * \var Camera3RequestDescriptor::StreamBuffer::status + * \brief Track the status of the buffer + * + * \var Camera3RequestDescriptor::StreamBuffer::internalBuffer + * \brief Pointer to a buffer internally handled by CameraStream (if any) + * + * \var Camera3RequestDescriptor::StreamBuffer::srcBuffer + * \brief Pointer to the source frame buffer used for post-processing + * + * \var Camera3RequestDescriptor::StreamBuffer::dstBuffer + * \brief Pointer to the destination frame buffer used for post-processing + * + * \var Camera3RequestDescriptor::StreamBuffer::request + * \brief Back pointer to Camera3RequestDescriptor to which StreamBuffer belongs + */ Camera3RequestDescriptor::StreamBuffer::StreamBuffer( CameraStream *cameraStream, const camera3_stream_buffer_t &buffer, Camera3RequestDescriptor *requestDescriptor) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index c2157450..94f1884c 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -244,6 +244,18 @@ void CameraStream::putBuffer(FrameBuffer *buffer) buffers_.push_back(buffer); } +/** + * \class CameraStream::PostProcessorWorker + * \brief Post-process a CameraStream in an internal thread + * + * If the association between CameraStream and camera3_stream_t dictated by + * CameraStream::Type is internal or mapped, the stream is generated by post + * processing of a libcamera stream. Such a request is queued to a + * PostProcessorWorker in CameraStream::process(). A queue of post-processing + * requests is maintained by the PostProcessorWorker and it will run the + * post-processing on an internal thread as soon as any request is available on + * its queue. + */ CameraStream::PostProcessorWorker::PostProcessorWorker(PostProcessor *postProcessor) : postProcessor_(postProcessor) { From patchwork Tue Jan 4 06:52:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 15241 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1C41EBE080 for ; Tue, 4 Jan 2022 06:52:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C4A1F6091C; Tue, 4 Jan 2022 07:52:19 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ElBj9DAa"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5F5C2604F8 for ; Tue, 4 Jan 2022 07:52:18 +0100 (CET) Received: from perceval.ideasonboard.com (unknown [27.57.188.152]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A87D7501; Tue, 4 Jan 2022 07:52:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1641279138; bh=U7uviNR4LnE0/CMD5qQ14DUMdvPt61nFHYd+j2wJrRI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ElBj9DAaUq5G1gGdzCpcOXEGQ43cTDT7Mg5nJ+XfoRKE8jqyrnCodRt+5b5bGpxh1 Z/xH6gzVYZKeI6qvfEBxjgidYMOFzZ5STVE8X5uWOIeCSAgzyANh2TXEz+LgTXmXjx dqjHlj2w1QkprU8AKatABYQPffz353GQCF/b1YGc= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Tue, 4 Jan 2022 12:22:01 +0530 Message-Id: <20220104065201.25744-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220104065201.25744-1-umang.jain@ideasonboard.com> References: <20220104065201.25744-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/2] android: camera_request: Lifetime of a Camera3RequestDescriptor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This commit provides a sketch regarding Camera3RequestDescriptor which aids tracking each capture reuqest placed by the android framework to libcamera HAL. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/android/camera_request.cpp | 88 ++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 027b46d3..fceaa1f7 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -18,6 +18,94 @@ using namespace libcamera; * * A utility class that groups information about a capture request to be later * reused at request complete time to notify the framework. + * + ******************************************************************************* + * Lifetime of a Camera3RequestDescriptor tracking a capture request placed by + * Android Framework + ******************************************************************************* + * + * + * Android Framework + * │ + * │ ┌──────────────────────────────────┐ + * │ │camera3_capture_request_t │ + * │ │ │ + * │ │Requested output streams │ + * │ │ stream1 stream2 stream3 ... │ + * │ └──────────────────────────────────┘ + * ▼ + * ┌─────────────────────────────────────────────────────────────┐ + * │ libcamera HAL │ + * ├─────────────────────────────────────────────────────────────┤ + * │ CameraDevice │ + * │ │ + * │ processCaptureRequest(camera3_capture_request_t request) │ + * │ │ + * │ - Create Camera3RequestDescriptor tracking this request │ + * │ - Streams requiring post-processing are stored in the │ + * │ pendingStreamsToProcess map │ + * │ - Add this Camera3RequestDescriptor to descriptors' queue │ + * │ CameraDevice::descriptors_ │ + * │ │ ┌─────────────────────────┐ + * │ - Queue the capture request to libcamera core ────────────┤►│libcamera core │ + * │ │ ├─────────────────────────┤ + * │ │ │- Capture from Camera │ + * │ │ │ │ + * │ │ │- Emit │ + * │ │ │ Camera::requestComplete│ + * │ requestCompleted(Request *request) ◄───────────────────────┼─┼──── │ + * │ │ │ │ + * │ - Check request completion status │ └─────────────────────────┘ + * │ │ + * │ - if (pendingStreamsToProcess > 0) │ + * │ Queue all entries from pendingStreamsToProcess │ + * │ else │ │ + * │ completeDescriptor() │ └──────────────────────┐ + * │ │ │ + * │ ┌──────────────────────────┴───┬──────────────────┐ │ + * │ │ │ │ │ + * │ ┌──────────▼────────────┐ ┌───────────▼─────────┐ ▼ │ + * │ │CameraStream1 │ │CameraStream2 │ .... │ + * │ ├┬───┬───┬──────────────┤ ├┬───┬───┬────────────┤ │ + * │ ││ │ │ │ ││ │ │ │ │ + * │ │▼───▼───▼──────────────┤ │▼───▼───▼────────────┤ │ + * │ │PostProcessorWorker │ │PostProcessorWorker │ │ + * │ │ │ │ │ │ + * │ │ +------------------+ │ │ +------------------+│ │ + * │ │ | PostProcessor | │ │ | PostProcessor |│ │ + * │ │ | process() | │ │ | process() |│ │ + * │ │ | | │ │ | |│ │ + * │ │ | Emit | │ │ | Emit |│ │ + * │ │ | processComplete | │ │ | processComplete |│ │ + * │ │ | | │ │ | |│ │ + * │ │ +--------------│---+ │ │ +--------------│---+│ │ + * │ │ │ │ │ │ │ │ + * │ │ │ │ │ │ │ │ + * │ └────────────────┼──────┘ └────────────────┼────┘ │ + * │ │ │ │ + * │ │ │ │ + * │ │ │ │ + * │ ▼ ▼ │ + * │ +---------------------------------------+ +--------------+ │ + * │ | CameraDevice | | | │ + * │ | | | | │ + * │ | streamProcessingComplete() | | | │ + * │ | | | | │ + * │ | - Check and set buffer status | | .... | │ + * │ | - Remove post+processing entry | | | │ + * │ | from pendingStreamsToProcess | | | │ + * │ | | | | │ + * │ | - if (pendingStreamsToProcess.empty())| | | │ + * │ | completeDescriptor | | | │ + * │ | | | | │ + * │ +---------------------------------------+ +--------------+ │ + * │ │ + * └────────────────────────────────────────────────────────────────────────────────────┘ + * + * +-------------+ + * | | - PostProcessorWorker's thread + * | | + * +-------------+ */ Camera3RequestDescriptor::Camera3RequestDescriptor(