From patchwork Mon Jan 10 16:55:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 15266 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E0FFC3259 for ; Mon, 10 Jan 2022 16:54:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C52F6093C; Mon, 10 Jan 2022 17:54:30 +0100 (CET) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8554F6021A for ; Mon, 10 Jan 2022 17:54:29 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id CBF24240007; Mon, 10 Jan 2022 16:54:28 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 10 Jan 2022 17:55:20 +0100 Message-Id: <20220110165524.72978-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220110165524.72978-1-jacopo@jmondi.org> References: <20220110165524.72978-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/5] android: camera_stream: Add sourceStream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Hirokazu Honda Add to the CameraStream class a sourceStream field, which for streams of type Mapped contains a reference to the stream which produces the actual image data. The sourceStream of mapped streams will be used in later patches to make sure for each Mapped stream at least one libcamera::Stream is queued to the libcamera::Camera. Signed-off-by: Hirokazu Honda Signed-off-by: Jacopo Mondi --- src/android/camera_device.cpp | 9 ++++++++- src/android/camera_stream.cpp | 6 ++++-- src/android/camera_stream.h | 6 +++++- 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 83825736ce31..005d95b51a0c 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -681,10 +681,17 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) for (const auto &streamConfig : streamConfigs) { config->addConfiguration(streamConfig.config); + CameraStream *sourceStream = nullptr; for (auto &stream : streamConfig.streams) { + streams_.emplace_back(this, config.get(), stream.type, - stream.stream, config->size() - 1); + stream.stream, sourceStream, + config->size() - 1); stream.stream->priv = static_cast(&streams_.back()); + + /* Mapped streams are always associated with a Direct one. */ + if (stream.type == CameraStream::Type::Direct) + sourceStream = &streams_.back(); } } diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index c21574501916..032b0407bd32 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -52,9 +52,11 @@ LOG_DECLARE_CATEGORY(HAL) CameraStream::CameraStream(CameraDevice *const cameraDevice, CameraConfiguration *config, Type type, - camera3_stream_t *camera3Stream, unsigned int index) + camera3_stream_t *camera3Stream, + CameraStream *const sourceStream, unsigned int index) : cameraDevice_(cameraDevice), config_(config), type_(type), - camera3Stream_(camera3Stream), index_(index) + camera3Stream_(camera3Stream), sourceStream_(sourceStream), + index_(index) { } diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index f9504462b253..4c5078b2c26d 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -114,7 +114,9 @@ public: }; CameraStream(CameraDevice *const cameraDevice, libcamera::CameraConfiguration *config, Type type, - camera3_stream_t *camera3Stream, unsigned int index); + camera3_stream_t *camera3Stream, + CameraStream *const sourceStream, + unsigned int index); CameraStream(CameraStream &&other); ~CameraStream(); @@ -122,6 +124,7 @@ public: camera3_stream_t *camera3Stream() const { return camera3Stream_; } const libcamera::StreamConfiguration &configuration() const; libcamera::Stream *stream() const; + CameraStream *sourceStream() const { return sourceStream_; } int configure(); int process(Camera3RequestDescriptor::StreamBuffer *streamBuffer); @@ -167,6 +170,7 @@ private: const libcamera::CameraConfiguration *config_; const Type type_; camera3_stream_t *camera3Stream_; + CameraStream *const sourceStream_; const unsigned int index_; std::unique_ptr allocator_; From patchwork Mon Jan 10 16:55:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 15267 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0393CBE080 for ; Mon, 10 Jan 2022 16:54:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6395F6021A; Mon, 10 Jan 2022 17:54:35 +0100 (CET) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 65A4A60931 for ; Mon, 10 Jan 2022 17:54:30 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id C0AA7240007; Mon, 10 Jan 2022 16:54:29 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 10 Jan 2022 17:55:21 +0100 Message-Id: <20220110165524.72978-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220110165524.72978-1-jacopo@jmondi.org> References: <20220110165524.72978-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/5] android: camera_stream: Create allocator unconditionally X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the introduction of PlatformBufferAllocator all CameraStream can be used to allocate buffers on-demand. Create CameraStream::allocator_ and the associated mutex for all types of stream. Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_stream.cpp | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 032b0407bd32..eeeb238f395c 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -128,10 +128,8 @@ int CameraStream::configure() worker_->start(); } - if (type_ == Type::Internal) { - allocator_ = std::make_unique(cameraDevice_); - mutex_ = std::make_unique(); - } + allocator_ = std::make_unique(cameraDevice_); + mutex_ = std::make_unique(); camera3Stream_->max_buffers = configuration().bufferCount; From patchwork Mon Jan 10 16:55:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 15268 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5FC4FC325A for ; Mon, 10 Jan 2022 16:54:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B43B2604F8; Mon, 10 Jan 2022 17:54:35 +0100 (CET) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C573A604F8 for ; Mon, 10 Jan 2022 17:54:31 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 96E93240005; Mon, 10 Jan 2022 16:54:30 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 10 Jan 2022 17:55:22 +0100 Message-Id: <20220110165524.72978-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220110165524.72978-1-jacopo@jmondi.org> References: <20220110165524.72978-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/5] android: camera_device: Post-pone mapped streams handling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Mapped streams are generated by post-processing and always require a source buffer from where to process image data from. In case a Mapped stream is requested but its source stream is not, it is required to allocate a buffer on the fly and add it to the libcamera::Request. Make sure a source stream is available for all mapped streams, and if that's not the case, add a dedicated buffer to the request for that purpose. Signed-off-by: Jacopo Mondi --- src/android/camera_device.cpp | 81 +++++++++++++++++++++++++++++++---- 1 file changed, 73 insertions(+), 8 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 005d95b51a0c..a44f199d25d8 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -924,6 +925,32 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques LOG(HAL, Debug) << "Queueing request " << descriptor->request_->cookie() << " with " << descriptor->buffers_.size() << " streams"; + /* + * Collect the CameraStream associated to each requested capture stream. + * Being requestedStreams an std::set<> no duplications can happen. + */ + std::set requestedStreams; + for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { + CameraStream *cameraStream = buffer.stream; + + switch (cameraStream->type()) { + case CameraStream::Type::Mapped: + requestedStreams.insert(cameraStream->sourceStream()); + break; + + case CameraStream::Type::Direct: + case CameraStream::Type::Internal: + requestedStreams.insert(cameraStream); + break; + } + } + + /* + * Process all the Direct and Internal streams, for which the CameraStream + * they refer to is the one that points to the right libcamera::Stream. + * + * Streams of type Mapped will be handled later. + */ for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { CameraStream *cameraStream = buffer.stream; camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); @@ -946,14 +973,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques switch (cameraStream->type()) { case CameraStream::Type::Mapped: - /* - * Mapped streams don't need buffers added to the - * Request. - */ - LOG(HAL, Debug) << ss.str() << " (mapped)"; - - descriptor->pendingStreamsToProcess_.insert( - { cameraStream, &buffer }); continue; case CameraStream::Type::Direct: @@ -997,6 +1016,52 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques auto fence = std::make_unique(std::move(acquireFence)); descriptor->request_->addBuffer(cameraStream->stream(), frameBuffer, std::move(fence)); + + requestedStreams.erase(cameraStream); + } + + /* + * Now handle the mapped streams. If no buffer has been addded for them + * as their corresponding direct source stream has not been requested, + * add it here. + */ + for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { + CameraStream *cameraStream = buffer.stream; + camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); + CameraStream *sourceStream = cameraStream->sourceStream(); + FrameBuffer *frameBuffer = nullptr; + + if (cameraStream->type() != CameraStream::Type::Mapped) + continue; + + LOG(HAL, Debug) << i << " - (" << camera3Stream->width << "x" + << camera3Stream->height << ")" + << "[" << utils::hex(camera3Stream->format) << "] -> " + << "(" << cameraStream->configuration().size.toString() << ")[" + << cameraStream->configuration().pixelFormat.toString() << "]" + << " (mapped)"; + + MutexLocker lock(descriptor->streamsProcessMutex_); + descriptor->pendingStreamsToProcess_.insert({ cameraStream, &buffer }); + + /* + * Make sure the CameraStream this stream is mapped on has been + * added to the request. + */ + if (requestedStreams.find(sourceStream) == requestedStreams.end()) + continue; + + /* + * If that's not the case, we need to add a buffer to the request + * for this stream. + */ + frameBuffer = cameraStream->getBuffer(); + buffer.internalBuffer = frameBuffer; + + descriptor->request_->addBuffer(sourceStream->stream(), + frameBuffer, nullptr); + + requestedStreams.erase(sourceStream); } /* From patchwork Mon Jan 10 16:55:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 15269 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 07814C325B for ; Mon, 10 Jan 2022 16:54:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 040DF6093F; Mon, 10 Jan 2022 17:54:36 +0100 (CET) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE896604F8 for ; Mon, 10 Jan 2022 17:54:32 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id F25EB240004; Mon, 10 Jan 2022 16:54:31 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 10 Jan 2022 17:55:23 +0100 Message-Id: <20220110165524.72978-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220110165524.72978-1-jacopo@jmondi.org> References: <20220110165524.72978-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/5] android: camera_device: Use YUV post-processor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Hirokazu Honda When creating the list of StreamConfiguration to be requested to the camera, map NV12 streams of equal size and format together, so that they will be generated by using the YUV post-processor. Signed-off-by: Jacopo Mondi --- src/android/camera_device.cpp | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a44f199d25d8..4e44bcfa0852 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -605,14 +605,40 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) continue; } + /* + * While gralloc usage flags are supposed to report usage + * patterns to select a suitable buffer allocation strategy, in + * practice they're also used to make other decisions, such as + * selecting the actual format for the IMPLEMENTATION_DEFINED + * HAL pixel format. To avoid issues, we thus have to set the + * GRALLOC_USAGE_HW_CAMERA_WRITE flag unconditionally, even for + * streams that will be produced in software. + */ + stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; + + /* + * If a CameraStream with the same size and format of the + * current stream has already been requested, associate the two. + */ + auto iter = std::find_if( + streamConfigs.begin(), streamConfigs.end(), + [&size, &format](const Camera3StreamConfig &streamConfig) { + return streamConfig.config.size == size && + streamConfig.config.pixelFormat == format; + }); + if (iter != streamConfigs.end()) { + /* Add usage to copy the buffer in streams[0] to stream. */ + iter->streams[0].stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; + stream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; + iter->streams.push_back({ stream, CameraStream::Type::Mapped }); + continue; + } + Camera3StreamConfig streamConfig; streamConfig.streams = { { stream, CameraStream::Type::Direct } }; streamConfig.config.size = size; streamConfig.config.pixelFormat = format; streamConfigs.push_back(std::move(streamConfig)); - - /* This stream will be produced by hardware. */ - stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; } /* Now handle the MJPEG streams, adding a new stream if required. */ From patchwork Mon Jan 10 16:55:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 15270 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B348C325C for ; Mon, 10 Jan 2022 16:54:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 40CF560942; Mon, 10 Jan 2022 17:54:36 +0100 (CET) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::221]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CB4D4604F8 for ; Mon, 10 Jan 2022 17:54:33 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 0064E240007; Mon, 10 Jan 2022 16:54:32 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 10 Jan 2022 17:55:24 +0100 Message-Id: <20220110165524.72978-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220110165524.72978-1-jacopo@jmondi.org> References: <20220110165524.72978-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/5] android: camera_device: Print the correct number of completed streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a request completes, a debug message is generated to help identify the request and the number of streams if contains. The printed number of streams is however the number of output buffers requested by the camera framework, not the number of streams generated by libcamera. In facts, some output buffers are generated by post-processing, and not directly from the camera. As the debug message prints the libcamera identifier for the Request, it is more logical to print the number of streams generated by the camera instead of the total number of streams. Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 4e44bcfa0852..88cb4f943757 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1196,7 +1196,7 @@ void CameraDevice::requestComplete(Request *request) notifyShutter(descriptor->frameNumber_, sensorTimestamp); LOG(HAL, Debug) << "Request " << request->cookie() << " completed with " - << descriptor->buffers_.size() << " streams"; + << descriptor->request_->buffers().size() << " streams"; /* * Generate the metadata associated with the captured buffers.