From patchwork Wed Sep 9 15:54:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9558 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A6C8BDB1D for ; Wed, 9 Sep 2020 15:51:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0813662D38; Wed, 9 Sep 2020 17:51:21 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6FB360534 for ; Wed, 9 Sep 2020 17:51:18 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id C30F61BF212; Wed, 9 Sep 2020 15:51:17 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:50 +0200 Message-Id: <20200909155457.153907-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/8] android: camera_device: Add CameraStream::Type X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Define the CameraStream::Type enumeration and assign it to each CameraStream instance at construction time. The CameraStream type will be used to decide if memory needs to be allocated on its behalf or if the stream is backed by memory externally allocated by the Android framework. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Kieran Bingham --- src/android/camera_device.cpp | 14 ++++-- src/android/camera_device.h | 87 ++++++++++++++++++++++++++++++++++- 2 files changed, 96 insertions(+), 5 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 50923df22a8f..3c58523e528e 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -168,9 +168,11 @@ MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer, } } -CameraStream::CameraStream(PixelFormat format, Size size, +CameraStream::CameraStream(PixelFormat format, Size size, Type type, unsigned int index, Encoder *encoder) - : format_(format), size_(size), index_(index), encoder_(encoder) + + : format_(format), size_(size), type_(type), index_(index), + encoder_(encoder) { } @@ -1215,12 +1217,14 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) config_->addConfiguration(streamConfiguration); unsigned int index = config_->size() - 1; - streams_.emplace_back(format, size, index); + streams_.emplace_back(format, size, CameraStream::Type::Direct, + index); stream->priv = static_cast(&streams_.back()); } /* Now handle MJPEG streams, adding a new stream if required. */ if (jpegStream) { + CameraStream::Type type; int index = -1; /* Search for a compatible stream in the non-JPEG ones. */ @@ -1238,6 +1242,7 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) LOG(HAL, Info) << "Android JPEG stream mapped on stream " << i; + type = CameraStream::Type::Mapped; index = i; break; } @@ -1262,6 +1267,7 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) LOG(HAL, Info) << "Adding " << streamConfiguration.toString() << " for MJPEG support"; + type = CameraStream::Type::Internal; config_->addConfiguration(streamConfiguration); index = config_->size() - 1; } @@ -1280,7 +1286,7 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return ret; } - streams_.emplace_back(formats::MJPEG, cfg.size, index, encoder); + streams_.emplace_back(formats::MJPEG, cfg.size, type, index, encoder); jpegStream->priv = static_cast(&streams_.back()); } diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 912c59aeb5a8..9dea7c42bdb5 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -30,17 +30,102 @@ class CameraMetadata; class CameraStream { public: + /* + * Enumeration of CameraStream types. + * + * A camera stream associates an Android stream to a libcamera stream. + * This enumeration describes how the two streams are associated and how + * and where data produced from libcamera are delivered to the + * Android framework. + * + * Direct: + * + * The Android stream is directly mapped onto a libcamera stream: frames + * are delivered by the library directly in the memory location + * specified by the Android stream (buffer_handle_t->data) and provided + * to the framework as they are. The Android stream characteristics are + * directly translated to the libcamera stream configuration. + * + * +-----+ +-----+ + * | A | | L | + * +-----+ +-----+ + * | | + * V V + * +-----+ +------+ + * | B |<---------------| FB | + * +-----+ +------+ + * + * + * Internal: + * + * Data for the Android stream are produced by processing a libcamera + * stream created by the HAL for that purpose. The libcamera stream + * needs to be supplied with intermediate buffers where the library + * delivers frames to be processed and then provided to the framework. + * The libcamera stream configuration is not a direct translation of the + * Android stream characteristics, but it describes the format and size + * required for the processing procedure to produce frames in the + * Android required format. + * + * +-----+ +-----+ +----+ + * | A | | L |--->| B | + * +-----+ +-----+ +----+ + * | | ^ + * V V | + * +-----+ +------+ | + * | B | | FB |-----+ + * +-----+ +------+ + * ^ | + * |-------Processing------| + * + * + * Mapped: + * + * Data for the Android stream are produced by processing a libcamera + * stream associated with another CameraStream. Mapped camera streams do + * not need any memory to be reserved for them as they process data + * produced by libcamera for a different stream whose format and size + * is compatible with the processing procedure requirements to produce + * frames in the Android required format. + * + * +-----+ +-----+ +-----+ + * | A | | A' | | L | + * +-----+ +-----+ +-----+ + * | | | + * V V V + * +-----+ +-----+ +------+ + * | B | | B' |<---------| FB | + * +-----+ +-----+ +------+ + * ^ | + * |--Processing--| + * + * + * -------------------------------------------------------------------- + * A = Android stream + * L = libcamera stream + * B = memory buffer + * FB = libcamera FrameBuffer + * "Processing" = Frame processing procedure (Encoding, scaling etc) + */ + enum Type { + Direct, + Internal, + Mapped, + }; + CameraStream(libcamera::PixelFormat format, libcamera::Size size, - unsigned int index, Encoder *encoder = nullptr); + Type type, unsigned int index, Encoder *encoder = nullptr); const libcamera::PixelFormat &format() const { return format_; } const libcamera::Size &size() const { return size_; } unsigned int index() const { return index_; } Encoder *encoder() const { return encoder_.get(); } + Type type() const { return type_; } private: libcamera::PixelFormat format_; libcamera::Size size_; + Type type_; /* * The index of the libcamera StreamConfiguration as added during * configureStreams(). A single libcamera Stream may be used to deliver From patchwork Wed Sep 9 15:54:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9559 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0C744BDB1D for ; Wed, 9 Sep 2020 15:51:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 970B662D17; Wed, 9 Sep 2020 17:51:21 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE7E660534 for ; Wed, 9 Sep 2020 17:51:19 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id E22BD1BF208; Wed, 9 Sep 2020 15:51:18 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:51 +0200 Message-Id: <20200909155457.153907-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/8] android: camera_device: Add frame allocator X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add to the CameraDevice class a FrameBufferAllocator class member. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 5 +++-- src/android/camera_device.h | 2 ++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 3c58523e528e..98cfa3d5aa59 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -14,6 +14,7 @@ #include #include +#include #include #include "libcamera/internal/formats.h" @@ -210,8 +211,8 @@ CameraDevice::Camera3RequestDescriptor::~Camera3RequestDescriptor() */ CameraDevice::CameraDevice(unsigned int id, const std::shared_ptr &camera) - : id_(id), running_(false), camera_(camera), staticMetadata_(nullptr), - facing_(CAMERA_FACING_FRONT), orientation_(0) + : id_(id), running_(false), camera_(camera), allocator_(camera), + staticMetadata_(nullptr), facing_(CAMERA_FACING_FRONT), orientation_(0) { camera_->requestCompleted.connect(this, &CameraDevice::requestComplete); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 9dea7c42bdb5..0e912030d7f2 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -16,6 +16,7 @@ #include #include +#include #include #include #include @@ -206,6 +207,7 @@ private: bool running_; std::shared_ptr camera_; std::unique_ptr config_; + libcamera::FrameBufferAllocator allocator_; CameraMetadata *staticMetadata_; std::map requestTemplates_; From patchwork Wed Sep 9 15:54:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9560 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F405EBDB1D for ; Wed, 9 Sep 2020 15:51:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BCC7262D53; Wed, 9 Sep 2020 17:51:22 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E53D862D04 for ; Wed, 9 Sep 2020 17:51:20 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 13EFD1BF20D; Wed, 9 Sep 2020 15:51:19 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:52 +0200 Message-Id: <20200909155457.153907-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/8] libcamera: frame_buffer_allocator: Add clear() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a clear() method to the FrameBufferAllocator class that frees all the buffers previously reserved by the allocator. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- include/libcamera/framebuffer_allocator.h | 1 + src/libcamera/framebuffer_allocator.cpp | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/include/libcamera/framebuffer_allocator.h b/include/libcamera/framebuffer_allocator.h index 78f1353964eb..2a4d538a0cb2 100644 --- a/include/libcamera/framebuffer_allocator.h +++ b/include/libcamera/framebuffer_allocator.h @@ -28,6 +28,7 @@ public: int allocate(Stream *stream); int free(Stream *stream); + void clear(); bool allocated() const { return !buffers_.empty(); } const std::vector> &buffers(Stream *stream) const; diff --git a/src/libcamera/framebuffer_allocator.cpp b/src/libcamera/framebuffer_allocator.cpp index 2fbba37a1b0b..7ed80011c845 100644 --- a/src/libcamera/framebuffer_allocator.cpp +++ b/src/libcamera/framebuffer_allocator.cpp @@ -125,6 +125,14 @@ int FrameBufferAllocator::free(Stream *stream) return 0; } +/** + * \brief Free all the buffers previously allocated + */ +void FrameBufferAllocator::clear() +{ + buffers_.clear(); +} + /** * \fn FrameBufferAllocator::allocated() * \brief Check if the allocator has allocated buffers for any stream From patchwork Wed Sep 9 15:54:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9561 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 27F3BBDB1D for ; Wed, 9 Sep 2020 15:51:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04CC362D5E; Wed, 9 Sep 2020 17:51:24 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B2A862D04 for ; Wed, 9 Sep 2020 17:51:22 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 16E301BF20C; Wed, 9 Sep 2020 15:51:20 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:53 +0200 Message-Id: <20200909155457.153907-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/8] android: camera_device: Clear allocator at configureStream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configureStream operation might be called by the Android framework in two successive capture session without going through a close(). Clear all the allocated buffers before configuring the camera streams. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 98cfa3d5aa59..d35c6b93b654 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1187,6 +1187,7 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) */ streams_.clear(); streams_.reserve(stream_list->num_streams); + allocator_.clear(); /* First handle all non-MJPEG streams. */ camera3_stream_t *jpegStream = nullptr; From patchwork Wed Sep 9 15:54:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9562 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 78114BDB1E for ; Wed, 9 Sep 2020 15:51:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4821F62D68; Wed, 9 Sep 2020 17:51:24 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1055262D04 for ; Wed, 9 Sep 2020 17:51:23 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 2FDFF1BF208; Wed, 9 Sep 2020 15:51:22 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:54 +0200 Message-Id: <20200909155457.153907-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/8] libcamera: framebuffer_allocator: Get and return buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add to the FrameBufferAllocator class two methods to get and return buffers from the pool of buffers allocated for a Stream. The two methods return pointer to the allocated buffers without transferring ownership to the caller. Signed-off-by: Jacopo Mondi --- include/libcamera/framebuffer_allocator.h | 5 ++ src/libcamera/framebuffer_allocator.cpp | 60 +++++++++++++++++++++-- 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/include/libcamera/framebuffer_allocator.h b/include/libcamera/framebuffer_allocator.h index 2a4d538a0cb2..1f3f10d4ec03 100644 --- a/include/libcamera/framebuffer_allocator.h +++ b/include/libcamera/framebuffer_allocator.h @@ -9,6 +9,7 @@ #include #include +#include #include namespace libcamera { @@ -33,9 +34,13 @@ public: bool allocated() const { return !buffers_.empty(); } const std::vector> &buffers(Stream *stream) const; + FrameBuffer *getBuffer(Stream *stream); + void returnBuffer(Stream *stream, FrameBuffer *buffer); + private: std::shared_ptr camera_; std::map>> buffers_; + std::map> availableBuffers_; }; } /* namespace libcamera */ diff --git a/src/libcamera/framebuffer_allocator.cpp b/src/libcamera/framebuffer_allocator.cpp index 7ed80011c845..7429d6b9edb7 100644 --- a/src/libcamera/framebuffer_allocator.cpp +++ b/src/libcamera/framebuffer_allocator.cpp @@ -64,7 +64,7 @@ FrameBufferAllocator::FrameBufferAllocator(std::shared_ptr camera) FrameBufferAllocator::~FrameBufferAllocator() { - buffers_.clear(); + clear(); } /** @@ -93,11 +93,17 @@ int FrameBufferAllocator::allocate(Stream *stream) } int ret = camera_->exportFrameBuffers(stream, &buffers_[stream]); - if (ret == -EINVAL) + if (ret == -EINVAL) { LOG(Allocator, Error) << "Stream is not part of " << camera_->id() << " active configuration"; - return ret; + return ret; + } + + for (const auto &buffer : buffers_[stream]) + availableBuffers_[stream].push(buffer.get()); + + return 0; } /** @@ -122,6 +128,9 @@ int FrameBufferAllocator::free(Stream *stream) buffers.clear(); buffers_.erase(iter); + availableBuffers_[stream] = {}; + availableBuffers_.erase(availableBuffers_.find(stream)); + return 0; } @@ -131,6 +140,7 @@ int FrameBufferAllocator::free(Stream *stream) void FrameBufferAllocator::clear() { buffers_.clear(); + availableBuffers_.clear(); } /** @@ -162,4 +172,48 @@ FrameBufferAllocator::buffers(Stream *stream) const return iter->second; } +/** + * \brief Get a pointer to a \a buffer for the \a stream + * \param[in] stream The stream to get a buffer for + * + * The method returns a pointer to a FrameBuffer but does transfer the buffer + * ownership to the caller: the returned pointer remains valid until the + * FrameBufferAllocator does not get deleted or the allocated buffers do not get + * released with a call for free() or clear(). + * + * \return A FrameBuffer pointer or nullptr if the no buffers is available + */ +FrameBuffer *FrameBufferAllocator::getBuffer(Stream *stream) +{ + if (!allocated() || buffers_[stream].empty()) + return nullptr; + + FrameBuffer *frameBuffer = availableBuffers_[stream].front(); + availableBuffers_[stream].pop(); + + return frameBuffer; +} + +/** + * \brief Return a \a buffer to the list of buffers available for the a \a stream + * \param[in] stream The stream to return buffer to + * \param[in] buffer The buffer to return + */ +void FrameBufferAllocator::returnBuffer(Stream *stream, FrameBuffer *buffer) +{ + if (!allocated()) + return; + + for (const auto &b : buffers_[stream]) { + /* + * Return the buffer to the available queue only if it was part + * of the vector of buffers allocated for the Stream. + */ + if (b.get() != buffer) + continue; + + availableBuffers_[stream].push(buffer); + } +} + } /* namespace libcamera */ From patchwork Wed Sep 9 15:54:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9563 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F7D9BDB1D for ; Wed, 9 Sep 2020 15:51:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6D1E762D70; Wed, 9 Sep 2020 17:51:26 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1B7EE62D62 for ; Wed, 9 Sep 2020 17:51:24 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 357361BF20E; Wed, 9 Sep 2020 15:51:23 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:55 +0200 Message-Id: <20200909155457.153907-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 6/8] android: camera_device: Allocate buffer pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" CameraStream instances that are not directly mapped to application facing Android streams need memory to be reserved in libcamera on their behalf. After the libcamera::Camera has been configured, reserve memory for the CameraStream isntances that need a pool of libcamera allocated buffers. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index d35c6b93b654..a7cb1be03b9a 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1329,6 +1329,26 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return ret; } + /* + * Allocate buffers in libcamera for stream not directly mapped to + * Android streams. + */ + for (unsigned int i = 0; i < stream_list->num_streams; ++i) { + camera3_stream_t *camera3Stream = stream_list->streams[i]; + CameraStream *cameraStream = static_cast(camera3Stream->priv); + StreamConfiguration &cfg = config_->at(cameraStream->index()); + + if (cameraStream->type() != CameraStream::Type::Internal) + continue; + + int ret = allocator_.allocate(cfg.stream()); + if (ret < 0) { + LOG(HAL, Error) + << "Failed to allocate buffers for stream " << i; + return ret; + } + } + return 0; } From patchwork Wed Sep 9 15:54:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9564 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CC781BDB1E for ; Wed, 9 Sep 2020 15:51:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 88C1E62D77; Wed, 9 Sep 2020 17:51:26 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A7DF62D62 for ; Wed, 9 Sep 2020 17:51:25 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 402381BF213; Wed, 9 Sep 2020 15:51:24 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:56 +0200 Message-Id: <20200909155457.153907-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 7/8] android: camera_device: Use libcamera buffer pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that we have reserved and made available to the camera HAL a pool of libcamera allocated buffers, use them when a CameraStream instance that requires internal allocation is processed. Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 47 +++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 13 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a7cb1be03b9a..f94b313581e7 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1422,6 +1422,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques for (unsigned int i = 0; i < descriptor->numBuffers; ++i) { CameraStream *cameraStream = static_cast(camera3Buffers[i].stream->priv); + const StreamConfiguration &config = config_->at(cameraStream->index()); + Stream *stream = config.stream(); /* * Keep track of which stream the request belongs to and store @@ -1430,27 +1432,39 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques descriptor->buffers[i].stream = camera3Buffers[i].stream; descriptor->buffers[i].buffer = camera3Buffers[i].buffer; - /* Software streams are handled after hardware streams complete. */ - if (cameraStream->format() == formats::MJPEG) + /* Mapped streams don't need to be added to the Request. */ + if (cameraStream->type() == CameraStream::Type::Mapped) continue; - /* - * Create a libcamera buffer using the dmabuf descriptors of - * the camera3Buffer for each stream. The FrameBuffer is - * directly associated with the Camera3RequestDescriptor for - * lifetime management only. - */ - FrameBuffer *buffer = createFrameBuffer(*camera3Buffers[i].buffer); + FrameBuffer *buffer; + if (cameraStream->type() == CameraStream::Type::Direct) { + /* + * Create a libcamera buffer using the dmabuf + * descriptors of the camera3Buffer for each stream and + * associate it with the Camera3RequestDescriptor for + * lifetime management only. + */ + buffer = createFrameBuffer(*camera3Buffers[i].buffer); + descriptor->frameBuffers.emplace_back(buffer); + + } else { + /* + * Get the frame buffer from the CameraStream internal + * buffer pool. The lifetime management of internal + * buffers is connected to the one of the + * FrameBufferAllocator instance. + * + * The retrieved buffer has to be returned to the + * allocator once it has been processed. + */ + buffer = allocator_.getBuffer(stream); + } if (!buffer) { LOG(HAL, Error) << "Failed to create buffer"; delete request; delete descriptor; return -ENOMEM; } - descriptor->frameBuffers.emplace_back(buffer); - - StreamConfiguration *streamConfiguration = &config_->at(cameraStream->index()); - Stream *stream = streamConfiguration->stream(); request->addBuffer(stream, buffer); } @@ -1561,6 +1575,13 @@ void CameraDevice::requestComplete(Request *request) const uint32_t jpeg_orientation = 0; resultMetadata->addEntry(ANDROID_JPEG_ORIENTATION, &jpeg_orientation, 1); + + /* + * Return the FrameBuffer to the CameraStream now that we're + * done processing it. + */ + if (cameraStream->type() == CameraStream::Type::Internal) + allocator_.returnBuffer(stream, buffer); } /* Prepare to call back the Android camera stack. */ From patchwork Wed Sep 9 15:54:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9565 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3B8A5BDB1D for ; Wed, 9 Sep 2020 15:51:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1036862D6A; Wed, 9 Sep 2020 17:51:28 +0200 (CEST) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4544162C30 for ; Wed, 9 Sep 2020 17:51:26 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 5C3E01BF208; Wed, 9 Sep 2020 15:51:25 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 9 Sep 2020 17:54:57 +0200 Message-Id: <20200909155457.153907-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200909155457.153907-1-jacopo@jmondi.org> References: <20200909155457.153907-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 8/8] android: camera_device: Add stream mapping log X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hanlinchen@chromium.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To ease following how Android stream gets mapped onto libcamera ones add a (quite verbose) printout before queueing a request to libcamera. The output looks like: 0 - (320x240)[0x00000022] -> (320x240)[NV12] (external) 1 - (640x480)[0x00000021] -> (640x480)[NV12] (internal) Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f94b313581e7..811f0a66cf2f 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1419,9 +1419,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques Request *request = camera_->createRequest(reinterpret_cast(descriptor)); + LOG(HAL, Debug) << "Queueing Request to libcamera with " + << descriptor->numBuffers << " streams"; for (unsigned int i = 0; i < descriptor->numBuffers; ++i) { + camera3_stream *camera3Stream = camera3Buffers[i].stream; CameraStream *cameraStream = - static_cast(camera3Buffers[i].stream->priv); + static_cast(camera3Stream->priv); const StreamConfiguration &config = config_->at(cameraStream->index()); Stream *stream = config.stream(); @@ -1432,9 +1435,18 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques descriptor->buffers[i].stream = camera3Buffers[i].stream; descriptor->buffers[i].buffer = camera3Buffers[i].buffer; - /* Mapped streams don't need to be added to the Request. */ - if (cameraStream->type() == CameraStream::Type::Mapped) + std::stringstream ss; + ss << i << " - ("; + ss << camera3Stream->width << "x" << camera3Stream->height << ")" + << "[" << utils::hex(camera3Stream->format) << "] -> " + << "(" << config.size.toString() << ")[" + << config.pixelFormat.toString() << "]"; + + /* Mapped streams don't need buffers added to the Request. */ + if (cameraStream->type() == CameraStream::Type::Mapped) { + LOG(HAL, Debug) << ss.str() << " (mapped)"; continue; + } FrameBuffer *buffer; if (cameraStream->type() == CameraStream::Type::Direct) { @@ -1446,6 +1458,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques */ buffer = createFrameBuffer(*camera3Buffers[i].buffer); descriptor->frameBuffers.emplace_back(buffer); + ss << " (external)"; } else { /* @@ -1458,7 +1471,11 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * allocator once it has been processed. */ buffer = allocator_.getBuffer(stream); + ss << " (internal)"; } + + LOG(HAL, Debug) << ss.str(); + if (!buffer) { LOG(HAL, Error) << "Failed to create buffer"; delete request;