From patchwork Sat Oct 10 09:58:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 10042 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89B9FBEEDF for ; Sat, 10 Oct 2020 09:54:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C675F609C4; Sat, 10 Oct 2020 11:54:38 +0200 (CEST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D4CD1603C1 for ; Sat, 10 Oct 2020 11:54:36 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 789EA40002; Sat, 10 Oct 2020 09:54:36 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Sat, 10 Oct 2020 11:58:28 +0200 Message-Id: <20201010095830.134084-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201010095830.134084-1-jacopo@jmondi.org> References: <20201010095830.134084-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/3] android: camera_worker: Introduce CameraWorker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Android camera framework provides for each buffer part of a capture request an acquisition fence the camera HAL is supposed to wait on before using the buffer. As the libcamera HAL runs in the camera service thread, it is not possible to perform a synchronous wait there. Introduce a CameraWorker class that runs an internal thread to wait on a set of fences before queueing a capture request to the libcamera::Camera. Fences completion is handled through a simple poll, similar in implementation to the sync_wait() function provided by libdrm. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- src/android/camera_worker.cpp | 122 ++++++++++++++++++++++++++++++++++ src/android/camera_worker.h | 63 ++++++++++++++++++ src/android/meson.build | 1 + 3 files changed, 186 insertions(+) create mode 100644 src/android/camera_worker.cpp create mode 100644 src/android/camera_worker.h diff --git a/src/android/camera_worker.cpp b/src/android/camera_worker.cpp new file mode 100644 index 000000000000..682ad1a82386 --- /dev/null +++ b/src/android/camera_worker.cpp @@ -0,0 +1,122 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * camera_worker.cpp - Process capture requests on behalf of the Camera HAL + */ + +#include "camera_worker.h" + +#include +#include +#include + +#include "camera_device.h" + +using namespace libcamera; + +LOG_DECLARE_CATEGORY(HAL); + +/* + * \class CaptureRequest + * \brief Wrap a libcamera::Request associated with buffers and fences + * + * A CaptureRequest is constructed by the CameraDevice, filled with + * buffers and fences provided by the camera3 framework and then processed + * by the CameraWorker which queues it to the libcamera::Camera after handling + * fences. + */ +CaptureRequest::CaptureRequest(libcamera::Camera *camera, uint64_t cookie) + : camera_(camera) +{ + request_ = camera_->createRequest(cookie); +} + +void CaptureRequest::addBuffer(Stream *stream, FrameBuffer *buffer, int fence) +{ + request_->addBuffer(stream, buffer); + acquireFences_.push_back(fence); +} + +void CaptureRequest::queue() +{ + camera_->queueRequest(request_.get()); +} + +/* + * \class CameraWorker + * \brief Process a CaptureRequest on an internal thread + * + * The CameraWorker class wraps a Worker that runs on an internal thread + * and schedules processing of CaptureRequest through it. + */ +CameraWorker::CameraWorker() +{ + worker_.moveToThread(&thread_); +} + +void CameraWorker::start() +{ + thread_.start(); +} + +void CameraWorker::stop() +{ + thread_.exit(); + thread_.wait(); +} + +void CameraWorker::queueRequest(CaptureRequest *request) +{ + /* Async process the request on the worker which runs its own thread. */ + worker_.invokeMethod(&Worker::processRequest, ConnectionTypeQueued, + request); +} + +/* + * \class CameraWorker::Worker + * \brief Process a CaptureRequest handling acquisition fences + */ +int CameraWorker::Worker::waitFence(int fence) +{ + /* + * \todo Better characterize the timeout. Currently equal to the one + * used by the Rockchip Camera HAL on ChromeOS. + */ + constexpr unsigned int timeoutMs = 300; + struct pollfd fds = { fence, POLLIN, 0 }; + + do { + int ret = poll(&fds, 1, timeoutMs); + if (ret == 0) + return -ETIME; + + if (ret > 0) { + if (fds.revents & (POLLERR | POLLNVAL)) + return -EINVAL; + + return 0; + } + } while (errno == EINTR || errno == EAGAIN); + + return -errno; +} + +void CameraWorker::Worker::processRequest(CaptureRequest *request) +{ + /* Wait on all fences before queuing the Request. */ + for (int fence : request->fences()) { + if (fence == -1) + continue; + + int ret = waitFence(fence); + close(fence); + if (ret < 0) { + LOG(HAL, Error) << "Failed handling fence: " + << fence << ": " << strerror(-ret); + return; + } + } + + request->queue(); +} diff --git a/src/android/camera_worker.h b/src/android/camera_worker.h new file mode 100644 index 000000000000..fff5021708d7 --- /dev/null +++ b/src/android/camera_worker.h @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * camera_worker.h - Process capture requests on behalf of the Camera HAL + */ +#ifndef __ANDROID_CAMERA_WORKER_H__ +#define __ANDROID_CAMERA_WORKER_H__ + +#include + +#include +#include +#include +#include +#include + +#include "libcamera/internal/thread.h" + +class CameraDevice; + +class CaptureRequest +{ +public: + CaptureRequest(libcamera::Camera *camera, uint64_t cookie); + + const std::vector &fences() const { return acquireFences_; } + + void addBuffer(libcamera::Stream *stream, + libcamera::FrameBuffer *buffer, int fence); + void queue(); + +private: + libcamera::Camera *camera_; + std::vector acquireFences_; + std::unique_ptr request_; +}; + +class CameraWorker +{ +public: + CameraWorker(); + + void start(); + void stop(); + + void queueRequest(CaptureRequest *request); + +private: + class Worker : public libcamera::Object + { + public: + void processRequest(CaptureRequest *request); + + private: + int waitFence(int fence); + }; + + Worker worker_; + libcamera::Thread thread_; +}; + +#endif /* __ANDROID_CAMERA_WORKER_H__ */ diff --git a/src/android/meson.build b/src/android/meson.build index 802bb89afe57..b2b2293cf62d 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -21,6 +21,7 @@ android_hal_sources = files([ 'camera_metadata.cpp', 'camera_ops.cpp', 'camera_stream.cpp', + 'camera_worker.cpp', 'jpeg/encoder_libjpeg.cpp', 'jpeg/exif.cpp', ]) From patchwork Sat Oct 10 09:58:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 10043 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36EE2BEEDF for ; Sat, 10 Oct 2020 09:54:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 05D2B60890; Sat, 10 Oct 2020 11:54:40 +0200 (CEST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 64847603C1 for ; Sat, 10 Oct 2020 11:54:37 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 04B0340002; Sat, 10 Oct 2020 09:54:36 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Sat, 10 Oct 2020 11:58:29 +0200 Message-Id: <20201010095830.134084-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201010095830.134084-1-jacopo@jmondi.org> References: <20201010095830.134084-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/3] android: camera_device: Queue request using Worker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a CameraWorker class member to the CameraDevice class and queue capture requests to it to delegate its handling. Start and stop the CameraWorker when the libcamera::Camera is started or stopped. Tie the CaptureRequest lifetime to the Camera3RequestDescriptor's one by storing it as unique_ptr<> in the descriptor to simplify handling of request creation and deletion. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- src/android/camera_device.cpp | 42 ++++++++++++++++++----------------- src/android/camera_device.h | 7 +++++- 2 files changed, 28 insertions(+), 21 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index c29fcb43fe2d..0036f1140560 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -168,11 +168,24 @@ MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer, */ CameraDevice::Camera3RequestDescriptor::Camera3RequestDescriptor( - unsigned int frameNumber, unsigned int numBuffers) + Camera *camera, unsigned int frameNumber, unsigned int numBuffers) : frameNumber(frameNumber), numBuffers(numBuffers) { buffers = new camera3_stream_buffer_t[numBuffers]; + + /* + * FrameBuffer instances created by wrapping a camera3 provided dmabuf + * are emplaced in this vector of unique_ptr<> for lifetime management. + */ frameBuffers.reserve(numBuffers); + + /* + * Create the libcamera::Request unique_ptr<> to tie its lifetime + * to the descriptor's one. Set the descriptor's address as the + * request's cookie to retrieve it at completion time. + */ + request = std::make_unique(camera, + reinterpret_cast(this)); } CameraDevice::Camera3RequestDescriptor::~Camera3RequestDescriptor() @@ -519,6 +532,7 @@ void CameraDevice::close() { streams_.clear(); + worker_.stop(); camera_->stop(); camera_->release(); @@ -1350,6 +1364,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques /* Start the camera if that's the first request we handle. */ if (!running_) { + worker_.start(); + int ret = camera_->start(); if (ret) { LOG(HAL, Error) << "Failed to start camera"; @@ -1372,16 +1388,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * at request complete time. */ Camera3RequestDescriptor *descriptor = - new Camera3RequestDescriptor(camera3Request->frame_number, + new Camera3RequestDescriptor(camera_.get(), camera3Request->frame_number, camera3Request->num_output_buffers); - std::unique_ptr request = - camera_->createRequest(reinterpret_cast(descriptor)); - if (!request) { - LOG(HAL, Error) << "Failed to create request"; - return -ENOMEM; - } - LOG(HAL, Debug) << "Queueing Request to libcamera with " << descriptor->numBuffers << " HAL streams"; for (unsigned int i = 0; i < descriptor->numBuffers; ++i) { @@ -1448,18 +1457,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques return -ENOMEM; } - request->addBuffer(cameraStream->stream(), buffer); - } - - int ret = camera_->queueRequest(request.get()); - if (ret) { - LOG(HAL, Error) << "Failed to queue request"; - delete descriptor; - return ret; + descriptor->request->addBuffer(cameraStream->stream(), buffer, + camera3Buffers[i].acquire_fence); } - /* The request will be deleted in the completion handler. */ - request.release(); + /* Queue the request on the CameraWorker. */ + worker_.queueRequest(descriptor->request.get()); return 0; } @@ -1564,7 +1567,6 @@ void CameraDevice::requestComplete(Request *request) callbacks_->process_capture_result(callbacks_, &captureResult); delete descriptor; - delete request; } std::string CameraDevice::logPrefix() const diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 777d1a35e801..86f2b8974b53 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -25,6 +25,7 @@ #include "libcamera/internal/message.h" #include "camera_stream.h" +#include "camera_worker.h" #include "jpeg/encoder.h" class CameraMetadata; @@ -73,7 +74,8 @@ private: CameraDevice(unsigned int id, const std::shared_ptr &camera); struct Camera3RequestDescriptor { - Camera3RequestDescriptor(unsigned int frameNumber, + Camera3RequestDescriptor(libcamera::Camera *camera, + unsigned int frameNumber, unsigned int numBuffers); ~Camera3RequestDescriptor(); @@ -81,6 +83,7 @@ private: uint32_t numBuffers; camera3_stream_buffer_t *buffers; std::vector> frameBuffers; + std::unique_ptr request; }; struct Camera3StreamConfiguration { @@ -108,6 +111,8 @@ private: unsigned int id_; camera3_device_t camera3Device_; + CameraWorker worker_; + bool running_; std::shared_ptr camera_; std::unique_ptr config_; From patchwork Sat Oct 10 09:58:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 10044 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9AA4DBEEE1 for ; Sat, 10 Oct 2020 09:54:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B29A60C0A; Sat, 10 Oct 2020 11:54:40 +0200 (CEST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB20C603C1 for ; Sat, 10 Oct 2020 11:54:37 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id 8F8CF40002; Sat, 10 Oct 2020 09:54:37 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Sat, 10 Oct 2020 11:58:30 +0200 Message-Id: <20201010095830.134084-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201010095830.134084-1-jacopo@jmondi.org> References: <20201010095830.134084-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] android: camera_stream: Add documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a brief documentation block to the CameraStream class. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- src/android/camera_stream.cpp | 21 +++++++++++++++++++++ src/android/camera_stream.h | 5 ----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index eac1480530f8..d8e45c2a274d 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -17,6 +17,27 @@ using namespace libcamera; LOG_DECLARE_CATEGORY(HAL); +/* + * \class CameraStream + * \brief Map a camera3_stream_t to a StreamConfiguration + * + * The CameraStream class maps a camera3_stream_t provided by Android + * camera framework to a libcamera::StreamConfiguration. + * + * The StreamConfiguration is represented by its index as recorded in the + * CameraConfiguration and not by pointer as StreamConfiguration is subject to + * relocation. + * + * A single StreamConfiguration may be used to deliver one or more streams to + * the Android framework. The mapping type between a camera3 stream to a + * StreamConfiguration is described by the CameraStream::Type. + * + * CameraStream handles all the aspects of producing a stream with the size + * and format requested by the camera3 stream from the data produced by + * the associated libcamera::Stream, including the creation of the encoder + * and buffer allocation. + */ + CameraStream::CameraStream(CameraDevice *cameraDevice, Type type, camera3_stream_t *camera3Stream, unsigned int index) : cameraDevice_(cameraDevice), type_(type), diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 8df0101c403c..47f3fc61ac2c 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -128,11 +128,6 @@ private: libcamera::CameraConfiguration *config_; Type type_; camera3_stream_t *camera3Stream_; - /* - * The index of the libcamera StreamConfiguration as added during - * configureStreams(). A single libcamera Stream may be used to deliver - * one or more streams to the Android framework. - */ unsigned int index_; std::unique_ptr encoder_;