From patchwork Mon Oct 18 13:29:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14162 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DA0A5C323E for ; Mon, 18 Oct 2021 13:29:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6EE9368F5E; Mon, 18 Oct 2021 15:29:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ehxGAnhO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1C3E868F5A for ; Mon, 18 Oct 2021 15:29:32 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 08E168C6; Mon, 18 Oct 2021 15:29:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563771; bh=IMH3CvuIP1TFvLkROhcZoFCCo2qEm3+QadHznJGOuMo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ehxGAnhONLe6mHs3fX2JIx4bjT22mznaKdf4mmIrmfQ6FNR2aONloaDrrzU121OQt xqhtawe+tIOyORcIPtq+XpU9m+zJYEk4WZXzrk7LpP37th3i59cA9SOKlRjrZNGUgj dpQKvfpN42qVC8C1jULj8vuUHZUaL9+pMVAKG2QU= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:13 +0530 Message-Id: <20211018132923.476242-2-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 01/11] camera_device: Remove private scope of Camera3RequestDescriptor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Camera3RequestDescriptor is a utility structure that groups information about a capture request. It can be and will be extended to preserve the context of a capture overall. Since the context of a capture needs to be shared among other classes (for e.g. CameraStream) having a private definition of the struct in CameraDevice class doesn't help. Hence, de-scope the structure so that it can be shared with other components (through references or pointers). Splitting the structure to a separate file will help avoiding circular dependencies when using it through the HAL implementation. Signed-off-by: Umang Jain Signed-off-by: Laurent Pinchart Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 43 ++++--------------------------- src/android/camera_device.h | 27 ++------------------ src/android/camera_request.cpp | 45 +++++++++++++++++++++++++++++++++ src/android/camera_request.h | 46 ++++++++++++++++++++++++++++++++++ src/android/meson.build | 1 + 5 files changed, 99 insertions(+), 63 deletions(-) create mode 100644 src/android/camera_request.cpp create mode 100644 src/android/camera_request.h diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 90186710..b4ab5da1 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -6,9 +6,6 @@ */ #include "camera_device.h" -#include "camera_hal_config.h" -#include "camera_ops.h" -#include "post_processor.h" #include #include @@ -27,6 +24,11 @@ #include "system/graphics.h" +#include "camera_hal_config.h" +#include "camera_ops.h" +#include "camera_request.h" +#include "post_processor.h" + using namespace libcamera; LOG_DECLARE_CATEGORY(HAL) @@ -213,41 +215,6 @@ bool validateCropRotate(const camera3_stream_configuration_t &streamList) } /* namespace */ -/* - * \struct Camera3RequestDescriptor - * - * A utility structure that groups information about a capture request to be - * later re-used at request complete time to notify the framework. - */ - -CameraDevice::Camera3RequestDescriptor::Camera3RequestDescriptor( - Camera *camera, const camera3_capture_request_t *camera3Request) -{ - frameNumber_ = camera3Request->frame_number; - - /* Copy the camera3 request stream information for later access. */ - const uint32_t numBuffers = camera3Request->num_output_buffers; - buffers_.resize(numBuffers); - for (uint32_t i = 0; i < numBuffers; i++) - buffers_[i] = camera3Request->output_buffers[i]; - - /* - * FrameBuffer instances created by wrapping a camera3 provided dmabuf - * are emplaced in this vector of unique_ptr<> for lifetime management. - */ - frameBuffers_.reserve(numBuffers); - - /* Clone the controls associated with the camera3 request. */ - settings_ = CameraMetadata(camera3Request->settings); - - /* - * Create the CaptureRequest, stored as a unique_ptr<> to tie its - * lifetime to the descriptor. - */ - request_ = std::make_unique(camera, - reinterpret_cast(this)); -} - /* * \class CameraDevice * diff --git a/src/android/camera_device.h b/src/android/camera_device.h index b7d774fe..86224aa1 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -33,7 +33,9 @@ #include "camera_worker.h" #include "jpeg/encoder.h" +struct Camera3RequestDescriptor; struct CameraConfigData; + class CameraDevice : protected libcamera::Loggable { public: @@ -73,31 +75,6 @@ private: CameraDevice(unsigned int id, std::shared_ptr camera); - struct Camera3RequestDescriptor { - enum class Status { - Pending, - Success, - Error, - }; - - Camera3RequestDescriptor() = default; - ~Camera3RequestDescriptor() = default; - Camera3RequestDescriptor(libcamera::Camera *camera, - const camera3_capture_request_t *camera3Request); - Camera3RequestDescriptor &operator=(Camera3RequestDescriptor &&) = default; - - bool isPending() const { return status_ == Status::Pending; } - - uint32_t frameNumber_ = 0; - std::vector buffers_; - std::vector> frameBuffers_; - CameraMetadata settings_; - std::unique_ptr request_; - - camera3_capture_result_t captureResult_ = {}; - Status status_ = Status::Pending; - }; - enum class State { Stopped, Flushing, diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp new file mode 100644 index 00000000..93e546bf --- /dev/null +++ b/src/android/camera_request.cpp @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019-2021, Google Inc. + * + * camera_request.cpp - libcamera Android Camera Request Descriptor + */ + +#include "camera_request.h" + +using namespace libcamera; + +/* + * \struct Camera3RequestDescriptor + * + * A utility structure that groups information about a capture request to be + * later re-used at request complete time to notify the framework. + */ + +Camera3RequestDescriptor::Camera3RequestDescriptor( + Camera *camera, const camera3_capture_request_t *camera3Request) +{ + frameNumber_ = camera3Request->frame_number; + + /* Copy the camera3 request stream information for later access. */ + const uint32_t numBuffers = camera3Request->num_output_buffers; + buffers_.resize(numBuffers); + for (uint32_t i = 0; i < numBuffers; i++) + buffers_[i] = camera3Request->output_buffers[i]; + + /* + * FrameBuffer instances created by wrapping a camera3 provided dmabuf + * are emplaced in this vector of unique_ptr<> for lifetime management. + */ + frameBuffers_.reserve(numBuffers); + + /* Clone the controls associated with the camera3 request. */ + settings_ = CameraMetadata(camera3Request->settings); + + /* + * Create the CaptureRequest, stored as a unique_ptr<> to tie its + * lifetime to the descriptor. + */ + request_ = std::make_unique(camera, + reinterpret_cast(this)); +} diff --git a/src/android/camera_request.h b/src/android/camera_request.h new file mode 100644 index 00000000..1346f6fa --- /dev/null +++ b/src/android/camera_request.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019-2021, Google Inc. + * + * camera_request.h - libcamera Android Camera Request Descriptor + */ +#ifndef __ANDROID_CAMERA_REQUEST_H__ +#define __ANDROID_CAMERA_REQUEST_H__ + +#include +#include + +#include +#include + +#include + +#include "camera_metadata.h" +#include "camera_worker.h" + +struct Camera3RequestDescriptor { + enum class Status { + Pending, + Success, + Error, + }; + + Camera3RequestDescriptor() = default; + ~Camera3RequestDescriptor() = default; + Camera3RequestDescriptor(libcamera::Camera *camera, + const camera3_capture_request_t *camera3Request); + Camera3RequestDescriptor &operator=(Camera3RequestDescriptor &&) = default; + + bool isPending() const { return status_ == Status::Pending; } + + uint32_t frameNumber_ = 0; + std::vector buffers_; + std::vector> frameBuffers_; + CameraMetadata settings_; + std::unique_ptr request_; + + camera3_capture_result_t captureResult_ = {}; + Status status_ = Status::Pending; +}; + +#endif /* __ANDROID_CAMERA_REQUEST_H__ */ diff --git a/src/android/meson.build b/src/android/meson.build index 7d1e7e85..332b177c 100644 --- a/src/android/meson.build +++ b/src/android/meson.build @@ -45,6 +45,7 @@ android_hal_sources = files([ 'camera_hal_manager.cpp', 'camera_metadata.cpp', 'camera_ops.cpp', + 'camera_request.cpp', 'camera_stream.cpp', 'camera_worker.cpp', 'jpeg/encoder_libjpeg.cpp', From patchwork Mon Oct 18 13:29:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14163 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 70FABC323E for ; Mon, 18 Oct 2021 13:29:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2F50E68F5A; Mon, 18 Oct 2021 15:29:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VcJkTqmH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 78BDE68F57 for ; Mon, 18 Oct 2021 15:29:35 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 866968C6; Mon, 18 Oct 2021 15:29:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563775; bh=5iB9km7IMBEDcUdM5ylOEdmn19Z5Ix4rCd45JQxh328=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VcJkTqmHrITZoV87loMX6L97Xam0+HgKcIeA+ztxoHFpbZ7qT4iyglq3Z4wolqwnW 8X7SiopLRS0KklbaUF5rXmC6dod6ERi6rppIbEnYXN7RsJDgFrLNsrKTC3xQrDFMtL x/uR5Q876slGyXX2HCohvag+3fq+4d12Am+do/AY= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:14 +0530 Message-Id: <20211018132923.476242-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 02/11] android: camera_request: Turn struct into a class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart The Camera3RequestDescriptor structure is growing into an object with member functions. Turn it into a class, uninline the destructor to reduce code size, explicitly disable copy as requests are not copyable, and delete the default constructor to force all instances to be fully constructed. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.h | 2 +- src/android/camera_request.cpp | 8 +++++--- src/android/camera_request.h | 13 +++++++++---- 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 86224aa1..863cf414 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -33,7 +33,7 @@ #include "camera_worker.h" #include "jpeg/encoder.h" -struct Camera3RequestDescriptor; +class Camera3RequestDescriptor; struct CameraConfigData; class CameraDevice : protected libcamera::Loggable diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 93e546bf..16a632b3 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -10,10 +10,10 @@ using namespace libcamera; /* - * \struct Camera3RequestDescriptor + * \class Camera3RequestDescriptor * - * A utility structure that groups information about a capture request to be - * later re-used at request complete time to notify the framework. + * A utility class that groups information about a capture request to be later + * reused at request complete time to notify the framework. */ Camera3RequestDescriptor::Camera3RequestDescriptor( @@ -43,3 +43,5 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( request_ = std::make_unique(camera, reinterpret_cast(this)); } + +Camera3RequestDescriptor::~Camera3RequestDescriptor() = default; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 1346f6fa..79dfdb58 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -10,6 +10,8 @@ #include #include +#include + #include #include @@ -18,18 +20,18 @@ #include "camera_metadata.h" #include "camera_worker.h" -struct Camera3RequestDescriptor { +class Camera3RequestDescriptor +{ +public: enum class Status { Pending, Success, Error, }; - Camera3RequestDescriptor() = default; - ~Camera3RequestDescriptor() = default; Camera3RequestDescriptor(libcamera::Camera *camera, const camera3_capture_request_t *camera3Request); - Camera3RequestDescriptor &operator=(Camera3RequestDescriptor &&) = default; + ~Camera3RequestDescriptor(); bool isPending() const { return status_ == Status::Pending; } @@ -41,6 +43,9 @@ struct Camera3RequestDescriptor { camera3_capture_result_t captureResult_ = {}; Status status_ = Status::Pending; + +private: + LIBCAMERA_DISABLE_COPY(Camera3RequestDescriptor) }; #endif /* __ANDROID_CAMERA_REQUEST_H__ */ From patchwork Mon Oct 18 13:29:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14164 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0112AC323E for ; Mon, 18 Oct 2021 13:29:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B753B68F58; Mon, 18 Oct 2021 15:29:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Bw6FHcsC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5945A68F5D for ; Mon, 18 Oct 2021 15:29:37 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 470F48C6; Mon, 18 Oct 2021 15:29:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563777; bh=fWstsiPj+w0dOPcWopYX900f/DDTQwtiBZbsnrUHnn4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bw6FHcsC5p0AhrwZWUlLLL00qMRjh4agxfckNmp5ZeSG4CF5gouCq7RG/yqgzu/8r d/uis13du0LpWfBrF0gYuEJ7bgBk4V4i/881W3eVLEZ3MMqcli0K3m6xjU/KTEOCgB 1rSU5Y4n/98jRCDgr+FxiXfQqhguwGyBHaA/lJU4= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:15 +0530 Message-Id: <20211018132923.476242-4-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 03/11] android: camera_device: Build capture_result dynamically X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart The camera3_capture_result_t is only needed to convey capture results to the camera service through the process_capture_result() callback. There's no need to store it in the Camera3RequestDescriptor. Build it dynamically in CameraDevice::sendCaptureResults() instead. This requires storing the result metadata created in CameraDevice::requestComplete() in the Camera3RequestDescriptor. A side effect of this change is that the request metadata lifetime will match the Camera3RequestDescriptor instead of being destroyed at the end of requestComplete(). This will be needed to support asynchronous post-processing, where the request completion will be signaled to the camera service asynchronously from requestComplete(). Signed-off-by: Laurent Pinchart Signed-off-by: Umang Jain Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 43 ++++++++++++++++------------------- src/android/camera_request.h | 2 +- 2 files changed, 21 insertions(+), 24 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index b4ab5da1..c6ae8930 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -830,19 +830,11 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const { notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); - camera3_capture_result_t &result = descriptor->captureResult_; - result.num_output_buffers = descriptor->buffers_.size(); - result.frame_number = descriptor->frameNumber_; - result.partial_result = 0; - - std::vector resultBuffers(result.num_output_buffers); - for (auto [i, buffer] : utils::enumerate(resultBuffers)) { - buffer = descriptor->buffers_[i]; - buffer.release_fence = descriptor->buffers_[i].acquire_fence; + for (auto &buffer : descriptor->buffers_) { + buffer.release_fence = buffer.acquire_fence; buffer.acquire_fence = -1; buffer.status = CAMERA3_BUFFER_STATUS_ERROR; } - result.output_buffers = resultBuffers.data(); descriptor->status_ = Camera3RequestDescriptor::Status::Error; } @@ -1090,9 +1082,6 @@ void CameraDevice::requestComplete(Request *request) * The buffer status is set to OK and later changed to ERROR if * post-processing/compression fails. */ - camera3_capture_result_t &captureResult = descriptor->captureResult_; - captureResult.frame_number = descriptor->frameNumber_; - captureResult.num_output_buffers = descriptor->buffers_.size(); for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { CameraStream *cameraStream = static_cast(buffer.stream->priv); @@ -1113,8 +1102,6 @@ void CameraDevice::requestComplete(Request *request) buffer.release_fence = -1; buffer.status = CAMERA3_BUFFER_STATUS_OK; } - captureResult.output_buffers = descriptor->buffers_.data(); - captureResult.partial_result = 1; /* * If the Request has failed, abort the request by notifying the error @@ -1128,7 +1115,6 @@ void CameraDevice::requestComplete(Request *request) notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); - captureResult.partial_result = 0; for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { /* * Signal to the framework it has to handle fences that @@ -1165,12 +1151,12 @@ void CameraDevice::requestComplete(Request *request) * Notify if the metadata generation has failed, but continue processing * buffers and return an empty metadata pack. */ - std::unique_ptr resultMetadata = getResultMetadata(*descriptor); - if (!resultMetadata) { + descriptor->resultMetadata_ = getResultMetadata(*descriptor); + if (!descriptor->resultMetadata_) { notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_RESULT); /* The camera framework expects an empty metadata pack on error. */ - resultMetadata = std::make_unique(0, 0); + descriptor->resultMetadata_ = std::make_unique(0, 0); } /* Handle post-processing. */ @@ -1192,7 +1178,7 @@ void CameraDevice::requestComplete(Request *request) int ret = cameraStream->process(*src, buffer, descriptor->settings_, - resultMetadata.get()); + descriptor->resultMetadata_.get()); /* * Return the FrameBuffer to the CameraStream now that we're * done processing it. @@ -1207,7 +1193,6 @@ void CameraDevice::requestComplete(Request *request) } } - captureResult.result = resultMetadata->get(); descriptor->status_ = Camera3RequestDescriptor::Status::Success; sendCaptureResults(); } @@ -1225,8 +1210,20 @@ void CameraDevice::sendCaptureResults() * impact on performance which should be measured. */ lock.unlock(); - callbacks_->process_capture_result(callbacks_, - &descriptor->captureResult_); + + camera3_capture_result_t captureResult = {}; + + captureResult.frame_number = descriptor->frameNumber_; + if (descriptor->resultMetadata_) + captureResult.result = descriptor->resultMetadata_->get(); + captureResult.num_output_buffers = descriptor->buffers_.size(); + captureResult.output_buffers = descriptor->buffers_.data(); + + if (descriptor->status_ == Camera3RequestDescriptor::Status::Success) + captureResult.partial_result = 1; + + callbacks_->process_capture_result(callbacks_, &captureResult); + lock.lock(); } } diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 79dfdb58..db13f624 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -40,8 +40,8 @@ public: std::vector> frameBuffers_; CameraMetadata settings_; std::unique_ptr request_; + std::unique_ptr resultMetadata_; - camera3_capture_result_t captureResult_ = {}; Status status_ = Status::Pending; private: From patchwork Mon Oct 18 13:29:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14165 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 701FBC324D for ; Mon, 18 Oct 2021 13:29:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C6BA68F63; Mon, 18 Oct 2021 15:29:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rSQtzkIM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 153F168F56 for ; Mon, 18 Oct 2021 15:29:39 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F3EF8C6; Mon, 18 Oct 2021 15:29:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563778; bh=vGYoy324HBSYP7H2PU+tdEcIsZJYuSj5ZKpFlHXAYDM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rSQtzkIMcNiQfcqwAy916Ke8u+p2HCa9WTCx4AvxxSdqtaaPrASm66drMNMS0iPBb 49CLZCtY2jYFEp3dS06UHa7P56MVZPDwvUcAwbBCkw4sr7/IhPhVK5lOLuMtS6S6f6 veRZvNFClmhuJWQjmNj41aQDUh+6rwxQdGeIN+bw= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:16 +0530 Message-Id: <20211018132923.476242-5-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 04/11] android: camera_stream: Plumb process() with Camera3RequestDescriptor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Data (or broader context) required for post processing of a camera request is saved via Camera3RequestDescriptor. Instead of passing individual arguments to CameraStream::process(), pass the Camera3RequestDescriptor pointer to it. All the arguments necessary to run the post-processor can be accessed from the descriptor. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 5 ++--- src/android/camera_stream.cpp | 5 ++--- src/android/camera_stream.h | 5 ++--- src/android/jpeg/post_processor_jpeg.cpp | 6 ++++-- src/android/jpeg/post_processor_jpeg.h | 3 +-- src/android/post_processor.h | 5 ++--- src/android/yuv/post_processor_yuv.cpp | 3 +-- src/android/yuv/post_processor_yuv.h | 3 +-- 8 files changed, 15 insertions(+), 20 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index c6ae8930..3bddb292 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1176,9 +1176,8 @@ void CameraDevice::requestComplete(Request *request) continue; } - int ret = cameraStream->process(*src, buffer, - descriptor->settings_, - descriptor->resultMetadata_.get()); + int ret = cameraStream->process(*src, buffer, descriptor); + /* * Return the FrameBuffer to the CameraStream now that we're * done processing it. diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 3b96d2e9..8f47e4d8 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -144,8 +144,7 @@ int CameraStream::waitFence(int fence) int CameraStream::process(const FrameBuffer &source, camera3_stream_buffer_t &camera3Dest, - const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) + Camera3RequestDescriptor *request) { /* Handle waiting on fences on the destination buffer. */ int fence = camera3Dest.acquire_fence; @@ -175,7 +174,7 @@ int CameraStream::process(const FrameBuffer &source, return -EINVAL; } - return postProcessor_->process(source, &dest, requestMetadata, resultMetadata); + return postProcessor_->process(source, &dest, request); } FrameBuffer *CameraStream::getBuffer() diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 03ecfa94..405b232d 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -19,8 +19,8 @@ #include #include +class Camera3RequestDescriptor; class CameraDevice; -class CameraMetadata; class PostProcessor; class CameraStream @@ -120,8 +120,7 @@ public: int configure(); int process(const libcamera::FrameBuffer &source, camera3_stream_buffer_t &camera3Buffer, - const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata); + Camera3RequestDescriptor *request); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index f6d47f63..699576ef 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -11,6 +11,7 @@ #include "../camera_device.h" #include "../camera_metadata.h" +#include "../camera_request.h" #include "encoder_libjpeg.h" #include "exif.h" @@ -99,14 +100,15 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, int PostProcessorJpeg::process(const FrameBuffer &source, CameraBuffer *destination, - const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) + Camera3RequestDescriptor *request) { if (!encoder_) return 0; ASSERT(destination->numPlanes() == 1); + const CameraMetadata &requestMetadata = request->settings_; + CameraMetadata *resultMetadata = request->resultMetadata_.get(); camera_metadata_ro_entry_t entry; int ret; diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index 6fd31022..0184d77e 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -24,8 +24,7 @@ public: const libcamera::StreamConfiguration &outcfg) override; int process(const libcamera::FrameBuffer &source, CameraBuffer *destination, - const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) override; + Camera3RequestDescriptor *request) override; private: void generateThumbnail(const libcamera::FrameBuffer &source, diff --git a/src/android/post_processor.h b/src/android/post_processor.h index ab2b2c60..27eaef88 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -12,7 +12,7 @@ #include "camera_buffer.h" -class CameraMetadata; +class Camera3RequestDescriptor; class PostProcessor { @@ -23,8 +23,7 @@ public: const libcamera::StreamConfiguration &outCfg) = 0; virtual int process(const libcamera::FrameBuffer &source, CameraBuffer *destination, - const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) = 0; + Camera3RequestDescriptor *request) = 0; }; #endif /* __ANDROID_POST_PROCESSOR_H__ */ diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 7b3b4960..8110a1f1 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -51,8 +51,7 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, int PostProcessorYuv::process(const FrameBuffer &source, CameraBuffer *destination, - [[maybe_unused]] const CameraMetadata &requestMetadata, - [[maybe_unused]] CameraMetadata *metadata) + [[maybe_unused]] Camera3RequestDescriptor *request) { if (!isValidBuffers(source, *destination)) return -EINVAL; diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h index 12f7af07..a4e0ff5d 100644 --- a/src/android/yuv/post_processor_yuv.h +++ b/src/android/yuv/post_processor_yuv.h @@ -20,8 +20,7 @@ public: const libcamera::StreamConfiguration &outcfg) override; int process(const libcamera::FrameBuffer &source, CameraBuffer *destination, - const CameraMetadata &requestMetadata, - CameraMetadata *metadata) override; + Camera3RequestDescriptor *request) override; private: bool isValidBuffers(const libcamera::FrameBuffer &source, From patchwork Mon Oct 18 13:29:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14166 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 207B9C323E for ; Mon, 18 Oct 2021 13:29:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D57B468F58; Mon, 18 Oct 2021 15:29:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JwMZqnVh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A71EA68F5E for ; Mon, 18 Oct 2021 15:29:40 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AF7418C6; Mon, 18 Oct 2021 15:29:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563780; bh=w/Z4OmnWPQDYNit8Zmqa+zbucJgMOBfdfcxeoI1atWU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JwMZqnVhK0wrBLv4QuWWTLiAMiiivEm7xwJlWP0rkSGFyESTSdMkS0/Uyz0or03Rv IrWTwgnCChu5bpR9NSaDAMEMoXAHpLSfxIEFbC0rxefz93s3kLZ5EysAB7M0eAT1eq HsaESrPYaPn0gwExxehHSV8Nt2Aex1xWscOSndAU= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:17 +0530 Message-Id: <20211018132923.476242-6-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 05/11] android: camera_device: Create struct to track per stream buffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Camera3RequestDescriptor structure stores, for each stream, the camera3_stream_buffer_t and the libcamera FrameBuffer in two separate vectors. This complicates buffer handling, as the code needs to keep both vectors in sync. Create a new structure to group all data about per-stream buffers to simplify this. As a side effect, we need to create a local vector of camera3_stream_buffer_t in CameraDevice::sendCaptureResults() as the camera3_stream_buffer_t instances stored in the new structure in Camera3RequestDescriptor are not contiguous anymore. This is a small price to pay for easier handling of buffers, and will be refactored in subsequent commits anyway. Signed-off-by: Umang Jain Signed-off-by: Laurent Pinchart Reviewed-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 75 ++++++++++++++++++---------------- src/android/camera_request.cpp | 9 +--- src/android/camera_request.h | 15 ++++++- 3 files changed, 55 insertions(+), 44 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 3bddb292..59557358 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -831,9 +831,9 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); for (auto &buffer : descriptor->buffers_) { - buffer.release_fence = buffer.acquire_fence; - buffer.acquire_fence = -1; - buffer.status = CAMERA3_BUFFER_STATUS_ERROR; + buffer.buffer.release_fence = buffer.buffer.acquire_fence; + buffer.buffer.acquire_fence = -1; + buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; } descriptor->status_ = Camera3RequestDescriptor::Status::Error; @@ -931,8 +931,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques LOG(HAL, Debug) << "Queueing request " << descriptor->request_->cookie() << " with " << descriptor->buffers_.size() << " streams"; - for (const auto &[i, camera3Buffer] : utils::enumerate(descriptor->buffers_)) { - camera3_stream *camera3Stream = camera3Buffer.stream; + for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { + camera3_stream *camera3Stream = buffer.buffer.stream; CameraStream *cameraStream = static_cast(camera3Stream->priv); std::stringstream ss; @@ -949,7 +949,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * while fences for streams of type Internal and Mapped are * handled at post-processing time. */ - FrameBuffer *buffer = nullptr; + FrameBuffer *frameBuffer = nullptr; int acquireFence = -1; switch (cameraStream->type()) { case CameraStream::Type::Mapped: @@ -967,13 +967,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * associate it with the Camera3RequestDescriptor for * lifetime management only. */ - descriptor->frameBuffers_.push_back( - createFrameBuffer(*camera3Buffer.buffer, + buffer.frameBuffer = + createFrameBuffer(*buffer.buffer.buffer, cameraStream->configuration().pixelFormat, - cameraStream->configuration().size)); - - buffer = descriptor->frameBuffers_.back().get(); - acquireFence = camera3Buffer.acquire_fence; + cameraStream->configuration().size); + frameBuffer = buffer.frameBuffer.get(); + acquireFence = buffer.buffer.acquire_fence; LOG(HAL, Debug) << ss.str() << " (direct)"; break; @@ -985,18 +984,18 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * The buffer has to be returned to the CameraStream * once it has been processed. */ - buffer = cameraStream->getBuffer(); + frameBuffer = cameraStream->getBuffer(); LOG(HAL, Debug) << ss.str() << " (internal)"; break; } - if (!buffer) { - LOG(HAL, Error) << "Failed to create buffer"; + if (!frameBuffer) { + LOG(HAL, Error) << "Failed to create frame buffer"; return -ENOMEM; } - descriptor->request_->addBuffer(cameraStream->stream(), buffer, - acquireFence); + descriptor->request_->addBuffer(cameraStream->stream(), + frameBuffer, acquireFence); } /* @@ -1082,9 +1081,9 @@ void CameraDevice::requestComplete(Request *request) * The buffer status is set to OK and later changed to ERROR if * post-processing/compression fails. */ - for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { + for (auto &buffer : descriptor->buffers_) { CameraStream *cameraStream = - static_cast(buffer.stream->priv); + static_cast(buffer.buffer.stream->priv); /* * Streams of type Direct have been queued to the @@ -1098,9 +1097,9 @@ void CameraDevice::requestComplete(Request *request) * fence to -1 once it has handled it and remove this check. */ if (cameraStream->type() == CameraStream::Type::Direct) - buffer.acquire_fence = -1; - buffer.release_fence = -1; - buffer.status = CAMERA3_BUFFER_STATUS_OK; + buffer.buffer.acquire_fence = -1; + buffer.buffer.release_fence = -1; + buffer.buffer.status = CAMERA3_BUFFER_STATUS_OK; } /* @@ -1115,15 +1114,15 @@ void CameraDevice::requestComplete(Request *request) notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); - for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { + for (auto &buffer : descriptor->buffers_) { /* * Signal to the framework it has to handle fences that * have not been waited on by setting the release fence * to the acquire fence value. */ - buffer.release_fence = buffer.acquire_fence; - buffer.acquire_fence = -1; - buffer.status = CAMERA3_BUFFER_STATUS_ERROR; + buffer.buffer.release_fence = buffer.buffer.acquire_fence; + buffer.buffer.acquire_fence = -1; + buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; } descriptor->status_ = Camera3RequestDescriptor::Status::Error; @@ -1160,9 +1159,9 @@ void CameraDevice::requestComplete(Request *request) } /* Handle post-processing. */ - for (camera3_stream_buffer_t &buffer : descriptor->buffers_) { + for (auto &buffer : descriptor->buffers_) { CameraStream *cameraStream = - static_cast(buffer.stream->priv); + static_cast(buffer.buffer.stream->priv); if (cameraStream->type() == CameraStream::Type::Direct) continue; @@ -1170,13 +1169,13 @@ void CameraDevice::requestComplete(Request *request) FrameBuffer *src = request->findBuffer(cameraStream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; - buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - notifyError(descriptor->frameNumber_, buffer.stream, + buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; + notifyError(descriptor->frameNumber_, buffer.buffer.stream, CAMERA3_MSG_ERROR_BUFFER); continue; } - int ret = cameraStream->process(*src, buffer, descriptor); + int ret = cameraStream->process(*src, buffer.buffer, descriptor); /* * Return the FrameBuffer to the CameraStream now that we're @@ -1186,8 +1185,8 @@ void CameraDevice::requestComplete(Request *request) cameraStream->putBuffer(src); if (ret) { - buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - notifyError(descriptor->frameNumber_, buffer.stream, + buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; + notifyError(descriptor->frameNumber_, buffer.buffer.stream, CAMERA3_MSG_ERROR_BUFFER); } } @@ -1213,10 +1212,16 @@ void CameraDevice::sendCaptureResults() camera3_capture_result_t captureResult = {}; captureResult.frame_number = descriptor->frameNumber_; + if (descriptor->resultMetadata_) captureResult.result = descriptor->resultMetadata_->get(); - captureResult.num_output_buffers = descriptor->buffers_.size(); - captureResult.output_buffers = descriptor->buffers_.data(); + + std::vector resultBuffers; + for (const auto &buffer : descriptor->buffers_) + resultBuffers.emplace_back(buffer.buffer); + + captureResult.num_output_buffers = resultBuffers.size(); + captureResult.output_buffers = resultBuffers.data(); if (descriptor->status_ == Camera3RequestDescriptor::Status::Success) captureResult.partial_result = 1; diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 16a632b3..614baed4 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -23,15 +23,10 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( /* Copy the camera3 request stream information for later access. */ const uint32_t numBuffers = camera3Request->num_output_buffers; + buffers_.resize(numBuffers); for (uint32_t i = 0; i < numBuffers; i++) - buffers_[i] = camera3Request->output_buffers[i]; - - /* - * FrameBuffer instances created by wrapping a camera3 provided dmabuf - * are emplaced in this vector of unique_ptr<> for lifetime management. - */ - frameBuffers_.reserve(numBuffers); + buffers_[i].buffer = camera3Request->output_buffers[i]; /* Clone the controls associated with the camera3 request. */ settings_ = CameraMetadata(camera3Request->settings); diff --git a/src/android/camera_request.h b/src/android/camera_request.h index db13f624..a030febf 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -29,6 +29,16 @@ public: Error, }; + struct StreamBuffer { + camera3_stream_buffer_t buffer; + /* + * FrameBuffer instances created by wrapping a camera3 provided + * dmabuf are emplaced in this vector of unique_ptr<> for + * lifetime management. + */ + std::unique_ptr frameBuffer; + }; + Camera3RequestDescriptor(libcamera::Camera *camera, const camera3_capture_request_t *camera3Request); ~Camera3RequestDescriptor(); @@ -36,8 +46,9 @@ public: bool isPending() const { return status_ == Status::Pending; } uint32_t frameNumber_ = 0; - std::vector buffers_; - std::vector> frameBuffers_; + + std::vector buffers_; + CameraMetadata settings_; std::unique_ptr request_; std::unique_ptr resultMetadata_; From patchwork Mon Oct 18 13:29:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14167 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 716EAC323E for ; Mon, 18 Oct 2021 13:29:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 01D0868F5D; Mon, 18 Oct 2021 15:29:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HwQQVNOp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1601468F58 for ; Mon, 18 Oct 2021 15:29:42 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4251E8C6; Mon, 18 Oct 2021 15:29:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563781; bh=YTN1gKdGQJFrFnUiorDerDOX92fgJl3k9kBXQqPCLiU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HwQQVNOpPn0iSZfhCJYZve6AgkPiFuhKEEy0YS81J2vf5HH7awqeFoD/mKBlPlR9U Iy8ubGaPUSbWDBfb07DYejY0RvCgOZnrRNCAuFkS9cRAEp/wjUwOgLJy/sYNLhHojw MxEPp7SyoSBQjKTNoIECPp9atx2g5tRkXMFKP/ow= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:18 +0530 Message-Id: <20211018132923.476242-7-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 06/11] android: camera_stream: Pass StreamBuffer to process() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Now that we have a proper structure to model a stream buffer, pass it to CameraStream::process() instead of the camera3_stream_buffer_t. This will allow accessing other members of StreamBuffer in subsequent commits. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 2 +- src/android/camera_stream.cpp | 14 +++++++------- src/android/camera_stream.h | 5 +++-- 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 59557358..cd9e9fe2 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1175,7 +1175,7 @@ void CameraDevice::requestComplete(Request *request) continue; } - int ret = cameraStream->process(*src, buffer.buffer, descriptor); + int ret = cameraStream->process(*src, buffer, descriptor); /* * Return the FrameBuffer to the CameraStream now that we're diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 8f47e4d8..f3cc77e7 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -143,15 +143,15 @@ int CameraStream::waitFence(int fence) } int CameraStream::process(const FrameBuffer &source, - camera3_stream_buffer_t &camera3Dest, + Camera3RequestDescriptor::StreamBuffer &dest, Camera3RequestDescriptor *request) { /* Handle waiting on fences on the destination buffer. */ - int fence = camera3Dest.acquire_fence; + int fence = dest.buffer.acquire_fence; if (fence != -1) { int ret = waitFence(fence); ::close(fence); - camera3Dest.acquire_fence = -1; + dest.buffer.acquire_fence = -1; if (ret < 0) { LOG(HAL, Error) << "Failed waiting for fence: " << fence << ": " << strerror(-ret); @@ -167,14 +167,14 @@ int CameraStream::process(const FrameBuffer &source, * separate thread. */ const StreamConfiguration &output = configuration(); - CameraBuffer dest(*camera3Dest.buffer, output.pixelFormat, output.size, - PROT_READ | PROT_WRITE); - if (!dest.isValid()) { + CameraBuffer destBuffer(*dest.buffer.buffer, output.pixelFormat, + output.size, PROT_READ | PROT_WRITE); + if (!destBuffer.isValid()) { LOG(HAL, Error) << "Failed to create destination buffer"; return -EINVAL; } - return postProcessor_->process(source, &dest, request); + return postProcessor_->process(source, &destBuffer, request); } FrameBuffer *CameraStream::getBuffer() diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 405b232d..197bd995 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -19,7 +19,8 @@ #include #include -class Camera3RequestDescriptor; +#include "camera_request.h" + class CameraDevice; class PostProcessor; @@ -119,7 +120,7 @@ public: int configure(); int process(const libcamera::FrameBuffer &source, - camera3_stream_buffer_t &camera3Buffer, + Camera3RequestDescriptor::StreamBuffer &dest, Camera3RequestDescriptor *request); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); From patchwork Mon Oct 18 13:29:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14168 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1DA18C323E for ; Mon, 18 Oct 2021 13:29:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CAD2C68F65; Mon, 18 Oct 2021 15:29:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FC7xpu4t"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BBC968F66 for ; Mon, 18 Oct 2021 15:29:43 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AE7C91911; Mon, 18 Oct 2021 15:29:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563783; bh=hfjpvSQqCs//72iNuuLxUjRm9PALFMcpyT8ckGvEmGM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FC7xpu4twwHcE5QI7xE+XDWAa8iq48JsccqXztoWZgFaMEcQ3NgG/21KGbRVvzvRf IF+Fpd807AXykjwyDy4ztoUMwd5GwSC8JeGJMgTmcpi7KtA1NhgAQ+lAYwFzRDGE96 tGEdsRbijErn7mjF4n1HFzFIFsukJ0y+yD6YlNAs= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:19 +0530 Message-Id: <20211018132923.476242-8-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 07/11] android: camera_stream: Return non-const pointer from camera3Stream() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart The camera3_stream_t instances are used to interact with the camera service, whose API uses non-const pointers. Replace the const reference returned by CameraStream::camera3Stream() with a non-const pointer. It turns out that nobody calls this function, but new users will be introduced in subsequent commits. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/android/camera_stream.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 197bd995..85064268 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -114,7 +114,7 @@ public: camera3_stream_t *camera3Stream, unsigned int index); Type type() const { return type_; } - const camera3_stream_t &camera3Stream() const { return *camera3Stream_; } + camera3_stream_t *camera3Stream() const { return camera3Stream_; } const libcamera::StreamConfiguration &configuration() const; libcamera::Stream *stream() const; From patchwork Mon Oct 18 13:29:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14169 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AE3A5C323E for ; Mon, 18 Oct 2021 13:29:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4837D68F5D; Mon, 18 Oct 2021 15:29:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="viEW46Lr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56EF568F58 for ; Mon, 18 Oct 2021 15:29:45 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6A6E11C15; Mon, 18 Oct 2021 15:29:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563785; bh=AcD+9yLKYUS53pk5mqhL69DvZzcXSQtmyGNQIKFQlDA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=viEW46Lr7hnHtakgBlAUrzFYeWr2K/fZW0O8fIxCIsz5KzrI+edt2SlAwKj6NiPWD crZPX1gICagrih04ZnYC7QskDNytxSjruARKRfuzjEU8PBPHyMpXSBcfyKTtWBIaVV S+8wuo1mquXAekZQ/ZWxft0R8Vb1fk5iuWpT9AkM= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:20 +0530 Message-Id: <20211018132923.476242-9-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 08/11] android: camera_device: Use abortRequest() instead of open-coding it X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Call abortRequest() in CameraDevice::requestComplete() instead of open-coding it. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Umang Jain --- src/android/camera_device.cpp | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index cd9e9fe2..216f29c2 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -831,6 +831,11 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); for (auto &buffer : descriptor->buffers_) { + /* + * Signal to the framework it has to handle fences that have not + * been waited on by setting the release fence to the acquire + * fence value. + */ buffer.buffer.release_fence = buffer.buffer.acquire_fence; buffer.buffer.acquire_fence = -1; buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; @@ -1111,21 +1116,7 @@ void CameraDevice::requestComplete(Request *request) << " not successfully completed: " << request->status(); - notifyError(descriptor->frameNumber_, nullptr, - CAMERA3_MSG_ERROR_REQUEST); - - for (auto &buffer : descriptor->buffers_) { - /* - * Signal to the framework it has to handle fences that - * have not been waited on by setting the release fence - * to the acquire fence value. - */ - buffer.buffer.release_fence = buffer.buffer.acquire_fence; - buffer.buffer.acquire_fence = -1; - buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - } - - descriptor->status_ = Camera3RequestDescriptor::Status::Error; + abortRequest(descriptor); sendCaptureResults(); return; From patchwork Mon Oct 18 13:29:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14170 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4694EC323E for ; Mon, 18 Oct 2021 13:29:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D975168F65; Mon, 18 Oct 2021 15:29:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Li3eHyU5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 15C0C68F57 for ; Mon, 18 Oct 2021 15:29:47 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 11AA58C6; Mon, 18 Oct 2021 15:29:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563786; bh=Hp1fGxT0zaO3TNOTIat4ybJaDsInBIYK3NZvP4dcObo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Li3eHyU5XEJpov57wriPeEdvHerJAwrXOZhIEjkUJV6UBFFkwn5vC7XdVM4LRIayj N4JMIreuDtv3iB1MCyBfZB4PsyhccXdsq4bxP4sn/DExxp9Ro7s17EIWGMCQ3BoG3/ nrfMVFphTSYuyZITHKKHiJfg/SzH00DlIb/iUrlM= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:21 +0530 Message-Id: <20211018132923.476242-10-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 09/11] android: camera_request: Don't embed full camera3_stream_buffer_t X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart The camera3_stream_buffer_t structure is meant to communicate between the camera service and the HAL. They are short-live structures that don't outlive the .process_capture_request() operation (when queuing requests) or the .process_capture_result() callback. We currently store copies of the camera3_stream_buffer_t passed to .process_capture_request() in Camera3RequestDescriptor::StreamBuffer to store the structure members that the HAL need, and reuse them when calling the .process_capture_result() callback. This is conceptually not right, as the camera3_stream_buffer_t pass to the callback are not the same objects as the ones received in .process_capture_request(). Store individual fields of the camera3_stream_buffer_t in StreamBuffer instead of copying the whole structure. This gives the HAL full control of how data is stored, and properly decouples request queueing from result reporting. Signed-off-by: Laurent Pinchart Signed-off-by: Umang Jain Reviewed-by: Umang Jain --- src/android/camera_device.cpp | 73 ++++++++++++++++++---------------- src/android/camera_request.cpp | 19 +++++++-- src/android/camera_request.h | 12 +++--- src/android/camera_stream.cpp | 6 +-- 4 files changed, 63 insertions(+), 47 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 216f29c2..c493b0a4 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -830,16 +830,8 @@ void CameraDevice::abortRequest(Camera3RequestDescriptor *descriptor) const { notifyError(descriptor->frameNumber_, nullptr, CAMERA3_MSG_ERROR_REQUEST); - for (auto &buffer : descriptor->buffers_) { - /* - * Signal to the framework it has to handle fences that have not - * been waited on by setting the release fence to the acquire - * fence value. - */ - buffer.buffer.release_fence = buffer.buffer.acquire_fence; - buffer.buffer.acquire_fence = -1; - buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - } + for (auto &buffer : descriptor->buffers_) + buffer.status = Camera3RequestDescriptor::Status::Error; descriptor->status_ = Camera3RequestDescriptor::Status::Error; } @@ -937,8 +929,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques << " with " << descriptor->buffers_.size() << " streams"; for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) { - camera3_stream *camera3Stream = buffer.buffer.stream; - CameraStream *cameraStream = static_cast(camera3Stream->priv); + CameraStream *cameraStream = buffer.stream; + camera3_stream_t *camera3Stream = cameraStream->camera3Stream(); std::stringstream ss; ss << i << " - (" << camera3Stream->width << "x" @@ -973,11 +965,11 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * lifetime management only. */ buffer.frameBuffer = - createFrameBuffer(*buffer.buffer.buffer, + createFrameBuffer(*buffer.camera3Buffer, cameraStream->configuration().pixelFormat, cameraStream->configuration().size); frameBuffer = buffer.frameBuffer.get(); - acquireFence = buffer.buffer.acquire_fence; + acquireFence = buffer.fence; LOG(HAL, Debug) << ss.str() << " (direct)"; break; @@ -1083,12 +1075,11 @@ void CameraDevice::requestComplete(Request *request) /* * Prepare the capture result for the Android camera stack. * - * The buffer status is set to OK and later changed to ERROR if + * The buffer status is set to Success and later changed to Error if * post-processing/compression fails. */ for (auto &buffer : descriptor->buffers_) { - CameraStream *cameraStream = - static_cast(buffer.buffer.stream->priv); + CameraStream *stream = buffer.stream; /* * Streams of type Direct have been queued to the @@ -1101,10 +1092,9 @@ void CameraDevice::requestComplete(Request *request) * \todo Instrument the CameraWorker to set the acquire * fence to -1 once it has handled it and remove this check. */ - if (cameraStream->type() == CameraStream::Type::Direct) - buffer.buffer.acquire_fence = -1; - buffer.buffer.release_fence = -1; - buffer.buffer.status = CAMERA3_BUFFER_STATUS_OK; + if (stream->type() == CameraStream::Type::Direct) + buffer.fence = -1; + buffer.status = Camera3RequestDescriptor::Status::Success; } /* @@ -1151,33 +1141,32 @@ void CameraDevice::requestComplete(Request *request) /* Handle post-processing. */ for (auto &buffer : descriptor->buffers_) { - CameraStream *cameraStream = - static_cast(buffer.buffer.stream->priv); + CameraStream *stream = buffer.stream; - if (cameraStream->type() == CameraStream::Type::Direct) + if (stream->type() == CameraStream::Type::Direct) continue; - FrameBuffer *src = request->findBuffer(cameraStream->stream()); + FrameBuffer *src = request->findBuffer(stream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; - buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - notifyError(descriptor->frameNumber_, buffer.buffer.stream, + buffer.status = Camera3RequestDescriptor::Status::Error; + notifyError(descriptor->frameNumber_, stream->camera3Stream(), CAMERA3_MSG_ERROR_BUFFER); continue; } - int ret = cameraStream->process(*src, buffer, descriptor); + int ret = stream->process(*src, buffer, descriptor); /* * Return the FrameBuffer to the CameraStream now that we're * done processing it. */ - if (cameraStream->type() == CameraStream::Type::Internal) - cameraStream->putBuffer(src); + if (stream->type() == CameraStream::Type::Internal) + stream->putBuffer(src); if (ret) { - buffer.buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - notifyError(descriptor->frameNumber_, buffer.buffer.stream, + buffer.status = Camera3RequestDescriptor::Status::Error; + notifyError(descriptor->frameNumber_, stream->camera3Stream(), CAMERA3_MSG_ERROR_BUFFER); } } @@ -1208,8 +1197,24 @@ void CameraDevice::sendCaptureResults() captureResult.result = descriptor->resultMetadata_->get(); std::vector resultBuffers; - for (const auto &buffer : descriptor->buffers_) - resultBuffers.emplace_back(buffer.buffer); + resultBuffers.reserve(descriptor->buffers_.size()); + + for (const auto &buffer : descriptor->buffers_) { + camera3_buffer_status status = CAMERA3_BUFFER_STATUS_ERROR; + + if (buffer.status == Camera3RequestDescriptor::Status::Success) + status = CAMERA3_BUFFER_STATUS_OK; + + /* + * Pass the buffer fence back to the camera framework as + * a release fence. This instructs the framework to wait + * on the acquire fence in case we haven't done so + * ourselves for any reason. + */ + resultBuffers.push_back({ buffer.stream->camera3Stream(), + buffer.camera3Buffer, status, + -1, buffer.fence }); + } captureResult.num_output_buffers = resultBuffers.size(); captureResult.output_buffers = resultBuffers.data(); diff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp index 614baed4..faa85ada 100644 --- a/src/android/camera_request.cpp +++ b/src/android/camera_request.cpp @@ -7,6 +7,8 @@ #include "camera_request.h" +#include + using namespace libcamera; /* @@ -22,11 +24,20 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( frameNumber_ = camera3Request->frame_number; /* Copy the camera3 request stream information for later access. */ - const uint32_t numBuffers = camera3Request->num_output_buffers; + const Span buffers{ + camera3Request->output_buffers, + camera3Request->num_output_buffers + }; + + buffers_.reserve(buffers.size()); + + for (const camera3_stream_buffer_t &buffer : buffers) { + CameraStream *stream = + static_cast(buffer.stream->priv); - buffers_.resize(numBuffers); - for (uint32_t i = 0; i < numBuffers; i++) - buffers_[i].buffer = camera3Request->output_buffers[i]; + buffers_.push_back({ stream, buffer.buffer, nullptr, + buffer.acquire_fence, Status::Pending }); + } /* Clone the controls associated with the camera3 request. */ settings_ = CameraMetadata(camera3Request->settings); diff --git a/src/android/camera_request.h b/src/android/camera_request.h index a030febf..05dabf89 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -20,6 +20,8 @@ #include "camera_metadata.h" #include "camera_worker.h" +class CameraStream; + class Camera3RequestDescriptor { public: @@ -30,13 +32,11 @@ public: }; struct StreamBuffer { - camera3_stream_buffer_t buffer; - /* - * FrameBuffer instances created by wrapping a camera3 provided - * dmabuf are emplaced in this vector of unique_ptr<> for - * lifetime management. - */ + CameraStream *stream; + buffer_handle_t *camera3Buffer; std::unique_ptr frameBuffer; + int fence; + Status status; }; Camera3RequestDescriptor(libcamera::Camera *camera, diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index f3cc77e7..9b5cd0c4 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -147,11 +147,11 @@ int CameraStream::process(const FrameBuffer &source, Camera3RequestDescriptor *request) { /* Handle waiting on fences on the destination buffer. */ - int fence = dest.buffer.acquire_fence; + int fence = dest.fence; if (fence != -1) { int ret = waitFence(fence); ::close(fence); - dest.buffer.acquire_fence = -1; + dest.fence = -1; if (ret < 0) { LOG(HAL, Error) << "Failed waiting for fence: " << fence << ": " << strerror(-ret); @@ -167,7 +167,7 @@ int CameraStream::process(const FrameBuffer &source, * separate thread. */ const StreamConfiguration &output = configuration(); - CameraBuffer destBuffer(*dest.buffer.buffer, output.pixelFormat, + CameraBuffer destBuffer(*dest.camera3Buffer, output.pixelFormat, output.size, PROT_READ | PROT_WRITE); if (!destBuffer.isValid()) { LOG(HAL, Error) << "Failed to create destination buffer"; From patchwork Mon Oct 18 13:29:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14171 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ACA70C323E for ; Mon, 18 Oct 2021 13:29:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6B0F268F64; Mon, 18 Oct 2021 15:29:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AOsD7tvU"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BDDC768F60 for ; Mon, 18 Oct 2021 15:29:48 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D01A28C6; Mon, 18 Oct 2021 15:29:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563788; bh=Pna2DOMPVmcbBo9n3dA7+M7H4FWppL8o3I67I00q8y0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AOsD7tvUJmuuiu3ExM8hGEpukl0OtUObX2d5UBLCkdiScWTPv9gWqOCVqYh5R+YYu 5jgC+3EbkJtJOXh3J3XEV6l0D0FCzb43J9DiNR1x12QGMgN1SuIRw+SgPY97mfKb9B hV/fJQxyaC8NySsIy6I09dBitdTVxNYZj5mT85oI= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:22 +0530 Message-Id: <20211018132923.476242-11-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 10/11] android: camera_stream: Don't close fence if wait times out X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart The camera HAL APIs requires that any acquire fence that hasn't been waited on to be sent back to the framework as a release fence. The CameraDevice already copies the acquire fence to the release fence when signaling request completion, but the CameraStream incorrectly closes the fence when a wait times out and sets it to -1. Fix it. Signed-off-by: Laurent Pinchart Reviewed-by: Umang Jain --- src/android/camera_stream.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 9b5cd0c4..8e6ccb83 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -147,16 +147,16 @@ int CameraStream::process(const FrameBuffer &source, Camera3RequestDescriptor *request) { /* Handle waiting on fences on the destination buffer. */ - int fence = dest.fence; - if (fence != -1) { - int ret = waitFence(fence); - ::close(fence); - dest.fence = -1; + if (dest.fence != -1) { + int ret = waitFence(dest.fence); if (ret < 0) { LOG(HAL, Error) << "Failed waiting for fence: " - << fence << ": " << strerror(-ret); + << dest.fence << ": " << strerror(-ret); return ret; } + + ::close(dest.fence); + dest.fence = -1; } if (!postProcessor_) From patchwork Mon Oct 18 13:29:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 14172 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 04AF3C323E for ; Mon, 18 Oct 2021 13:29:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C6DB768F65; Mon, 18 Oct 2021 15:29:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="N0gGLi94"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D0C068F68 for ; Mon, 18 Oct 2021 15:29:50 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.238.109.14]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6ED5D18F1; Mon, 18 Oct 2021 15:29:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1634563790; bh=p0YC5dtjjwIk7h7mOycYhhAoFD5Yd9Q0JRAT4rl0R7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N0gGLi94A8aNC8liNDP5oPHq5EZ8ZEs5Ee+B+bhbxd5qqbl4v27IViTesZrCRXxL4 tCgP3OzoirVbJc79tkeYDAvKODypON/rR8mFDSbNI40hc1G+J5NgzLbR3gFt1OwhFc f0Flxw2a2lfIQ4U3M/d3xqIMjmmRNunq3xmERKfs= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Oct 2021 18:59:23 +0530 Message-Id: <20211018132923.476242-12-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211018132923.476242-1-umang.jain@ideasonboard.com> References: <20211018132923.476242-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 11/11] android: camera_stream: Define explicit move constructor and destructors X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart There's no need for the move constructor and the destructor to be inline. Define them explicitly, with default implementations. This allows usage of the CameraStream class without a complete definition of the PostProcessor class. Signed-off-by: Laurent Pinchart Reviewed-by: Jacopo Mondi Reviewed-by: Umang Jain --- src/android/camera_stream.cpp | 4 ++++ src/android/camera_stream.h | 2 ++ 2 files changed, 6 insertions(+) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 8e6ccb83..f44a2717 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -56,6 +56,10 @@ CameraStream::CameraStream(CameraDevice *const cameraDevice, { } +CameraStream::CameraStream(CameraStream &&other) = default; + +CameraStream::~CameraStream() = default; + const StreamConfiguration &CameraStream::configuration() const { return config_->at(index_); diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 85064268..f242336e 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -112,6 +112,8 @@ public: CameraStream(CameraDevice *const cameraDevice, libcamera::CameraConfiguration *config, Type type, camera3_stream_t *camera3Stream, unsigned int index); + CameraStream(CameraStream &&other); + ~CameraStream(); Type type() const { return type_; } camera3_stream_t *camera3Stream() const { return camera3Stream_; }