From patchwork Fri Sep 10 07:06:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13799 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0BAF7BDC71 for ; Fri, 10 Sep 2021 07:06:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B489369171; Fri, 10 Sep 2021 09:06:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="F75aT11e"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 00A1D69169 for ; Fri, 10 Sep 2021 09:06:49 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CF552883; Fri, 10 Sep 2021 09:06:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257608; bh=2IFpf4dHn+uD7uZClcdMZY0XdUC1HjfReOWrd+vTWx0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F75aT11eRT6lQllgmTuJFnrmTPl7BvChEMBZiT/ifwRNqT79QO5YGjVuXVE5p3PZF gQY+x0Xyz6n/sVYgzt6q899Mg0//eRCF5HeCvPygnRRUu3gXwkqmjh179Wz9zFFu5s /uGHDAq1ELarcVd48OmZ6UB5H1NhA4LphSdbw0ss= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:30 +0530 Message-Id: <20210910070638.467294-2-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/9] android: camera_stream: Pass FrameBuffer pointer instead of reference X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass the libcamera::FrameBuffer pointer to the post-processor instead of passing it by reference. Passing by reference is fine as long as the post processing is done synchronously. However in subsequent commits, the post processing is planned to be moved to a separate thread. This will conflict as the reference argument (in current case 'source') is copied when we will try to invoke a method on separate thread(which will run the post-processor) using Object::invokeMethod(). The cause of conflict is the LIBCAMERA_DISABLE_COPY_AND_MOVE rule applied on the FrameBuffer class. To resolve the conflict, pass in the FrameBuffer pointer instead of reference. This requires changes to the existing PostProcessor interface and all its implemented classes. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_device.cpp | 2 +- src/android/camera_stream.cpp | 2 +- src/android/camera_stream.h | 2 +- src/android/jpeg/encoder.h | 2 +- src/android/jpeg/encoder_libjpeg.cpp | 4 ++-- src/android/jpeg/encoder_libjpeg.h | 2 +- src/android/jpeg/post_processor_jpeg.cpp | 4 ++-- src/android/jpeg/post_processor_jpeg.h | 4 ++-- src/android/jpeg/thumbnailer.cpp | 4 ++-- src/android/jpeg/thumbnailer.h | 2 +- src/android/post_processor.h | 2 +- src/android/yuv/post_processor_yuv.cpp | 18 +++++++++--------- src/android/yuv/post_processor_yuv.h | 4 ++-- 13 files changed, 26 insertions(+), 26 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index ab31bdda..f2f36f32 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1148,7 +1148,7 @@ void CameraDevice::requestComplete(Request *request) continue; } - int ret = cameraStream->process(*src, *buffer.buffer, + int ret = cameraStream->process(src, *buffer.buffer, descriptor.settings_, resultMetadata.get()); /* diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 0f5ae947..0fed5382 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -98,7 +98,7 @@ int CameraStream::configure() return 0; } -int CameraStream::process(const FrameBuffer &source, +int CameraStream::process(const FrameBuffer *source, buffer_handle_t camera3Dest, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 2dab6c3a..5c232cb6 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -118,7 +118,7 @@ public: libcamera::Stream *stream() const; int configure(); - int process(const libcamera::FrameBuffer &source, + int process(const libcamera::FrameBuffer *source, buffer_handle_t camera3Dest, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata); diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h index a28522f4..7b6140e7 100644 --- a/src/android/jpeg/encoder.h +++ b/src/android/jpeg/encoder.h @@ -18,7 +18,7 @@ public: virtual ~Encoder() = default; virtual int configure(const libcamera::StreamConfiguration &cfg) = 0; - virtual int encode(const libcamera::FrameBuffer &source, + virtual int encode(const libcamera::FrameBuffer *source, libcamera::Span destination, libcamera::Span exifData, unsigned int quality) = 0; diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp index 21a3b33d..3114ed4b 100644 --- a/src/android/jpeg/encoder_libjpeg.cpp +++ b/src/android/jpeg/encoder_libjpeg.cpp @@ -178,10 +178,10 @@ void EncoderLibJpeg::compressNV(const std::vector> &planes) } } -int EncoderLibJpeg::encode(const FrameBuffer &source, Span dest, +int EncoderLibJpeg::encode(const FrameBuffer *source, Span dest, Span exifData, unsigned int quality) { - MappedFrameBuffer frame(&source, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer frame(source, MappedFrameBuffer::MapFlag::Read); if (!frame.isValid()) { LOG(JPEG, Error) << "Failed to map FrameBuffer : " << strerror(frame.error()); diff --git a/src/android/jpeg/encoder_libjpeg.h b/src/android/jpeg/encoder_libjpeg.h index 45ffbd7f..ae4e1e32 100644 --- a/src/android/jpeg/encoder_libjpeg.h +++ b/src/android/jpeg/encoder_libjpeg.h @@ -22,7 +22,7 @@ public: ~EncoderLibJpeg(); int configure(const libcamera::StreamConfiguration &cfg) override; - int encode(const libcamera::FrameBuffer &source, + int encode(const libcamera::FrameBuffer *source, libcamera::Span destination, libcamera::Span exifData, unsigned int quality) override; diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index ef2d98cc..cb45f86b 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -50,7 +50,7 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg, return encoder_->configure(inCfg); } -void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, +void PostProcessorJpeg::generateThumbnail(const FrameBuffer *source, const Size &targetSize, unsigned int quality, std::vector *thumbnail) @@ -97,7 +97,7 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, } } -int PostProcessorJpeg::process(const FrameBuffer &source, +int PostProcessorJpeg::process(const FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index 6fd31022..c4b2e9ef 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -22,13 +22,13 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; - int process(const libcamera::FrameBuffer &source, + int process(const libcamera::FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) override; private: - void generateThumbnail(const libcamera::FrameBuffer &source, + void generateThumbnail(const libcamera::FrameBuffer *source, const libcamera::Size &targetSize, unsigned int quality, std::vector *thumbnail); diff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp index 1fab8072..ffac6a15 100644 --- a/src/android/jpeg/thumbnailer.cpp +++ b/src/android/jpeg/thumbnailer.cpp @@ -37,11 +37,11 @@ void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat) valid_ = true; } -void Thumbnailer::createThumbnail(const FrameBuffer &source, +void Thumbnailer::createThumbnail(const FrameBuffer *source, const Size &targetSize, std::vector *destination) { - MappedFrameBuffer frame(&source, MappedFrameBuffer::MapFlag::Read); + MappedFrameBuffer frame(source, MappedFrameBuffer::MapFlag::Read); if (!frame.isValid()) { LOG(Thumbnailer, Error) << "Failed to map FrameBuffer : " diff --git a/src/android/jpeg/thumbnailer.h b/src/android/jpeg/thumbnailer.h index 4d086c49..0f3caf40 100644 --- a/src/android/jpeg/thumbnailer.h +++ b/src/android/jpeg/thumbnailer.h @@ -19,7 +19,7 @@ public: void configure(const libcamera::Size &sourceSize, libcamera::PixelFormat pixelFormat); - void createThumbnail(const libcamera::FrameBuffer &source, + void createThumbnail(const libcamera::FrameBuffer *source, const libcamera::Size &targetSize, std::vector *dest); const libcamera::PixelFormat &pixelFormat() const { return pixelFormat_; } diff --git a/src/android/post_processor.h b/src/android/post_processor.h index ab2b2c60..61dfb6d4 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -21,7 +21,7 @@ public: virtual int configure(const libcamera::StreamConfiguration &inCfg, const libcamera::StreamConfiguration &outCfg) = 0; - virtual int process(const libcamera::FrameBuffer &source, + virtual int process(const libcamera::FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata) = 0; diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 7b3b4960..0a874886 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -49,7 +49,7 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, return 0; } -int PostProcessorYuv::process(const FrameBuffer &source, +int PostProcessorYuv::process(const FrameBuffer *source, CameraBuffer *destination, [[maybe_unused]] const CameraMetadata &requestMetadata, [[maybe_unused]] CameraMetadata *metadata) @@ -57,7 +57,7 @@ int PostProcessorYuv::process(const FrameBuffer &source, if (!isValidBuffers(source, *destination)) return -EINVAL; - const MappedFrameBuffer sourceMapped(&source, MappedFrameBuffer::MapFlag::Read); + const MappedFrameBuffer sourceMapped(source, MappedFrameBuffer::MapFlag::Read); if (!sourceMapped.isValid()) { LOG(YUV, Error) << "Failed to mmap camera frame buffer"; return -EINVAL; @@ -83,12 +83,12 @@ int PostProcessorYuv::process(const FrameBuffer &source, return 0; } -bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source, +bool PostProcessorYuv::isValidBuffers(const FrameBuffer *source, const CameraBuffer &destination) const { - if (source.planes().size() != 2) { + if (source->planes().size() != 2) { LOG(YUV, Error) << "Invalid number of source planes: " - << source.planes().size(); + << source->planes().size(); return false; } if (destination.numPlanes() != 2) { @@ -97,12 +97,12 @@ bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source, return false; } - if (source.planes()[0].length < sourceLength_[0] || - source.planes()[1].length < sourceLength_[1]) { + if (source->planes()[0].length < sourceLength_[0] || + source->planes()[1].length < sourceLength_[1]) { LOG(YUV, Error) << "The source planes lengths are too small, actual size: {" - << source.planes()[0].length << ", " - << source.planes()[1].length + << source->planes()[0].length << ", " + << source->planes()[1].length << "}, expected size: {" << sourceLength_[0] << ", " << sourceLength_[1] << "}"; diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h index f8b1ba23..44a04113 100644 --- a/src/android/yuv/post_processor_yuv.h +++ b/src/android/yuv/post_processor_yuv.h @@ -20,13 +20,13 @@ public: int configure(const libcamera::StreamConfiguration &incfg, const libcamera::StreamConfiguration &outcfg) override; - int process(const libcamera::FrameBuffer &source, + int process(const libcamera::FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, CameraMetadata *metadata) override; private: - bool isValidBuffers(const libcamera::FrameBuffer &source, + bool isValidBuffers(const libcamera::FrameBuffer *source, const CameraBuffer &destination) const; void calculateLengths(const libcamera::StreamConfiguration &inCfg, const libcamera::StreamConfiguration &outCfg); From patchwork Fri Sep 10 07:06:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13800 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 927B9BDC71 for ; Fri, 10 Sep 2021 07:06:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E95E69177; Fri, 10 Sep 2021 09:06:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FPVrgvw2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 17B6869172 for ; Fri, 10 Sep 2021 09:06:51 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD058883; Fri, 10 Sep 2021 09:06:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257610; bh=bOc6AVlQgLXHqvMXPCP/wH2aQa1Gz63W71FLGsnRgTo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FPVrgvw2wgMkunqzm/UVqbhNPgCkIgZCkys2V+fEZKtjw1q7XXvdASWWqNaYDHZjk EWOzCy75uaXShrY6KV35Ttab5dHALTemcFTZNZ9lNxKH057F14APXfbEZfa3iP11Qb tRSlHCttjbhevoTxC/H+yZfbUB3cWvpgiOg/a3BI= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:31 +0530 Message-Id: <20210910070638.467294-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/9] camera_device: Remove private scope of Camera3RequestDescriptor X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Camera3RequestDescriptor is a utility structure that groups information about a capture request. It can be and will be extended to preserve the context of a capture overall. Since the context of a capture needs to be shared among other classes(for e.g. CameraStream) having a private definition of the struct doesn't help. Hence, de-scope the structure so that it can be shared with other components (through references or pointers). Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_device.cpp | 2 +- src/android/camera_device.h | 29 +++++++++++++++-------------- 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index f2f36f32..0840c056 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -220,7 +220,7 @@ bool validateCropRotate(const camera3_stream_configuration_t &streamList) * later re-used at request complete time to notify the framework. */ -CameraDevice::Camera3RequestDescriptor::Camera3RequestDescriptor( +Camera3RequestDescriptor::Camera3RequestDescriptor( Camera *camera, const camera3_capture_request_t *camera3Request) { frameNumber_ = camera3Request->frame_number; diff --git a/src/android/camera_device.h b/src/android/camera_device.h index a5576927..b59ac3e7 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -33,6 +33,21 @@ #include "jpeg/encoder.h" struct CameraConfigData; + +struct Camera3RequestDescriptor { + Camera3RequestDescriptor() = default; + ~Camera3RequestDescriptor() = default; + Camera3RequestDescriptor(libcamera::Camera *camera, + const camera3_capture_request_t *camera3Request); + Camera3RequestDescriptor &operator=(Camera3RequestDescriptor &&) = default; + + uint32_t frameNumber_ = 0; + std::vector buffers_; + std::vector> frameBuffers_; + CameraMetadata settings_; + std::unique_ptr request_; +}; + class CameraDevice : protected libcamera::Loggable { public: @@ -71,20 +86,6 @@ private: CameraDevice(unsigned int id, std::shared_ptr camera); - struct Camera3RequestDescriptor { - Camera3RequestDescriptor() = default; - ~Camera3RequestDescriptor() = default; - Camera3RequestDescriptor(libcamera::Camera *camera, - const camera3_capture_request_t *camera3Request); - Camera3RequestDescriptor &operator=(Camera3RequestDescriptor &&) = default; - - uint32_t frameNumber_ = 0; - std::vector buffers_; - std::vector> frameBuffers_; - CameraMetadata settings_; - std::unique_ptr request_; - }; - enum class State { Stopped, Flushing, From patchwork Fri Sep 10 07:06:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13801 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 11A5ABDC71 for ; Fri, 10 Sep 2021 07:06:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CD99269179; Fri, 10 Sep 2021 09:06:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="koe+4uDr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0134669169 for ; Fri, 10 Sep 2021 09:06:53 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D7E1D883; Fri, 10 Sep 2021 09:06:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257612; bh=uquyLzokirPsp+ruweahkcaBqeUYhQGzlxPivJ/R1vs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=koe+4uDrjLstlHrPHlF9TXxwUNUrLAI6cvPWCottBKW70Dv8lOBQfm63DpuFgXZDp EOtDt8y9KmE4twfrDpl8bO1lQ8lM0CzzSfPBpROUGp4baoAeQSWHq1zhcpu6nyPVdt LYDI1OQwz859rF+nXQyad5AMqQaOQhQs0GFlydRM= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:32 +0530 Message-Id: <20210910070638.467294-4-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/9] android: post_processor: Plumb down the process() with context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The context required for post processing of a camera request is saved via Camera3RequestDescriptor. The context needs to be plumbed to PostProcessor::process() which should be passed a const pointer to Camera3RequestDescriptor. In subsequent commits, we will prepare the post-processor to run asynchrnoously hence, it will require the context pointer to be emitted back to the upper layers, to identify for which request the post-processing has been completed. Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_device.cpp | 3 ++- src/android/camera_stream.cpp | 6 ++++-- src/android/camera_stream.h | 5 ++++- src/android/jpeg/post_processor_jpeg.cpp | 3 ++- src/android/jpeg/post_processor_jpeg.h | 3 ++- src/android/post_processor.h | 5 ++++- src/android/yuv/post_processor_yuv.cpp | 3 ++- src/android/yuv/post_processor_yuv.h | 3 ++- 8 files changed, 22 insertions(+), 9 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 0840c056..84549d04 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1150,7 +1150,8 @@ void CameraDevice::requestComplete(Request *request) int ret = cameraStream->process(src, *buffer.buffer, descriptor.settings_, - resultMetadata.get()); + resultMetadata.get(), + &descriptor); /* * Return the FrameBuffer to the CameraStream now that we're * done processing it. diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 0fed5382..d1c54797 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -101,7 +101,8 @@ int CameraStream::configure() int CameraStream::process(const FrameBuffer *source, buffer_handle_t camera3Dest, const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) + CameraMetadata *resultMetadata, + const Camera3RequestDescriptor *context) { if (!postProcessor_) return 0; @@ -118,7 +119,8 @@ int CameraStream::process(const FrameBuffer *source, return -EINVAL; } - return postProcessor_->process(source, &dest, requestMetadata, resultMetadata); + return postProcessor_->process(source, &dest, requestMetadata, + resultMetadata, context); } FrameBuffer *CameraStream::getBuffer() diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 5c232cb6..d589f699 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -23,6 +23,8 @@ class CameraDevice; class CameraMetadata; class PostProcessor; +struct Camera3RequestDescriptor; + class CameraStream { public: @@ -121,7 +123,8 @@ public: int process(const libcamera::FrameBuffer *source, buffer_handle_t camera3Dest, const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata); + CameraMetadata *resultMetadata, + const Camera3RequestDescriptor *context); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index cb45f86b..741eeb21 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -100,7 +100,8 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer *source, int PostProcessorJpeg::process(const FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) + CameraMetadata *resultMetadata, + const Camera3RequestDescriptor *context) { if (!encoder_) return 0; diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index c4b2e9ef..72c38fdb 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -25,7 +25,8 @@ public: int process(const libcamera::FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) override; + CameraMetadata *resultMetadata, + const Camera3RequestDescriptor *context) override; private: void generateThumbnail(const libcamera::FrameBuffer *source, diff --git a/src/android/post_processor.h b/src/android/post_processor.h index 61dfb6d4..1213e7e6 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -14,6 +14,8 @@ class CameraMetadata; +struct Camera3RequestDescriptor; + class PostProcessor { public: @@ -24,7 +26,8 @@ public: virtual int process(const libcamera::FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, - CameraMetadata *resultMetadata) = 0; + CameraMetadata *resultMetadata, + const Camera3RequestDescriptor *context) = 0; }; #endif /* __ANDROID_POST_PROCESSOR_H__ */ diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 0a874886..13a78abe 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -52,7 +52,8 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, int PostProcessorYuv::process(const FrameBuffer *source, CameraBuffer *destination, [[maybe_unused]] const CameraMetadata &requestMetadata, - [[maybe_unused]] CameraMetadata *metadata) + [[maybe_unused]] CameraMetadata *metadata, + [[maybe_unused]] const Camera3RequestDescriptor *context) { if (!isValidBuffers(source, *destination)) return -EINVAL; diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h index 44a04113..38a30f91 100644 --- a/src/android/yuv/post_processor_yuv.h +++ b/src/android/yuv/post_processor_yuv.h @@ -23,7 +23,8 @@ public: int process(const libcamera::FrameBuffer *source, CameraBuffer *destination, const CameraMetadata &requestMetadata, - CameraMetadata *metadata) override; + CameraMetadata *metadata, + const Camera3RequestDescriptor *context) override; private: bool isValidBuffers(const libcamera::FrameBuffer *source, From patchwork Fri Sep 10 07:06:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13802 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76D21BDC71 for ; Fri, 10 Sep 2021 07:06:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3927869176; Fri, 10 Sep 2021 09:06:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kWJcQ1tV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 09D7169169 for ; Fri, 10 Sep 2021 09:06:55 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DB959883; Fri, 10 Sep 2021 09:06:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257614; bh=m9I01tWs4PuAhksXpy0rzOChudu48aF0JkDThhFLk0Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kWJcQ1tVUYeShVrr2ZwnnF+1GGjyqqk6wLfqmQc5wWh0eETu5F6CWmmulZaUQWptn iErQ7KHRL5Fdf7x/auD/jjttHDQpylFeZLJcDUB/UoCkGgT0VlrmmxO+33r5eMy01T X69D5pnwDwd2xmMZMUmANjKVDceL6hAqL94jtzx8= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:33 +0530 Message-Id: <20210910070638.467294-5-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/9] android: post_processor: Notify post processing completion status X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We should be able to know if post-processing has been completed successfully or encountered some errors. This commit introduces a Signal<> which when connected, will notify that the post-processing has been completed. The status of PostProcessor::process() will be passed as a PostProcessor::Status argument. The signal will be required when the post-processor is meant to run asynchronously (in subsequent commits). Signed-off-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/jpeg/post_processor_jpeg.cpp | 7 ++++++- src/android/post_processor.h | 8 ++++++++ src/android/yuv/post_processor_yuv.cpp | 4 ++++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index 741eeb21..1df3dfb1 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -103,8 +103,10 @@ int PostProcessorJpeg::process(const FrameBuffer *source, CameraMetadata *resultMetadata, const Camera3RequestDescriptor *context) { - if (!encoder_) + if (!encoder_) { + processComplete.emit(PostProcessor::Status::Failed, context); return 0; + } ASSERT(destination->numPlanes() == 1); @@ -197,6 +199,7 @@ int PostProcessorJpeg::process(const FrameBuffer *source, exif.data(), quality); if (jpeg_size < 0) { LOG(JPEG, Error) << "Failed to encode stream image"; + processComplete.emit(PostProcessor::Status::Failed, context); return jpeg_size; } @@ -211,5 +214,7 @@ int PostProcessorJpeg::process(const FrameBuffer *source, /* Update the JPEG result Metadata. */ resultMetadata->addEntry(ANDROID_JPEG_SIZE, jpeg_size); + processComplete.emit(PostProcessor::Status::Succeeded, context); + return 0; } diff --git a/src/android/post_processor.h b/src/android/post_processor.h index 1213e7e6..25c9411f 100644 --- a/src/android/post_processor.h +++ b/src/android/post_processor.h @@ -7,6 +7,8 @@ #ifndef __ANDROID_POST_PROCESSOR_H__ #define __ANDROID_POST_PROCESSOR_H__ +#include + #include #include @@ -28,6 +30,12 @@ public: const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata, const Camera3RequestDescriptor *context) = 0; + + enum Status { + Succeeded, + Failed, + }; + libcamera::Signal processComplete; }; #endif /* __ANDROID_POST_PROCESSOR_H__ */ diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 13a78abe..d2df110a 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -61,6 +61,7 @@ int PostProcessorYuv::process(const FrameBuffer *source, const MappedFrameBuffer sourceMapped(source, MappedFrameBuffer::MapFlag::Read); if (!sourceMapped.isValid()) { LOG(YUV, Error) << "Failed to mmap camera frame buffer"; + processComplete.emit(PostProcessor::Status::Failed, context); return -EINVAL; } @@ -78,9 +79,12 @@ int PostProcessorYuv::process(const FrameBuffer *source, libyuv::FilterMode::kFilterBilinear); if (ret) { LOG(YUV, Error) << "Failed NV12 scaling: " << ret; + processComplete.emit(PostProcessor::Status::Failed, context); return -EINVAL; } + processComplete.emit(PostProcessor::Status::Succeeded, context); + return 0; } From patchwork Fri Sep 10 07:06:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13803 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DAB87BDC71 for ; Fri, 10 Sep 2021 07:06:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 999026916D; Fri, 10 Sep 2021 09:06:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wK8/itDF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D356369169 for ; Fri, 10 Sep 2021 09:06:56 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CE520883; Fri, 10 Sep 2021 09:06:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257616; bh=EjrgOUPInzOsFwr8WIeUqDQz29bU+sJYG1tCgeGwvbw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wK8/itDFrmrBKEYxWXGA7OG4tMnoop7B2QLYY9QfyCyeS7Li4kW0Tatn2xWDvkg8g 9jMp5LkfHUqVD5DT1UPNSKJoEpzg6tm/3spKQdyvU1ppzT9C+tK/yA23ocr9FXw5HW 25CTvQLkrHZ0JhoV6tn4TcYmXShBKgb6NC96Lxrk= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:34 +0530 Message-Id: <20210910070638.467294-6-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/9] android: camera_stream: Query post-processing status X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" CameraStream takes care of any post-processing required via the CameraStream::process(). Since the post-processor will be moved to a separate thread (in subsequent commits), the caller of PostProcessor::process() will notify the associated CameraStream instance about the post-processing completion via processComplete signal. The CameraStream class should handle this event to query the status and take next steps of reporting it back to upper layers (i.e. CameraDevice). Signed-off-by: Umang Jain --- src/android/camera_stream.cpp | 13 +++++++++++++ src/android/camera_stream.h | 13 ++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index d1c54797..996779c4 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -81,6 +81,9 @@ int CameraStream::configure() int ret = postProcessor_->configure(configuration(), output); if (ret) return ret; + + postProcessor_->processComplete.connect( + this, &CameraStream::postProcessingComplete); } if (allocator_) { @@ -123,6 +126,16 @@ int CameraStream::process(const FrameBuffer *source, resultMetadata, context); } +void CameraStream::postProcessingComplete(PostProcessor::Status status, + const Camera3RequestDescriptor *context) +{ + ProcessStatus processStatus; + if (status == PostProcessor::Status::Succeeded) + processStatus = ProcessStatus::Succeeded; + else + processStatus = ProcessStatus::Failed; +} + FrameBuffer *CameraStream::getBuffer() { if (!allocator_) diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index d589f699..be076995 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -13,15 +13,18 @@ #include +#include + #include #include #include #include #include +#include "post_processor.h" + class CameraDevice; class CameraMetadata; -class PostProcessor; struct Camera3RequestDescriptor; @@ -128,7 +131,15 @@ public: libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); + enum ProcessStatus { + Succeeded, + Failed, + }; + private: + void postProcessingComplete(PostProcessor::Status status, + const Camera3RequestDescriptor *context); + CameraDevice *const cameraDevice_; const libcamera::CameraConfiguration *config_; const Type type_; From patchwork Fri Sep 10 07:06:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13804 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 53416BDC71 for ; Fri, 10 Sep 2021 07:07:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 248BE69177; Fri, 10 Sep 2021 09:07:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="coRZ+SeZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 90BD069169 for ; Fri, 10 Sep 2021 09:06:58 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8F6F6BC0; Fri, 10 Sep 2021 09:06:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257618; bh=OqW2lZGCcnuofLg+JxPbMd736XyeEBFLOPyj2TNi1wA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=coRZ+SeZyQrRcxXa726pBAyCt3dyoj728WCg2o/jjCGsnjy5G6o6Ctmouj/CDGp6M dVMlL3qS2RH6amAiUX36bxCRfEe5vZQNoedIpr7AKWF4X51PTFFsQi1ZTm5Q5iZpg+ p1XWVqNLUkDFm6y/uUeqbEiid6fFglWLSNsEZqSk= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:35 +0530 Message-Id: <20210910070638.467294-7-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/9] android: camera_device: Add a queue for sending capture results X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a camera capture request completes, the next step is to send the capture results to the framework via process_capture_results(). All the capture associated result metadata is prepared and populated. If any post-processing is required, it will also take place in the same thread (which can be blocking for a subtaintial amount of time) before the results can be sent back to the framework. Subsequent patches will move the post-processing to run in a separate thread. In order to do so, there is few bits of groundwork which this patch entails. Mainly, we need to preserve the order of sending the capture results back in which they were queued by the framework to the HAL (i.e. the order is sequential). Hence, we need to add a queue in which capture results can be queued with context. As per this patch, the post-processor still runs synchronously as before, but it will queue up the current capture results with context. Later down the line, the capture results are dequeud in order and sent back to the framework via sendQueuedCaptureResults(). If the queue is empty or unused, the current behaviour is to skip the queue and send capture results directly. The context is preserved using Camera3RequestDescriptor utility structure. It has been expanded accordingly to address the needs of the functionality. --- Check if we can drop unique_ptr to camera3descriptor --- Signed-off-by: Umang Jain --- src/android/camera_device.cpp | 106 ++++++++++++++++++++++++++++++---- src/android/camera_device.h | 22 +++++++ src/android/camera_stream.cpp | 2 + 3 files changed, 119 insertions(+), 11 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 84549d04..7f04d225 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -240,6 +240,8 @@ Camera3RequestDescriptor::Camera3RequestDescriptor( /* Clone the controls associated with the camera3 request. */ settings_ = CameraMetadata(camera3Request->settings); + internalBuffer_ = nullptr; + /* * Create the CaptureRequest, stored as a unique_ptr<> to tie its * lifetime to the descriptor. @@ -989,6 +991,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques * once it has been processed. */ buffer = cameraStream->getBuffer(); + descriptor.internalBuffer_ = buffer; LOG(HAL, Debug) << ss.str() << " (internal)"; break; } @@ -1148,26 +1151,107 @@ void CameraDevice::requestComplete(Request *request) continue; } + + /* + * Save the current context of capture result and queue the + * descriptor before processing the camera stream. + * + * When the processing completes, the descriptor will be + * dequeued and capture results will be sent to the framework. + */ + descriptor.status_ = Camera3RequestDescriptor::Pending; + descriptor.resultMetadata_ = std::move(resultMetadata); + descriptor.captureResult_ = captureResult; + + std::unique_ptr reqDescriptor = + std::make_unique(); + *reqDescriptor = std::move(descriptor); + queuedDescriptor_.push_back(std::move(reqDescriptor)); + + Camera3RequestDescriptor *currentDescriptor = queuedDescriptor_.back().get(); int ret = cameraStream->process(src, *buffer.buffer, - descriptor.settings_, - resultMetadata.get(), - &descriptor); + currentDescriptor->settings_, + currentDescriptor->resultMetadata_.get(), + currentDescriptor); + return; + } + + if (queuedDescriptor_.empty()) { + captureResult.result = resultMetadata->get(); + callbacks_->process_capture_result(callbacks_, &captureResult); + } else { + /* + * Previous capture results waiting to be sent to framework + * hence, queue the current capture results as well. After that, + * check if any results are ready to be sent back to the + * framework. + */ + descriptor.status_ = Camera3RequestDescriptor::Succeeded; + descriptor.resultMetadata_ = std::move(resultMetadata); + descriptor.captureResult_ = captureResult; + std::unique_ptr reqDescriptor = + std::make_unique(); + *reqDescriptor = std::move(descriptor); + queuedDescriptor_.push_back(std::move(reqDescriptor)); + + sendQueuedCaptureResults(); + } +} + +void CameraDevice::streamProcessingComplete(CameraStream *cameraStream, + CameraStream::ProcessStatus status, + const Camera3RequestDescriptor *context) +{ + for (auto &d : queuedDescriptor_) { + if (d.get() != context) + continue; + /* * Return the FrameBuffer to the CameraStream now that we're * done processing it. */ if (cameraStream->type() == CameraStream::Type::Internal) - cameraStream->putBuffer(src); - - if (ret) { - buffer.status = CAMERA3_BUFFER_STATUS_ERROR; - notifyError(descriptor.frameNumber_, buffer.stream, - CAMERA3_MSG_ERROR_BUFFER); + cameraStream->putBuffer(d->internalBuffer_); + + if (status == CameraStream::ProcessStatus::Succeeded) { + d->status_ = Camera3RequestDescriptor::Succeeded; + } else { + d->status_ = Camera3RequestDescriptor::Failed; + + d->captureResult_.partial_result = 0; + for (camera3_stream_buffer_t &buffer : d->buffers_) { + CameraStream *cs = static_cast(buffer.stream->priv); + + if (cs->camera3Stream().format != HAL_PIXEL_FORMAT_BLOB) + continue; + buffer.status = CAMERA3_BUFFER_STATUS_ERROR; + notifyError(d->frameNumber_, buffer.stream, + CAMERA3_MSG_ERROR_BUFFER); + } } + break; } - captureResult.result = resultMetadata->get(); - callbacks_->process_capture_result(callbacks_, &captureResult); + /* + * Send back capture results to the framework by inspecting the queue. + * The framework can defer queueing further requests to the HAL (via + * process_capture_request) unless until it receives the capture results + * for already queued requests. + */ + sendQueuedCaptureResults(); +} + +void CameraDevice::sendQueuedCaptureResults() +{ + while (!queuedDescriptor_.empty()) { + std::unique_ptr &d = queuedDescriptor_.front(); + if (d->status_ == Camera3RequestDescriptor::Pending) + return; + + d->captureResult_.result = d->resultMetadata_->get(); + callbacks_->process_capture_result(callbacks_, &(d->captureResult_)); + queuedDescriptor_.pop_front(); + } } std::string CameraDevice::logPrefix() const diff --git a/src/android/camera_device.h b/src/android/camera_device.h index b59ac3e7..e7318358 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -7,6 +7,7 @@ #ifndef __ANDROID_CAMERA_DEVICE_H__ #define __ANDROID_CAMERA_DEVICE_H__ +#include #include #include #include @@ -46,6 +47,20 @@ struct Camera3RequestDescriptor { std::vector> frameBuffers_; CameraMetadata settings_; std::unique_ptr request_; + + /* + * Below are utility placeholders used when a capture result + * needs to be queued before completion via process_capture_result() + */ + enum completionStatus { + Pending, + Succeeded, + Failed, + }; + std::unique_ptr resultMetadata_; + camera3_capture_result_t captureResult_; + libcamera::FrameBuffer *internalBuffer_; + completionStatus status_; }; class CameraDevice : protected libcamera::Loggable @@ -78,6 +93,9 @@ public: int processCaptureRequest(camera3_capture_request_t *request); void requestComplete(libcamera::Request *request); + void streamProcessingComplete(CameraStream *stream, + CameraStream::ProcessStatus status, + const Camera3RequestDescriptor *context); protected: std::string logPrefix() const override; @@ -106,6 +124,8 @@ private: std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; + void sendQueuedCaptureResults(); + unsigned int id_; camera3_device_t camera3Device_; @@ -126,6 +146,8 @@ private: libcamera::Mutex descriptorsMutex_; /* Protects descriptors_. */ std::map descriptors_; + std::deque> queuedDescriptor_; + std::string maker_; std::string model_; diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 996779c4..c7d874b2 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -134,6 +134,8 @@ void CameraStream::postProcessingComplete(PostProcessor::Status status, processStatus = ProcessStatus::Succeeded; else processStatus = ProcessStatus::Failed; + + cameraDevice_->streamProcessingComplete(this, processStatus, context); } FrameBuffer *CameraStream::getBuffer() From patchwork Fri Sep 10 07:06:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13805 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 03711BEFBE for ; Fri, 10 Sep 2021 07:07:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A203269174; Fri, 10 Sep 2021 09:07:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hFcGIGiM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8316069169 for ; Fri, 10 Sep 2021 09:07:00 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 68D68883; Fri, 10 Sep 2021 09:06:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257620; bh=L5ElbkmgBrisKXGQ7Lfz+IPE87H2WKrEoB/zaDYeFPM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hFcGIGiM4Jooqch2N/zHVfKrIjzLMJBBSDA9GsnedOTMceZq/xHzK54jcdGQMPPus 27oqykhvIYVzABluy8AtAGIhWmIbTV3ZZGh0/gNBWXdDq19cMOWDS5Awh22JZ98Q5d MFGakRu+vWJ5SBht4FnTNs+92OyZlmzLABAOqZrQ= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:36 +0530 Message-Id: <20210910070638.467294-8-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 7/9] android: camera_device: Move buffer mapping for post processing X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Buffer mapping for post processing currently happens in CameraStream::process(). However, it can be easily be moved to CameraDevice just before the call to CameraStream::process(). The reason to do so is that, we can hold the mapped destination buffer pointer as a part of Camera3RequestDescriptor. Camera3RequestDescriptor already hold other post-processing related context to enable async post processing in subsequent commits. Signed-off-by: Umang Jain --- src/android/camera_device.cpp | 14 +++++++++++++- src/android/camera_device.h | 1 + src/android/camera_stream.cpp | 16 ++-------------- src/android/camera_stream.h | 2 +- 4 files changed, 17 insertions(+), 16 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 7f04d225..988d4232 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1163,13 +1163,25 @@ void CameraDevice::requestComplete(Request *request) descriptor.resultMetadata_ = std::move(resultMetadata); descriptor.captureResult_ = captureResult; + /* \todo Buffer mapping should be moved to a separate thread? */ + const StreamConfiguration &output = cameraStream->configuration(); + descriptor.destBuffer_ = std::make_unique( + *buffer.buffer, formats::MJPEG, output.size, + PROT_READ | PROT_WRITE); + + if (!descriptor.destBuffer_->isValid()) { + LOG(HAL, Error) << "Failed to map android blob buffer"; + return; + } + std::unique_ptr reqDescriptor = std::make_unique(); *reqDescriptor = std::move(descriptor); queuedDescriptor_.push_back(std::move(reqDescriptor)); Camera3RequestDescriptor *currentDescriptor = queuedDescriptor_.back().get(); - int ret = cameraStream->process(src, *buffer.buffer, + int ret = cameraStream->process(src, + currentDescriptor->destBuffer_.get(), currentDescriptor->settings_, currentDescriptor->resultMetadata_.get(), currentDescriptor); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index e7318358..b62d373c 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -58,6 +58,7 @@ struct Camera3RequestDescriptor { Failed, }; std::unique_ptr resultMetadata_; + std::unique_ptr destBuffer_; camera3_capture_result_t captureResult_; libcamera::FrameBuffer *internalBuffer_; completionStatus status_; diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index c7d874b2..5fd04bbf 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -102,7 +102,7 @@ int CameraStream::configure() } int CameraStream::process(const FrameBuffer *source, - buffer_handle_t camera3Dest, + CameraBuffer *destBuffer, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata, const Camera3RequestDescriptor *context) @@ -110,19 +110,7 @@ int CameraStream::process(const FrameBuffer *source, if (!postProcessor_) return 0; - /* - * \todo Buffer mapping and processing should be moved to a - * separate thread. - */ - const StreamConfiguration &output = configuration(); - CameraBuffer dest(camera3Dest, formats::MJPEG, output.size, - PROT_READ | PROT_WRITE); - if (!dest.isValid()) { - LOG(HAL, Error) << "Failed to map android blob buffer"; - return -EINVAL; - } - - return postProcessor_->process(source, &dest, requestMetadata, + return postProcessor_->process(source, destBuffer, requestMetadata, resultMetadata, context); } diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index be076995..8097ddbc 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -124,7 +124,7 @@ public: int configure(); int process(const libcamera::FrameBuffer *source, - buffer_handle_t camera3Dest, + CameraBuffer *destBuffer, const CameraMetadata &requestMetadata, CameraMetadata *resultMetadata, const Camera3RequestDescriptor *context); From patchwork Fri Sep 10 07:06:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13806 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1E01FBDC71 for ; Fri, 10 Sep 2021 07:07:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B84126917B; Fri, 10 Sep 2021 09:07:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Js9BbFM9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9700B6916E for ; Fri, 10 Sep 2021 09:07:02 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8DB4B883; Fri, 10 Sep 2021 09:07:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257622; bh=GhFCseGpcp3egLXZPZSTCi008bvqr/gET9XO/EDdE6c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Js9BbFM9nHF3wCdXq8Ssm5eGBmvtxPNNLA0CfNNHnsPIfsE9R93fvc0OYRPUwBC/U +v6omaGw14mwKTiNw0nKs/Y6OXhmFQdEc+7ni/T7buN5w4c0dVOqvDKM2WUBEsreQJ hEXw+7FND291MHWPnf2XR22p8H5QNsf2shsQu4QI= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:37 +0530 Message-Id: <20210910070638.467294-9-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 8/9] android: camera_stream: Run post-processor in a separate thread X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In CameraStream, introduce a new private PostProcessorWorker class derived from Object. A instance of PostProcessorWorker is moved to a separate thread instance which will be responsible to run the post-processor. Running PostProcessor asynchronously should entail that all the data context needed by the PostProcessor should remain valid for the entire duration of its run. Most of the context preserving part has been addressed in the previous commits, we just need to ensure the source framebuffer data that comes via Camera::Request, should remain valid for the entire duration of post-processing running asynchronously. In order to so, we maintain a separate copy of the framebuffer data and add it to the Camera3RequestDescriptor structure in which we preserve rest of the context. Signed-off-by: Umang Jain --- src/android/camera_device.cpp | 19 ++++++++++++++++++- src/android/camera_device.h | 4 ++++ src/android/camera_stream.cpp | 16 ++++++++++++++-- src/android/camera_stream.h | 28 ++++++++++++++++++++++++++++ 4 files changed, 64 insertions(+), 3 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 988d4232..73eb5758 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1174,13 +1174,29 @@ void CameraDevice::requestComplete(Request *request) return; } + FrameBuffer *source = src; + if (cameraStream->type() != CameraStream::Type::Internal) { + /* + * The source buffer is owned by Request object which + * can be reused by libcamera. Since post-processor will + * run asynchrnously, we need to copy the request's + * frame buffer and use that as the source buffer for + * post processing. + */ + for (const auto &plane : src->planes()) + descriptor.srcPlanes_.push_back(plane); + descriptor.srcFramebuffer_ = + std::make_unique(descriptor.srcPlanes_); + source = descriptor.srcFramebuffer_.get(); + } + std::unique_ptr reqDescriptor = std::make_unique(); *reqDescriptor = std::move(descriptor); queuedDescriptor_.push_back(std::move(reqDescriptor)); Camera3RequestDescriptor *currentDescriptor = queuedDescriptor_.back().get(); - int ret = cameraStream->process(src, + int ret = cameraStream->process(source, currentDescriptor->destBuffer_.get(), currentDescriptor->settings_, currentDescriptor->resultMetadata_.get(), @@ -1255,6 +1271,7 @@ void CameraDevice::streamProcessingComplete(CameraStream *cameraStream, void CameraDevice::sendQueuedCaptureResults() { + MutexLocker lock(queuedDescriptorsMutex_); while (!queuedDescriptor_.empty()) { std::unique_ptr &d = queuedDescriptor_.front(); if (d->status_ == Camera3RequestDescriptor::Pending) diff --git a/src/android/camera_device.h b/src/android/camera_device.h index b62d373c..ecdda06e 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -62,6 +62,9 @@ struct Camera3RequestDescriptor { camera3_capture_result_t captureResult_; libcamera::FrameBuffer *internalBuffer_; completionStatus status_; + + std::unique_ptr srcFramebuffer_; + std::vector srcPlanes_; }; class CameraDevice : protected libcamera::Loggable @@ -147,6 +150,7 @@ private: libcamera::Mutex descriptorsMutex_; /* Protects descriptors_. */ std::map descriptors_; + libcamera::Mutex queuedDescriptorsMutex_; /* Protects queuedDescriptor_. */ std::deque> queuedDescriptor_; std::string maker_; diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 5fd04bbf..845e2462 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -55,6 +55,15 @@ CameraStream::CameraStream(CameraDevice *const cameraDevice, * is what we instantiate here. */ postProcessor_ = std::make_unique(cameraDevice_); + ppWorker_ = std::make_unique(postProcessor_.get()); + + thread_ = std::make_unique(); + ppWorker_->moveToThread(thread_.get()); + /* + * \todo: Class is MoveConstructible, so where to stop thread + * if we don't user-defined destructor? See RFC patch at the end. + */ + thread_->start(); } if (type == Type::Internal) { @@ -110,8 +119,11 @@ int CameraStream::process(const FrameBuffer *source, if (!postProcessor_) return 0; - return postProcessor_->process(source, destBuffer, requestMetadata, - resultMetadata, context); + ppWorker_->invokeMethod(&PostProcessorWorker::process, + ConnectionTypeQueued, source, destBuffer, + requestMetadata, resultMetadata, context); + + return 0; } void CameraStream::postProcessingComplete(PostProcessor::Status status, diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 8097ddbc..dbb7eee3 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -13,7 +13,9 @@ #include +#include #include +#include #include #include @@ -23,6 +25,7 @@ #include "post_processor.h" +class CameraBuffer; class CameraDevice; class CameraMetadata; @@ -137,6 +140,29 @@ public: }; private: + class PostProcessorWorker : public libcamera::Object + { + public: + PostProcessorWorker(PostProcessor *postProcessor) + { + postProcessor_ = postProcessor; + } + + void process(const libcamera::FrameBuffer *source, + CameraBuffer *destination, + const CameraMetadata &requestMetadata, + CameraMetadata *resultMetadata, + const Camera3RequestDescriptor *context) + { + postProcessor_->process(source, destination, + requestMetadata, resultMetadata, + context); + } + + private: + PostProcessor *postProcessor_; + }; + void postProcessingComplete(PostProcessor::Status status, const Camera3RequestDescriptor *context); @@ -154,6 +180,8 @@ private: */ std::unique_ptr mutex_; std::unique_ptr postProcessor_; + std::unique_ptr ppWorker_; + std::unique_ptr thread_; }; #endif /* __ANDROID_CAMERA_STREAM__ */ From patchwork Fri Sep 10 07:06:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13807 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B4B89BDC71 for ; Fri, 10 Sep 2021 07:07:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 811536917E; Fri, 10 Sep 2021 09:07:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WoVkS3ZQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D1A269173 for ; Fri, 10 Sep 2021 09:07:04 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.149]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 67C21BC0; Fri, 10 Sep 2021 09:07:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631257624; bh=F2quetUbPCNcaM/qIjB8+u9Jj2pS/LI0AsvgPyb0TXM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WoVkS3ZQXDvSMKnWMgTyMZi8AFT9IU4IJv55HU8ZVhlsJtF9AFGC/LRr/pRjMTSJA MK3OFZhPf7BuxAO48Xilv8ZVPZGU7RrO9Y3io2tRTSDO6Zyb/NfhVihXhORDFQGN4x N1CJEeV8oyCKZkyfX25gl9RoshNeyRgI67r32n0M= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 10 Sep 2021 12:36:38 +0530 Message-Id: <20210910070638.467294-10-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210910070638.467294-1-umang.jain@ideasonboard.com> References: <20210910070638.467294-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 9/9] RFC: Stop PostProcessor when camera is stopped X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stopping the post-processor thread from CameraDevice is not ideal. But it temporarily avoids the crash on stopping the camera. Signed-off-by: Umang Jain --- src/android/camera_device.cpp | 8 ++++++++ src/android/camera_stream.h | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 73eb5758..fa2db72f 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -425,6 +425,14 @@ int CameraDevice::open(const hw_module_t *hardwareModule) void CameraDevice::close() { + for (CameraStream &cameraStream : streams_) { + /* CameraStream doesn't have user defined destructor as + * it is MoveConstructible. Stop the post-processor thread + * from here before clearing streams_. + */ + cameraStream.stopPostProcessorThread(); + } + streams_.clear(); stop(); diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index dbb7eee3..bd37473e 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -133,6 +133,13 @@ public: const Camera3RequestDescriptor *context); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); + void stopPostProcessorThread() + { + if (thread_) { + thread_->exit(); + thread_->wait(); + } + } enum ProcessStatus { Succeeded,