{"id":13871,"url":"https://patchwork.libcamera.org/api/1.1/patches/13871/?format=json","web_url":"https://patchwork.libcamera.org/patch/13871/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210920173752.1346190-8-umang.jain@ideasonboard.com>","date":"2021-09-20T17:37:49","name":"[libcamera-devel,v3,07/10] android: post_processor: Notify post processing completion status","commit_ref":null,"pull_url":null,"state":"rfc","archived":false,"hash":"7b136926190f1219697ee8eaf9a1d70195882447","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/1.1/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":{"id":12,"url":"https://patchwork.libcamera.org/api/1.1/users/12/?format=json","username":"uajain","first_name":"Umang","last_name":"Jain","email":"umang.jain@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/13871/mbox/","series":[{"id":2539,"url":"https://patchwork.libcamera.org/api/1.1/series/2539/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2539","date":"2021-09-20T17:37:42","name":"Async post processor","version":3,"mbox":"https://patchwork.libcamera.org/series/2539/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13871/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13871/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 184E3BF01C\n\tfor ;\n\tMon, 20 Sep 2021 17:38:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D92A369191;\n\tMon, 20 Sep 2021 19:38:17 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0501D69188\n\tfor ;\n\tMon, 20 Sep 2021 19:38:15 +0200 (CEST)","from perceval.ideasonboard.com (unknown [103.251.226.144])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id ED54A45E;\n\tMon, 20 Sep 2021 19:38:13 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"hvZt05rg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1632159494;\n\tbh=otjbcXFzfV0R3vWLVQjawlrwWfIpS2GtnJkv5aBbcHw=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=hvZt05rgNsC24DqKcxfSGlJWQ9EUddRKUX4Y+khsSnH+RH9BPq2N6urlI3Ge7taoB\n\tHXnTEYfpzyO3iDKP+kDVZ2N8dlcGL4iCeu+Uwod0IiM74VaCcPZulpKGpZZmVGt220\n\ttJbm3rQX+k140F4ZQUS1TqaWohiAmIPJ9zAE7GMw=","From":"Umang Jain ","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 20 Sep 2021 23:07:49 +0530","Message-Id":"<20210920173752.1346190-8-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","In-Reply-To":"<20210920173752.1346190-1-umang.jain@ideasonboard.com>","References":"<20210920173752.1346190-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 07/10] android: post_processor: Notify\n\tpost processing completion status","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"We should be able to know if post-processing has been completed\nsuccessfully or encountered some errors. This commit introduces a\nSignal<> that will notify that the post-processing has been completed.\nIf the post processing was successful, status on the request descriptor\nwill be set to Camera3RequestDescriptor::ProcessStatus::Success.\nThe signal will be required when the post-processor is meant to\nrun asynchronously (in subsequent commits) and capture results need\nto be sent back to the framework from the signal's slot instead.\n\nSigned-off-by: Umang Jain \n---\n src/android/camera_device.cpp | 18 ++++++++++++++++++\n src/android/camera_device.h | 11 +++++++++++\n src/android/camera_stream.cpp | 15 ++++++++++++++-\n src/android/camera_stream.h | 2 ++\n src/android/jpeg/post_processor_jpeg.cpp | 10 +++++++++-\n src/android/post_processor.h | 4 ++++\n src/android/yuv/post_processor_yuv.cpp | 16 ++++++++++++++--\n 7 files changed, 72 insertions(+), 4 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex fa462368..e2de4012 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -246,6 +246,12 @@ Camera3RequestDescriptor::Camera3RequestDescriptor(\n \t */\n \trequest_ = std::make_unique(camera,\n \t\t\t\t\t\t reinterpret_cast(this));\n+\n+\t/*\n+\t * Denotes the post-processing status required by any stream. It is set\n+\t * to ProcessStatus::Success if processing is successful.\n+\t */\n+\tstatus_ = ProcessStatus::None;\n }\n \n /*\n@@ -1150,6 +1156,8 @@ void CameraDevice::requestComplete(Request *request)\n \t\t\tcontinue;\n \t\t}\n \n+\t\tdescriptor->status_ = Camera3RequestDescriptor::ProcessStatus::Processing;\n+\n \t\tint ret = cameraStream->process(src, *buffer.buffer, descriptor);\n \n \t\t/*\n@@ -1172,6 +1180,16 @@ void CameraDevice::requestComplete(Request *request)\n \tdescriptors_.pop_front();\n }\n \n+\n+void CameraDevice::streamProcessingComplete([[maybe_unused]] CameraStream *cameraStream,\n+\t\t\t\t\t [[maybe_unused]] Camera3RequestDescriptor *request)\n+{\n+\t/*\n+\t * \\todo Stream processing is completed hence, check for errors and\n+\t * if any, mark the corresponding buffer with CAMERA3_BUFFER_STATUS_ERROR.\n+\t */\n+}\n+\n std::string CameraDevice::logPrefix() const\n {\n \treturn \"'\" + camera_->id() + \"'\";\ndiff --git a/src/android/camera_device.h b/src/android/camera_device.h\nindex b2871e52..60c134dc 100644\n--- a/src/android/camera_device.h\n+++ b/src/android/camera_device.h\n@@ -36,6 +36,13 @@\n struct CameraConfigData;\n \n struct Camera3RequestDescriptor {\n+\tenum class ProcessStatus {\n+\t\tNone,\n+\t\tProcessing,\n+\t\tSuccess,\n+\t\tError\n+\t};\n+\n \tCamera3RequestDescriptor() = default;\n \t~Camera3RequestDescriptor() = default;\n \tCamera3RequestDescriptor(libcamera::Camera *camera,\n@@ -48,6 +55,8 @@ struct Camera3RequestDescriptor {\n \tCameraMetadata settings_;\n \tstd::unique_ptr request_;\n \tstd::unique_ptr resultMetadata_;\n+\n+\tProcessStatus status_;\n };\n \n class CameraDevice : protected libcamera::Loggable\n@@ -79,6 +88,8 @@ public:\n \tint configureStreams(camera3_stream_configuration_t *stream_list);\n \tint processCaptureRequest(camera3_capture_request_t *request);\n \tvoid requestComplete(libcamera::Request *request);\n+\tvoid streamProcessingComplete(CameraStream *cameraStream,\n+\t\t\t\t Camera3RequestDescriptor *request);\n \n protected:\n \tstd::string logPrefix() const override;\ndiff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp\nindex d494f5d5..70471959 100644\n--- a/src/android/camera_stream.cpp\n+++ b/src/android/camera_stream.cpp\n@@ -81,6 +81,9 @@ int CameraStream::configure()\n \t\tint ret = postProcessor_->configure(configuration(), output);\n \t\tif (ret)\n \t\t\treturn ret;\n+\n+\t\tpostProcessor_->processComplete.connect(\n+\t\t\tthis, &CameraStream::handleProcessComplete);\n \t}\n \n \tif (allocator_) {\n@@ -102,8 +105,11 @@ int CameraStream::process(const FrameBuffer *source,\n \t\t\t buffer_handle_t camera3Dest,\n \t\t\t Camera3RequestDescriptor *request)\n {\n-\tif (!postProcessor_)\n+\tif (!postProcessor_) {\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\thandleProcessComplete(request);\n \t\treturn 0;\n+\t}\n \n \t/*\n \t * \\todo Buffer mapping and processing should be moved to a\n@@ -114,12 +120,19 @@ int CameraStream::process(const FrameBuffer *source,\n \t\t\t PROT_READ | PROT_WRITE);\n \tif (!dest.isValid()) {\n \t\tLOG(HAL, Error) << \"Failed to map android blob buffer\";\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\thandleProcessComplete(request);\n \t\treturn -EINVAL;\n \t}\n \n \treturn postProcessor_->process(source, &dest, request);\n }\n \n+void CameraStream::handleProcessComplete(Camera3RequestDescriptor *request)\n+{\n+\tcameraDevice_->streamProcessingComplete(this, request);\n+}\n+\n FrameBuffer *CameraStream::getBuffer()\n {\n \tif (!allocator_)\ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\nindex 68789700..f8ad6245 100644\n--- a/src/android/camera_stream.h\n+++ b/src/android/camera_stream.h\n@@ -127,6 +127,8 @@ public:\n \tvoid putBuffer(libcamera::FrameBuffer *buffer);\n \n private:\n+\tvoid handleProcessComplete(Camera3RequestDescriptor *request);\n+\n \tCameraDevice *const cameraDevice_;\n \tconst libcamera::CameraConfiguration *config_;\n \tconst Type type_;\ndiff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\nindex 31f68330..87252acd 100644\n--- a/src/android/jpeg/post_processor_jpeg.cpp\n+++ b/src/android/jpeg/post_processor_jpeg.cpp\n@@ -101,8 +101,11 @@ int PostProcessorJpeg::process(const FrameBuffer *source,\n \t\t\t CameraBuffer *destination,\n \t\t\t Camera3RequestDescriptor *request)\n {\n-\tif (!encoder_)\n+\tif (!encoder_) {\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\tprocessComplete.emit(request);\n \t\treturn 0;\n+\t}\n \n \tASSERT(destination->numPlanes() == 1);\n \n@@ -197,6 +200,8 @@ int PostProcessorJpeg::process(const FrameBuffer *source,\n \t\t\t\t\t exif.data(), quality);\n \tif (jpeg_size < 0) {\n \t\tLOG(JPEG, Error) << \"Failed to encode stream image\";\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\tprocessComplete.emit(request);\n \t\treturn jpeg_size;\n \t}\n \n@@ -211,5 +216,8 @@ int PostProcessorJpeg::process(const FrameBuffer *source,\n \t/* Update the JPEG result Metadata. */\n \tresultMetadata->addEntry(ANDROID_JPEG_SIZE, jpeg_size);\n \n+\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Success;\n+\tprocessComplete.emit(request);\n+\n \treturn 0;\n }\ndiff --git a/src/android/post_processor.h b/src/android/post_processor.h\nindex bdd6afe7..f639b237 100644\n--- a/src/android/post_processor.h\n+++ b/src/android/post_processor.h\n@@ -7,6 +7,8 @@\n #ifndef __ANDROID_POST_PROCESSOR_H__\n #define __ANDROID_POST_PROCESSOR_H__\n \n+#include \n+\n #include \n #include \n \n@@ -26,6 +28,8 @@ public:\n \tvirtual int process(const libcamera::FrameBuffer *source,\n \t\t\t CameraBuffer *destination,\n \t\t\t Camera3RequestDescriptor *request) = 0;\n+\n+\tlibcamera::Signal processComplete;\n };\n \n #endif /* __ANDROID_POST_PROCESSOR_H__ */\ndiff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\nindex 5e18caee..b144649a 100644\n--- a/src/android/yuv/post_processor_yuv.cpp\n+++ b/src/android/yuv/post_processor_yuv.cpp\n@@ -18,6 +18,8 @@\n #include \"libcamera/internal/formats.h\"\n #include \"libcamera/internal/mapped_framebuffer.h\"\n \n+#include \"../camera_device.h\"\n+\n using namespace libcamera;\n \n LOG_DEFINE_CATEGORY(YUV)\n@@ -51,14 +53,19 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n \n int PostProcessorYuv::process(const FrameBuffer *source,\n \t\t\t CameraBuffer *destination,\n-\t\t\t [[maybe_unused]] Camera3RequestDescriptor *request)\n+\t\t\t Camera3RequestDescriptor *request)\n {\n-\tif (!isValidBuffers(source, *destination))\n+\tif (!isValidBuffers(source, *destination)) {\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\tprocessComplete.emit(request);\n \t\treturn -EINVAL;\n+\t}\n \n \tconst MappedFrameBuffer sourceMapped(source, MappedFrameBuffer::MapFlag::Read);\n \tif (!sourceMapped.isValid()) {\n \t\tLOG(YUV, Error) << \"Failed to mmap camera frame buffer\";\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\tprocessComplete.emit(request);\n \t\treturn -EINVAL;\n \t}\n \n@@ -76,9 +83,14 @@ int PostProcessorYuv::process(const FrameBuffer *source,\n \t\t\t\t libyuv::FilterMode::kFilterBilinear);\n \tif (ret) {\n \t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n+\t\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Error;\n+\t\tprocessComplete.emit(request);\n \t\treturn -EINVAL;\n \t}\n \n+\trequest->status_ = Camera3RequestDescriptor::ProcessStatus::Success;\n+\tprocessComplete.emit(request);\n+\n \treturn 0;\n }\n \n","prefixes":["libcamera-devel","v3","07/10"]}