{"id":14306,"url":"https://patchwork.libcamera.org/api/1.1/patches/14306/?format=json","web_url":"https://patchwork.libcamera.org/patch/14306/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20211026072148.164831-5-umang.jain@ideasonboard.com>","date":"2021-10-26T07:21:45","name":"[libcamera-devel,v8,4/7] android: post_processor: Consolidate contextual information","commit_ref":"79cdb1f19d8033cf2e291b284fe1419098d5df81","pull_url":null,"state":"accepted","archived":false,"hash":"c8608c819fa3bf5e734df32f841464df8d2a59ae","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/1.1/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/14306/mbox/","series":[{"id":2654,"url":"https://patchwork.libcamera.org/api/1.1/series/2654/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2654","date":"2021-10-26T07:21:41","name":"Async Post Processor","version":8,"mbox":"https://patchwork.libcamera.org/series/2654/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/14306/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/14306/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 701D0BF415\n\tfor ;\n\tTue, 26 Oct 2021 07:22:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0F5EC64880;\n\tTue, 26 Oct 2021 09:22:06 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 28B246487B\n\tfor ;\n\tTue, 26 Oct 2021 09:22:04 +0200 (CEST)","from perceval.ideasonboard.com (unknown [103.251.226.211])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E37103F0;\n\tTue, 26 Oct 2021 09:22:02 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"AFos+7q0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1635232923;\n\tbh=dikKYI/5rhOzWhBAkOllqX1XkieEm++qNwEEluHEC1c=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=AFos+7q0NzQ8fmO2ds49c/0SgYFmrTlu19Uyzc/cq/kDLzEwCZMt14aZMGjgLlVWv\n\tnYLOYrP5CcPAAEi03DpYVytfYmkFQl35ukDiPOYqxSS8AfA34o6BujeUFkfETKvTG/\n\tdlufmZbwmdbz4D8G1ZHjYfx1z6lrOP2ipA0Vcbss=","From":"Umang Jain ","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 26 Oct 2021 12:51:45 +0530","Message-Id":"<20211026072148.164831-5-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","In-Reply-To":"<20211026072148.164831-1-umang.jain@ideasonboard.com>","References":"<20211026072148.164831-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v8 4/7] android: post_processor:\n\tConsolidate contextual information","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Save and provide the context for post-processor of a camera stream\nvia Camera3RequestDescriptor::StreamBuffer. We extend the structure\nto include source and destination buffers for the post processor, along\nwith CameraStream::Type::Internal buffer pointer (if any). In addition\nto that, a back pointer to Camera3RequestDescriptor is convenient to\nget access to overall descriptor (status, metadata settings etc.).\n\nAlso, migrate CameraStream::process() and PostProcessor::process()\nsignature to use Camera3RequestDescriptor::StreamBuffer only. This\nwill be helpful when we move to async post-processing in subsequent\ncommits.\n\nSigned-off-by: Umang Jain \nReviewed-by: Laurent Pinchart \nReviewed-by: Hirokazu Honda \n---\n src/android/camera_device.cpp | 13 ++++++++-----\n src/android/camera_request.cpp | 5 ++++-\n src/android/camera_request.h | 5 +++++\n src/android/camera_stream.cpp | 23 +++++++++++------------\n src/android/camera_stream.h | 4 +---\n src/android/jpeg/post_processor_jpeg.cpp | 12 +++++++-----\n src/android/jpeg/post_processor_jpeg.h | 4 +---\n src/android/post_processor.h | 7 ++-----\n src/android/yuv/post_processor_yuv.cpp | 7 ++++---\n src/android/yuv/post_processor_yuv.h | 4 +---\n 10 files changed, 44 insertions(+), 40 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex bf9a2e69..9155728a 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -953,6 +953,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \t\t\t * once it has been processed.\n \t\t\t */\n \t\t\tframeBuffer = cameraStream->getBuffer();\n+\t\t\tbuffer.internalBuffer = frameBuffer;\n \t\t\tLOG(HAL, Debug) << ss.str() << \" (internal)\";\n \t\t\tbreak;\n \t\t}\n@@ -1133,14 +1134,16 @@ void CameraDevice::requestComplete(Request *request)\n \t\t\tcontinue;\n \t\t}\n \n-\t\tint ret = stream->process(*src, buffer, descriptor);\n+\t\tbuffer.srcBuffer = src;\n+\n+\t\tint ret = stream->process(&buffer);\n \n \t\t/*\n-\t\t * Return the FrameBuffer to the CameraStream now that we're\n-\t\t * done processing it.\n+\t\t * If the framebuffer is internal to CameraStream return it back\n+\t\t * now that we're done processing it.\n \t\t */\n-\t\tif (stream->type() == CameraStream::Type::Internal)\n-\t\t\tstream->putBuffer(src);\n+\t\tif (buffer.internalBuffer)\n+\t\t\tstream->putBuffer(buffer.internalBuffer);\n \n \t\tif (ret) {\n \t\t\tbuffer.status = Camera3RequestDescriptor::Status::Error;\ndiff --git a/src/android/camera_request.cpp b/src/android/camera_request.cpp\nindex 16cf266f..5bac1b8f 100644\n--- a/src/android/camera_request.cpp\n+++ b/src/android/camera_request.cpp\n@@ -9,6 +9,8 @@\n \n #include \n \n+#include \"camera_buffer.h\"\n+\n using namespace libcamera;\n \n /*\n@@ -36,7 +38,8 @@ Camera3RequestDescriptor::Camera3RequestDescriptor(\n \t\t\tstatic_cast(buffer.stream->priv);\n \n \t\tbuffers_.push_back({ stream, buffer.buffer, nullptr,\n-\t\t\t\t buffer.acquire_fence, Status::Success });\n+\t\t\t\t buffer.acquire_fence, Status::Success,\n+\t\t\t\t nullptr, nullptr, nullptr, this });\n \t}\n \n \t/* Clone the controls associated with the camera3 request. */\ndiff --git a/src/android/camera_request.h b/src/android/camera_request.h\nindex 4d80ef32..ebe2e89d 100644\n--- a/src/android/camera_request.h\n+++ b/src/android/camera_request.h\n@@ -20,6 +20,7 @@\n #include \"camera_metadata.h\"\n #include \"camera_worker.h\"\n \n+class CameraBuffer;\n class CameraStream;\n \n class Camera3RequestDescriptor\n@@ -36,6 +37,10 @@ public:\n \t\tstd::unique_ptr frameBuffer;\n \t\tint fence;\n \t\tStatus status;\n+\t\tlibcamera::FrameBuffer *internalBuffer;\n+\t\tconst libcamera::FrameBuffer *srcBuffer;\n+\t\tstd::unique_ptr dstBuffer;\n+\t\tCamera3RequestDescriptor *request;\n \t};\n \n \tCamera3RequestDescriptor(libcamera::Camera *camera,\ndiff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp\nindex 5d991fe5..282b19b0 100644\n--- a/src/android/camera_stream.cpp\n+++ b/src/android/camera_stream.cpp\n@@ -146,23 +146,21 @@ int CameraStream::waitFence(int fence)\n \treturn -errno;\n }\n \n-int CameraStream::process(const FrameBuffer &source,\n-\t\t\t Camera3RequestDescriptor::StreamBuffer &dest,\n-\t\t\t Camera3RequestDescriptor *request)\n+int CameraStream::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer)\n {\n \tASSERT(type_ != Type::Direct);\n \n \t/* Handle waiting on fences on the destination buffer. */\n-\tif (dest.fence != -1) {\n-\t\tint ret = waitFence(dest.fence);\n+\tif (streamBuffer->fence != -1) {\n+\t\tint ret = waitFence(streamBuffer->fence);\n \t\tif (ret < 0) {\n \t\t\tLOG(HAL, Error) << \"Failed waiting for fence: \"\n-\t\t\t\t\t<< dest.fence << \": \" << strerror(-ret);\n+\t\t\t\t\t<< streamBuffer->fence << \": \" << strerror(-ret);\n \t\t\treturn ret;\n \t\t}\n \n-\t\t::close(dest.fence);\n-\t\tdest.fence = -1;\n+\t\t::close(streamBuffer->fence);\n+\t\tstreamBuffer->fence = -1;\n \t}\n \n \t/*\n@@ -170,14 +168,15 @@ int CameraStream::process(const FrameBuffer &source,\n \t * separate thread.\n \t */\n \tconst StreamConfiguration &output = configuration();\n-\tCameraBuffer destBuffer(*dest.camera3Buffer, output.pixelFormat,\n-\t\t\t\toutput.size, PROT_READ | PROT_WRITE);\n-\tif (!destBuffer.isValid()) {\n+\tstreamBuffer->dstBuffer = std::make_unique(\n+\t\t*streamBuffer->camera3Buffer, output.pixelFormat, output.size,\n+\t\tPROT_READ | PROT_WRITE);\n+\tif (!streamBuffer->dstBuffer->isValid()) {\n \t\tLOG(HAL, Error) << \"Failed to create destination buffer\";\n \t\treturn -EINVAL;\n \t}\n \n-\treturn postProcessor_->process(source, &destBuffer, request);\n+\treturn postProcessor_->process(streamBuffer);\n }\n \n FrameBuffer *CameraStream::getBuffer()\ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\nindex f242336e..e74a9a3b 100644\n--- a/src/android/camera_stream.h\n+++ b/src/android/camera_stream.h\n@@ -121,9 +121,7 @@ public:\n \tlibcamera::Stream *stream() const;\n \n \tint configure();\n-\tint process(const libcamera::FrameBuffer &source,\n-\t\t Camera3RequestDescriptor::StreamBuffer &dest,\n-\t\t Camera3RequestDescriptor *request);\n+\tint process(Camera3RequestDescriptor::StreamBuffer *streamBuffer);\n \tlibcamera::FrameBuffer *getBuffer();\n \tvoid putBuffer(libcamera::FrameBuffer *buffer);\n \ndiff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\nindex 49483836..240e29f6 100644\n--- a/src/android/jpeg/post_processor_jpeg.cpp\n+++ b/src/android/jpeg/post_processor_jpeg.cpp\n@@ -98,15 +98,17 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n \t}\n }\n \n-int PostProcessorJpeg::process(const FrameBuffer &source,\n-\t\t\t CameraBuffer *destination,\n-\t\t\t Camera3RequestDescriptor *request)\n+int PostProcessorJpeg::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer)\n {\n \tASSERT(encoder_);\n+\n+\tconst FrameBuffer &source = *streamBuffer->srcBuffer;\n+\tCameraBuffer *destination = streamBuffer->dstBuffer.get();\n+\n \tASSERT(destination->numPlanes() == 1);\n \n-\tconst CameraMetadata &requestMetadata = request->settings_;\n-\tCameraMetadata *resultMetadata = request->resultMetadata_.get();\n+\tconst CameraMetadata &requestMetadata = streamBuffer->request->settings_;\n+\tCameraMetadata *resultMetadata = streamBuffer->request->resultMetadata_.get();\n \tcamera_metadata_ro_entry_t entry;\n \tint ret;\n \ndiff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h\nindex 0184d77e..92385548 100644\n--- a/src/android/jpeg/post_processor_jpeg.h\n+++ b/src/android/jpeg/post_processor_jpeg.h\n@@ -22,9 +22,7 @@ public:\n \n \tint configure(const libcamera::StreamConfiguration &incfg,\n \t\t const libcamera::StreamConfiguration &outcfg) override;\n-\tint process(const libcamera::FrameBuffer &source,\n-\t\t CameraBuffer *destination,\n-\t\t Camera3RequestDescriptor *request) override;\n+\tint process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override;\n \n private:\n \tvoid generateThumbnail(const libcamera::FrameBuffer &source,\ndiff --git a/src/android/post_processor.h b/src/android/post_processor.h\nindex 27eaef88..128161c8 100644\n--- a/src/android/post_processor.h\n+++ b/src/android/post_processor.h\n@@ -11,8 +11,7 @@\n #include \n \n #include \"camera_buffer.h\"\n-\n-class Camera3RequestDescriptor;\n+#include \"camera_request.h\"\n \n class PostProcessor\n {\n@@ -21,9 +20,7 @@ public:\n \n \tvirtual int configure(const libcamera::StreamConfiguration &inCfg,\n \t\t\t const libcamera::StreamConfiguration &outCfg) = 0;\n-\tvirtual int process(const libcamera::FrameBuffer &source,\n-\t\t\t CameraBuffer *destination,\n-\t\t\t Camera3RequestDescriptor *request) = 0;\n+\tvirtual int process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) = 0;\n };\n \n #endif /* __ANDROID_POST_PROCESSOR_H__ */\ndiff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\nindex 8110a1f1..70385ab3 100644\n--- a/src/android/yuv/post_processor_yuv.cpp\n+++ b/src/android/yuv/post_processor_yuv.cpp\n@@ -49,10 +49,11 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n \treturn 0;\n }\n \n-int PostProcessorYuv::process(const FrameBuffer &source,\n-\t\t\t CameraBuffer *destination,\n-\t\t\t [[maybe_unused]] Camera3RequestDescriptor *request)\n+int PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer)\n {\n+\tconst FrameBuffer &source = *streamBuffer->srcBuffer;\n+\tCameraBuffer *destination = streamBuffer->dstBuffer.get();\n+\n \tif (!isValidBuffers(source, *destination))\n \t\treturn -EINVAL;\n \ndiff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h\nindex a4e0ff5d..5954e11b 100644\n--- a/src/android/yuv/post_processor_yuv.h\n+++ b/src/android/yuv/post_processor_yuv.h\n@@ -18,9 +18,7 @@ public:\n \n \tint configure(const libcamera::StreamConfiguration &incfg,\n \t\t const libcamera::StreamConfiguration &outcfg) override;\n-\tint process(const libcamera::FrameBuffer &source,\n-\t\t CameraBuffer *destination,\n-\t\t Camera3RequestDescriptor *request) override;\n+\tint process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) override;\n \n private:\n \tbool isValidBuffers(const libcamera::FrameBuffer &source,\n","prefixes":["libcamera-devel","v8","4/7"]}