{"id":13966,"url":"https://patchwork.libcamera.org/api/1.1/patches/13966/?format=json","web_url":"https://patchwork.libcamera.org/patch/13966/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210928122515.28034-2-jacopo@jmondi.org>","date":"2021-09-28T12:25:15","name":"[libcamera-devel,v3,1/1] android: Wait on fences in CameraStream::process()","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"4062342b5124497ee4c1bbc114fbc97b57410a85","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/1.1/people/3/?format=json","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/13966/mbox/","series":[{"id":2580,"url":"https://patchwork.libcamera.org/api/1.1/series/2580/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2580","date":"2021-09-28T12:25:14","name":"android: Wait on acquisition fences in CameraStream","version":3,"mbox":"https://patchwork.libcamera.org/series/2580/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13966/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13966/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1B8D1C3243\n\tfor ;\n\tTue, 28 Sep 2021 12:24:37 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E96B06918C;\n\tTue, 28 Sep 2021 14:24:36 +0200 (CEST)","from relay12.mail.gandi.net (relay12.mail.gandi.net\n\t[217.70.178.232])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C4ABA69186\n\tfor ;\n\tTue, 28 Sep 2021 14:24:33 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby relay12.mail.gandi.net (Postfix) with ESMTPSA id 20248200002;\n\tTue, 28 Sep 2021 12:24:32 +0000 (UTC)"],"From":"Jacopo Mondi ","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 28 Sep 2021 14:25:15 +0200","Message-Id":"<20210928122515.28034-2-jacopo@jmondi.org>","X-Mailer":"git-send-email 2.32.0","In-Reply-To":"<20210928122515.28034-1-jacopo@jmondi.org>","References":"<20210928122515.28034-1-jacopo@jmondi.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 1/1] android: Wait on fences in\n\tCameraStream::process()","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Acquire fences for streams of type Mapped generated by\npost-processing are not correctly handled and are currently\nignored by the camera HAL.\n\nFix this by adding CameraStream::waitFence(), executed before\nstarting the post-processing in CameraStream::process().\n\nThe change applies to all streams generated by post-processing (Mapped\nand Internal) but currently acquire fences of Internal streams are\nhandled by the camera worker. Post-pone that to post-processing time by\npassing -1 to the Worker for Internal streams.\n\nAlso correct the release_fence handling for failed captures, as the\nframework requires the release fences to be set to the acquire fence\nvalue if the acquire fence has not been waited on.\n\nSigned-off-by: Jacopo Mondi \nReviewed-by: Hirokazu Honda \n---\n src/android/camera_device.cpp | 39 +++++++++++++++++++++++----\n src/android/camera_stream.cpp | 50 +++++++++++++++++++++++++++++++++--\n src/android/camera_stream.h | 4 ++-\n 3 files changed, 85 insertions(+), 8 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex ab38116800cd..0fe11fe833b0 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -983,9 +983,13 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \n \t\t/*\n \t\t * Inspect the camera stream type, create buffers opportunely\n-\t\t * and add them to the Request if required.\n+\t\t * and add them to the Request if required. Only acquire fences\n+\t\t * for streams of type Direct are handled by the CameraWorker,\n+\t\t * while fences for streams of type Internal and Mapped are\n+\t\t * handled at post-processing time.\n \t\t */\n \t\tFrameBuffer *buffer = nullptr;\n+\t\tint acquireFence = -1;\n \t\tswitch (cameraStream->type()) {\n \t\tcase CameraStream::Type::Mapped:\n \t\t\t/*\n@@ -1008,6 +1012,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \t\t\t\t\t\t cameraStream->configuration().size));\n \n \t\t\tbuffer = descriptor.frameBuffers_.back().get();\n+\t\t\tacquireFence = camera3Buffer.acquire_fence;\n \t\t\tLOG(HAL, Debug) << ss.str() << \" (direct)\";\n \t\t\tbreak;\n \n@@ -1030,7 +1035,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n \t\t}\n \n \t\tdescriptor.request_->addBuffer(cameraStream->stream(), buffer,\n-\t\t\t\t\t\tcamera3Buffer.acquire_fence);\n+\t\t\t\t\t acquireFence);\n \t}\n \n \t/*\n@@ -1110,7 +1115,22 @@ void CameraDevice::requestComplete(Request *request)\n \tcaptureResult.frame_number = descriptor.frameNumber_;\n \tcaptureResult.num_output_buffers = descriptor.buffers_.size();\n \tfor (camera3_stream_buffer_t &buffer : descriptor.buffers_) {\n-\t\tbuffer.acquire_fence = -1;\n+\t\tCameraStream *cameraStream =\n+\t\t\tstatic_cast(buffer.stream->priv);\n+\n+\t\t/*\n+\t\t * Streams of type Direct have been queued to the\n+\t\t * libcamera::Camera and their acquisition fences have\n+\t\t * already been waited on by the CameraWorker.\n+\t\t *\n+\t\t * Acquire fences of streams of type Internal and Mapped\n+\t\t * will be handled during post-processing.\n+\t\t *\n+\t\t * \\todo Instrument the CameraWorker to set the acquire\n+\t\t * fence to -1 once it has handled it and remove this check.\n+\t\t */\n+\t\tif (cameraStream->type() == CameraStream::Type::Direct)\n+\t\t\tbuffer.acquire_fence = -1;\n \t\tbuffer.release_fence = -1;\n \t\tbuffer.status = CAMERA3_BUFFER_STATUS_OK;\n \t}\n@@ -1130,8 +1150,17 @@ void CameraDevice::requestComplete(Request *request)\n \t\t\t CAMERA3_MSG_ERROR_REQUEST);\n \n \t\tcaptureResult.partial_result = 0;\n-\t\tfor (camera3_stream_buffer_t &buffer : descriptor.buffers_)\n+\t\tfor (camera3_stream_buffer_t &buffer : descriptor.buffers_) {\n+\t\t\t/*\n+\t\t\t * Signal to the framework it has to handle fences that\n+\t\t\t * have not been waited on by setting the release fence\n+\t\t\t * to the acquire fence value.\n+\t\t\t */\n+\t\t\tbuffer.release_fence = buffer.acquire_fence;\n+\t\t\tbuffer.acquire_fence = -1;\n \t\t\tbuffer.status = CAMERA3_BUFFER_STATUS_ERROR;\n+\t\t}\n+\n \t\tcallbacks_->process_capture_result(callbacks_, &captureResult);\n \n \t\treturn;\n@@ -1181,7 +1210,7 @@ void CameraDevice::requestComplete(Request *request)\n \t\t\tcontinue;\n \t\t}\n \n-\t\tint ret = cameraStream->process(*src, *buffer.buffer,\n+\t\tint ret = cameraStream->process(*src, buffer,\n \t\t\t\t\t\tdescriptor.settings_,\n \t\t\t\t\t\tresultMetadata.get());\n \t\t/*\ndiff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp\nindex e30c7ee4fcb3..2363985398fb 100644\n--- a/src/android/camera_stream.cpp\n+++ b/src/android/camera_stream.cpp\n@@ -7,7 +7,11 @@\n \n #include \"camera_stream.h\"\n \n+#include \n+#include \n #include \n+#include \n+#include \n \n #include \n \n@@ -109,11 +113,53 @@ int CameraStream::configure()\n \treturn 0;\n }\n \n+int CameraStream::waitFence(int fence)\n+{\n+\t/*\n+\t * \\todo The implementation here is copied from camera_worker.cpp\n+\t * and both should be removed once libcamera is instrumented to handle\n+\t * fences waiting in the core.\n+\t *\n+\t * \\todo Better characterize the timeout. Currently equal to the one\n+\t * used by the Rockchip Camera HAL on ChromeOS.\n+\t */\n+\tconstexpr unsigned int timeoutMs = 300;\n+\tstruct pollfd fds = { fence, POLLIN, 0 };\n+\n+\tdo {\n+\t\tint ret = poll(&fds, 1, timeoutMs);\n+\t\tif (ret == 0)\n+\t\t\treturn -ETIME;\n+\n+\t\tif (ret > 0) {\n+\t\t\tif (fds.revents & (POLLERR | POLLNVAL))\n+\t\t\t\treturn -EINVAL;\n+\n+\t\t\treturn 0;\n+\t\t}\n+\t} while (errno == EINTR || errno == EAGAIN);\n+\n+\treturn -errno;\n+}\n+\n int CameraStream::process(const FrameBuffer &source,\n-\t\t\t buffer_handle_t camera3Dest,\n+\t\t\t camera3_stream_buffer_t &camera3Buffer,\n \t\t\t const CameraMetadata &requestMetadata,\n \t\t\t CameraMetadata *resultMetadata)\n {\n+\t/* Handle waiting on fences on the destination buffer. */\n+\tint fence = camera3Buffer.acquire_fence;\n+\tif (fence != -1) {\n+\t\tint ret = waitFence(fence);\n+\t\t::close(fence);\n+\t\tcamera3Buffer.acquire_fence = -1;\n+\t\tif (ret < 0) {\n+\t\t\tLOG(HAL, Error) << \"Failed waiting for fence: \"\n+\t\t\t\t\t<< fence << \": \" << strerror(-ret);\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\n \tif (!postProcessor_)\n \t\treturn 0;\n \n@@ -122,7 +168,7 @@ int CameraStream::process(const FrameBuffer &source,\n \t * separate thread.\n \t */\n \tconst StreamConfiguration &output = configuration();\n-\tCameraBuffer dest(camera3Dest, formats::MJPEG, output.size,\n+\tCameraBuffer dest(*camera3Buffer.buffer, formats::MJPEG, output.size,\n \t\t\t PROT_READ | PROT_WRITE);\n \tif (!dest.isValid()) {\n \t\tLOG(HAL, Error) << \"Failed to map android blob buffer\";\ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\nindex 2dab6c3a37d1..03ecfa94fc8c 100644\n--- a/src/android/camera_stream.h\n+++ b/src/android/camera_stream.h\n@@ -119,13 +119,15 @@ public:\n \n \tint configure();\n \tint process(const libcamera::FrameBuffer &source,\n-\t\t buffer_handle_t camera3Dest,\n+\t\t camera3_stream_buffer_t &camera3Buffer,\n \t\t const CameraMetadata &requestMetadata,\n \t\t CameraMetadata *resultMetadata);\n \tlibcamera::FrameBuffer *getBuffer();\n \tvoid putBuffer(libcamera::FrameBuffer *buffer);\n \n private:\n+\tint waitFence(int fence);\n+\n \tCameraDevice *const cameraDevice_;\n \tconst libcamera::CameraConfiguration *config_;\n \tconst Type type_;\n","prefixes":["libcamera-devel","v3","1/1"]}