{"id":4137,"url":"https://patchwork.libcamera.org/api/1.1/patches/4137/?format=json","web_url":"https://patchwork.libcamera.org/patch/4137/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200623020930.1781469-11-niklas.soderlund@ragnatech.se>","date":"2020-06-23T02:09:30","name":"[libcamera-devel,v3,10/10] libcamera: ipu3: Allow zero-copy RAW stream capture","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"5f1c8240e47000b05668826b914e033b1bf6344b","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/1.1/people/5/?format=json","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/4137/mbox/","series":[{"id":1027,"url":"https://patchwork.libcamera.org/api/1.1/series/1027/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1027","date":"2020-06-23T02:09:20","name":"libcamera: ipu3: Allow zero-copy RAW stream","version":3,"mbox":"https://patchwork.libcamera.org/series/1027/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/4137/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/4137/checks/","tags":{},"headers":{"Return-Path":"","Received":["from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net\n\t[195.74.38.227])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 69500603BF\n\tfor ;\n\tTue, 23 Jun 2020 04:09:50 +0200 (CEST)","from bismarck.berto.se (p4fca2eca.dip0.t-ipconnect.de\n\t[79.202.46.202]) by bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA\n\tid 9dd3b906-b4f6-11ea-933e-005056917a89;\n\tTue, 23 Jun 2020 04:09:49 +0200 (CEST)"],"X-Halon-ID":"9dd3b906-b4f6-11ea-933e-005056917a89","Authorized-sender":"niklas@soderlund.pp.se","From":"=?utf-8?q?Niklas_S=C3=B6derlund?= ","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 23 Jun 2020 04:09:30 +0200","Message-Id":"<20200623020930.1781469-11-niklas.soderlund@ragnatech.se>","X-Mailer":"git-send-email 2.27.0","In-Reply-To":"<20200623020930.1781469-1-niklas.soderlund@ragnatech.se>","References":"<20200623020930.1781469-1-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 10/10] libcamera: ipu3: Allow zero-copy\n\tRAW stream capture","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Tue, 23 Jun 2020 02:09:50 -0000"},"content":"With the refactored CIO2 interface it's now easy to add zero-copy for\nbuffers in the RAW stream. Use the internally allocated buffers inside\nthe CIO2Device if no buffer for the RAW stream is provided by the\napplication, or use the application-provided buffer if any.\n\nSigned-off-by: Niklas Söderlund \nReviewed-by: Jacopo Mondi \n---\n* Changes since v2\n- Add todo of potential common need code.\n\n* Changes since v1\n- Update comments.\n- Use Request::findBuffer() instead of own logic.\n---\n src/libcamera/pipeline/ipu3/cio2.cpp | 58 ++++++++++++++---------\n src/libcamera/pipeline/ipu3/cio2.h | 7 ++-\n src/libcamera/pipeline/ipu3/ipu3.cpp | 69 +++++++++-------------------\n 3 files changed, 61 insertions(+), 73 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\nindex efaa460b246697a6..ab7f96dad19b0560 100644\n--- a/src/libcamera/pipeline/ipu3/cio2.cpp\n+++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n@@ -213,31 +213,16 @@ int CIO2Device::exportBuffers(unsigned int count,\n \treturn output_->exportBuffers(count, buffers);\n }\n \n-FrameBuffer *CIO2Device::getBuffer()\n-{\n-\tif (availableBuffers_.empty()) {\n-\t\tLOG(IPU3, Error) << \"CIO2 buffer underrun\";\n-\t\treturn nullptr;\n-\t}\n-\n-\tFrameBuffer *buffer = availableBuffers_.front();\n-\n-\tavailableBuffers_.pop();\n-\n-\treturn buffer;\n-}\n-\n-void CIO2Device::putBuffer(FrameBuffer *buffer)\n-{\n-\tavailableBuffers_.push(buffer);\n-}\n-\n int CIO2Device::start()\n {\n-\tint ret = output_->allocateBuffers(CIO2_BUFFER_COUNT, &buffers_);\n+\tint ret = output_->exportBuffers(CIO2_BUFFER_COUNT, &buffers_);\n \tif (ret < 0)\n \t\treturn ret;\n \n+\tret = output_->importBuffers(CIO2_BUFFER_COUNT);\n+\tif (ret)\n+\t\tLOG(IPU3, Error) << \"Failed to import CIO2 buffers\";\n+\n \tfor (std::unique_ptr &buffer : buffers_)\n \t\tavailableBuffers_.push(buffer.get());\n \n@@ -269,11 +254,42 @@ int CIO2Device::stop()\n \treturn ret;\n }\n \n-int CIO2Device::queueBuffer(FrameBuffer *buffer)\n+int CIO2Device::queueBuffer(Request *request, FrameBuffer *rawBuffer)\n {\n+\tFrameBuffer *buffer = rawBuffer;\n+\n+\t/* If no buffer is provided in the request, use an internal one. */\n+\tif (!buffer) {\n+\t\tif (availableBuffers_.empty()) {\n+\t\t\tLOG(IPU3, Error) << \"CIO2 buffer underrun\";\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tbuffer = availableBuffers_.front();\n+\t\tavailableBuffers_.pop();\n+\t}\n+\n+\tbuffer->setRequest(request);\n+\n \treturn output_->queueBuffer(buffer);\n }\n \n+void CIO2Device::tryReturnBuffer(FrameBuffer *buffer)\n+{\n+\t/*\n+\t * \\todo Once more pipelines deal with buffers that may be allocated\n+\t * internally or externally at the this pattern might become a common\n+\t * need. At that point this check should be moved to something clever\n+\t * in FrameBuffer.\n+\t */\n+\tfor (const std::unique_ptr &buf : buffers_) {\n+\t\tif (buf.get() == buffer) {\n+\t\t\tavailableBuffers_.push(buffer);\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n void CIO2Device::cio2BufferReady(FrameBuffer *buffer)\n {\n \tbufferReady.emit(buffer);\ndiff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\nindex 405e6c03755367c4..9f2e8a98af7043eb 100644\n--- a/src/libcamera/pipeline/ipu3/cio2.h\n+++ b/src/libcamera/pipeline/ipu3/cio2.h\n@@ -18,6 +18,7 @@ namespace libcamera {\n class CameraSensor;\n class FrameBuffer;\n class MediaDevice;\n+class Request;\n class V4L2DeviceFormat;\n class V4L2Subdevice;\n class V4L2VideoDevice;\n@@ -40,15 +41,13 @@ public:\n \tint exportBuffers(unsigned int count,\n \t\t\t std::vector> *buffers);\n \n-\tFrameBuffer *getBuffer();\n-\tvoid putBuffer(FrameBuffer *buffer);\n-\n \tint start();\n \tint stop();\n \n \tCameraSensor *sensor() { return sensor_; }\n \n-\tint queueBuffer(FrameBuffer *buffer);\n+\tint queueBuffer(Request *request, FrameBuffer *rawBuffer);\n+\tvoid tryReturnBuffer(FrameBuffer *buffer);\n \tSignal bufferReady;\n \n private:\ndiff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex 4818027e8db1f7a3..6c43169eb0915965 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -122,7 +122,6 @@ public:\n \t}\n \n \tvoid imguOutputBufferReady(FrameBuffer *buffer);\n-\tvoid imguInputBufferReady(FrameBuffer *buffer);\n \tvoid cio2BufferReady(FrameBuffer *buffer);\n \n \tCIO2Device cio2_;\n@@ -737,25 +736,24 @@ void PipelineHandlerIPU3::stop(Camera *camera)\n int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request)\n {\n \tIPU3CameraData *data = cameraData(camera);\n-\tFrameBuffer *buffer;\n \tint error = 0;\n \n-\t/* Get a CIO2 buffer, associate it with the request and queue it. */\n-\tbuffer = data->cio2_.getBuffer();\n-\tif (!buffer)\n-\t\treturn -EINVAL;\n-\n-\tbuffer->setRequest(request);\n-\tdata->cio2_.queueBuffer(buffer);\n+\t/*\n+\t * Queue a buffer on the CIO2, using the raw stream buffer provided in\n+\t * the request, if any, or a CIO2 internal buffer otherwise.\n+\t */\n+\tFrameBuffer *rawBuffer = request->findBuffer(&data->rawStream_);\n+\terror = data->cio2_.queueBuffer(request, rawBuffer);\n+\tif (error)\n+\t\treturn error;\n \n+\t/* Queue all buffers from the request aimed for the ImgU. */\n \tfor (auto it : request->buffers()) {\n \t\tIPU3Stream *stream = static_cast(it.first);\n-\t\tbuffer = it.second;\n-\n-\t\t/* Skip raw streams, they are copied from the CIO2 buffer. */\n \t\tif (stream->raw_)\n \t\t\tcontinue;\n \n+\t\tFrameBuffer *buffer = it.second;\n \t\tint ret = stream->device_->dev->queueBuffer(buffer);\n \t\tif (ret < 0)\n \t\t\terror = ret;\n@@ -885,8 +883,8 @@ int PipelineHandlerIPU3::registerCameras()\n \t\t */\n \t\tdata->cio2_.bufferReady.connect(data.get(),\n \t\t\t\t\t&IPU3CameraData::cio2BufferReady);\n-\t\tdata->imgu_->input_->bufferReady.connect(data.get(),\n-\t\t\t\t\t&IPU3CameraData::imguInputBufferReady);\n+\t\tdata->imgu_->input_->bufferReady.connect(&data->cio2_,\n+\t\t\t\t\t&CIO2Device::tryReturnBuffer);\n \t\tdata->imgu_->output_.dev->bufferReady.connect(data.get(),\n \t\t\t\t\t&IPU3CameraData::imguOutputBufferReady);\n \t\tdata->imgu_->viewfinder_.dev->bufferReady.connect(data.get(),\n@@ -915,22 +913,6 @@ int PipelineHandlerIPU3::registerCameras()\n * Buffer Ready slots\n */\n \n-/**\n- * \\brief Handle buffers completion at the ImgU input\n- * \\param[in] buffer The completed buffer\n- *\n- * Buffers completed from the ImgU input are immediately queued back to the\n- * CIO2 unit to continue frame capture.\n- */\n-void IPU3CameraData::imguInputBufferReady(FrameBuffer *buffer)\n-{\n-\t/* \\todo Handle buffer failures when state is set to BufferError. */\n-\tif (buffer->metadata().status == FrameMetadata::FrameCancelled)\n-\t\treturn;\n-\n-\tcio2_.putBuffer(buffer);\n-}\n-\n /**\n * \\brief Handle buffers completion at the ImgU output\n * \\param[in] buffer The completed buffer\n@@ -963,27 +945,18 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n \t\treturn;\n \n \tRequest *request = buffer->request();\n-\tFrameBuffer *raw = request->findBuffer(&rawStream_);\n \n-\tif (!raw) {\n-\t\t/* No RAW buffers present, just queue to IMGU. */\n-\t\timgu_->input_->queueBuffer(buffer);\n-\t\treturn;\n-\t}\n-\n-\t/* RAW buffers present, special care is needed. */\n-\tif (request->buffers().size() > 1)\n-\t\timgu_->input_->queueBuffer(buffer);\n-\n-\tif (raw->copyFrom(buffer))\n-\t\tLOG(IPU3, Debug) << \"Copy of FrameBuffer failed\";\n-\n-\tpipe_->completeBuffer(camera_, request, raw);\n-\n-\tif (request->buffers().size() == 1) {\n-\t\tcio2_.putBuffer(buffer);\n+\t/*\n+\t * If the request contains a buffer for the RAW stream only, complete it\n+\t * now as there's no need for ImgU processing.\n+\t */\n+\tif (request->findBuffer(&rawStream_) &&\n+\t pipe_->completeBuffer(camera_, request, buffer)) {\n \t\tpipe_->completeRequest(camera_, request);\n+\t\treturn;\n \t}\n+\n+\timgu_->input_->queueBuffer(buffer);\n }\n \n /* -----------------------------------------------------------------------------\n","prefixes":["libcamera-devel","v3","10/10"]}