{"id":8848,"url":"https://patchwork.libcamera.org/api/patches/8848/?format=json","web_url":"https://patchwork.libcamera.org/patch/8848/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200717085410.732308-11-naush@raspberrypi.com>","date":"2020-07-17T08:54:10","name":"[libcamera-devel,v3,10/10] libcamera: pipeline: raspberrypi: Handle any externally allocated FrameBuffer","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"3f95fdd004ea9aa1360157903614443b622aad94","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/8848/mbox/","series":[{"id":1114,"url":"https://patchwork.libcamera.org/api/series/1114/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1114","date":"2020-07-17T08:54:00","name":"Zero-copy RAW stream work","version":3,"mbox":"https://patchwork.libcamera.org/series/1114/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/8848/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/8848/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 70071C0109\n\tfor ;\n\tFri, 17 Jul 2020 08:54:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 3352F6053A;\n\tFri, 17 Jul 2020 10:54:28 +0200 (CEST)","from mail-wm1-x333.google.com (mail-wm1-x333.google.com\n\t[IPv6:2a00:1450:4864:20::333])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DAF46054C\n\tfor ;\n\tFri, 17 Jul 2020 10:54:25 +0200 (CEST)","by mail-wm1-x333.google.com with SMTP id f139so16039115wmf.5\n\tfor ;\n\tFri, 17 Jul 2020 01:54:25 -0700 (PDT)","from naushir-VirtualBox.patuck.local ([88.97.76.4])\n\tby smtp.gmail.com with ESMTPSA id\n\t92sm14582375wrr.96.2020.07.17.01.54.23\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 17 Jul 2020 01:54:23 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"nRbqjsSx\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=YqGfCQ7TR779z8O5CQTjcZsNXHDUsTejqHSEgFaeGIU=;\n\tb=nRbqjsSxf+Dp3HjgLjBCK7krXVWbBXF9bwXqvzDZJfOpBtunFqju2Cccknd4PLgIwe\n\tMLLlQTNTM3hmTmZBXv031DgVLhTToJcBmGrAzBrar649cydDqXSG+tGjv9uFbLVs22wE\n\t2cOeXQJdjMyCtpg7wcyj10HE0S0uKYrxdiK2Bs43b+xe01pppejpvRgTpdJg85bnjGMP\n\t8h18wLkOAywvgWvsq7O4n/5Se6jAXHBjo4kGFeaWZ+A05C9Sa1VL8f9mKzgQnsZFlzZP\n\t4581NeoSb597xHm54MNinaeHvcZvPBmXZ59bKQ96Xw0rWJwVT0gpBDLjb4uDvKJ7PfAP\n\tB9qA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=YqGfCQ7TR779z8O5CQTjcZsNXHDUsTejqHSEgFaeGIU=;\n\tb=qI7zeCuvtZ9RiPCIzAY4FVdaTCa0TAAFYpYcZvdg35QaNWFxZqhAKO2lVcxQeikqGS\n\tcSWEKQPRASnE+cPtfdFlbI9Uq3o6iKudlIyA8KPnXeaT3W2hZNtQfFEOX9WrmvVfB3CB\n\tw3gbkqEmQ1gRsNdCpfWxj89SVoCaZNj+72ShEbmAE/jA1ZQlm8f+Hf3JwmQIPyb/QiOQ\n\teLZxZY8Mk58HU6oz5B8R5r7vym8ziomGmGQJMWD3u/z90orjXly/6NuEGHeeoB0k3eb2\n\teldIc/81fBKTNXnspy0hcPAjMI27Bp0H6XMNakoQVuUFMihItKfWZgKtZG+irzBb8E+u\n\t/6Xg==","X-Gm-Message-State":"AOAM533LZmsZJ3boWjQXI0aKBUkcpMzK3+S+jHBxIPWzcE9HJcddKQKx\n\taAD04lTFefkCtbHCcgtkqXmVSS3XdWQ=","X-Google-Smtp-Source":"ABdhPJzT3qtvlOvtMtyhBrYRq6x6jD0AE3t5OlfmW2cwb5mSezoVWIZW/EC9opQgS/OAlxl2/PYaiQ==","X-Received":"by 2002:a1c:7510:: with SMTP id\n\to16mr7734154wmc.146.1594976064398; \n\tFri, 17 Jul 2020 01:54:24 -0700 (PDT)","From":"Naushir Patuck ","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 17 Jul 2020 09:54:10 +0100","Message-Id":"<20200717085410.732308-11-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.25.1","In-Reply-To":"<20200717085410.732308-1-naush@raspberrypi.com>","References":"<20200717085410.732308-1-naush@raspberrypi.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v3 10/10] libcamera: pipeline:\n\traspberrypi: Handle any externally allocated FrameBuffer","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Handle the case where a FrameBuffer that has been externally allocated\n(i.e. not through the v4l2 video device) is passed into a Request.\n\nWe must store the buffer pointer in our internal buffer list to identify\nwhen used, as well as mmap the buffer in the IPA if needed.\n\nSigned-off-by: Naushir Patuck \n---\n .../pipeline/raspberrypi/raspberrypi.cpp | 94 +++++++++++--------\n .../pipeline/raspberrypi/rpi_stream.cpp | 5 +\n .../pipeline/raspberrypi/rpi_stream.h | 1 +\n 3 files changed, 63 insertions(+), 37 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\nindex e400c10c..99896eee 100644\n--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n@@ -171,8 +171,8 @@ public:\n \tRPi::RPiDevice isp_;\n \t/* The vector below is just for convenience when iterating over all streams. */\n \tstd::vector streams_;\n-\t/* Buffers passed to the IPA. */\n-\tstd::vector ipaBuffers_;\n+\t/* Stores the ids of the buffers mapped in the IPA. */\n+\tstd::vector ipaBufferIds_;\n \n \t/* VCSM allocation helper. */\n \t::RPi::Vcsm vcsm_;\n@@ -192,7 +192,6 @@ public:\n \tstd::queue bayerQueue_;\n \tstd::queue embeddedQueue_;\n \tstd::deque requestQueue_;\n-\n \tunsigned int dropFrameCount_;\n \n private:\n@@ -243,6 +242,8 @@ private:\n \tint queueAllBuffers(Camera *camera);\n \tint prepareBuffers(Camera *camera);\n \tvoid freeBuffers(Camera *camera);\n+\tvoid mapBuffers(Camera *camera, const std::vector &buffers,\n+\t\t\tunsigned int startId);\n \n \tMediaDevice *unicam_;\n \tMediaDevice *isp_;\n@@ -736,20 +737,38 @@ int PipelineHandlerRPi::queueRequestDevice(Camera *camera, Request *request)\n \n \t/* Push all buffers supplied in the Request to the respective streams. */\n \tfor (auto stream : data->streams_) {\n-\t\tif (stream->isExternal()) {\n-\t\t\tFrameBuffer *buffer = request->findBuffer(stream);\n+\t\tif (!stream->isExternal())\n+\t\t\tcontinue;\n+\n+\t\tFrameBuffer *buffer = request->findBuffer(stream);\n+\t\tif (buffer && stream->getBufferIndex(buffer) == -1) {\n \t\t\t/*\n-\t\t\t * A nullptr in buffer means that we should queue an internally\n-\t\t\t * allocated buffer.\n-\t\t\t *\n-\t\t\t * The below queueBuffer() call will do nothing if there are not\n-\t\t\t * enough internal buffers allocated, but this will be handled by\n-\t\t\t * queuing the request for buffers in the RPiStream object.\n+\t\t\t * This buffer is not recognised, so it must have been allocated\n+\t\t\t * outside the v4l2 device. Store it in the stream buffer list\n+\t\t\t * so we can track it.\n \t\t\t */\n-\t\t\tint ret = stream->queueBuffer(buffer);\n-\t\t\tif (ret)\n-\t\t\t\treturn ret;\n+\t\t\tstream->setExternalBuffer(buffer);\n+\n+\t\t\t/* Also get the IPA to mmap it if needed. */\n+\t\t\tif (stream == &data->unicam_[Unicam::Embedded]) {\n+\t\t\t\tmapBuffers(camera, { buffer },\n+\t\t\t\t\t RPiIpaMask::EMBEDDED_DATA | (stream->getBuffers().size() - 1));\n+\t\t\t} else if (stream == &data->isp_[Isp::Stats]) {\n+\t\t\t\tmapBuffers(camera, { buffer },\n+\t\t\t\t\t RPiIpaMask::STATS | (stream->getBuffers().size() - 1));\n+\t\t\t}\n \t\t}\n+\t\t/*\n+\t\t * A nullptr in buffer means that we should queue an internally\n+\t\t * allocated buffer.\n+\t\t *\n+\t\t * The below queueBuffer() call will do nothing if there are not\n+\t\t * enough internal buffers allocated, but this will be handled by\n+\t\t * queuing the request for buffers in the RPiStream object.\n+\t\t */\n+\t\tint ret = stream->queueBuffer(buffer);\n+\t\tif (ret)\n+\t\t\treturn ret;\n \t}\n \n \t/* Push the request to the back of the queue. */\n@@ -888,7 +907,6 @@ int PipelineHandlerRPi::queueAllBuffers(Camera *camera)\n int PipelineHandlerRPi::prepareBuffers(Camera *camera)\n {\n \tRPiCameraData *data = cameraData(camera);\n-\tunsigned int index;\n \tint ret;\n \n \t/*\n@@ -908,42 +926,44 @@ int PipelineHandlerRPi::prepareBuffers(Camera *camera)\n \t\t\treturn ret;\n \t}\n \n+\t/*\n+\t * Pass the stats and embedded data buffers to the IPA. No other\n+\t * buffers need to be passed.\n+\t */\n+\tmapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPiIpaMask::STATS);\n+\tmapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPiIpaMask::EMBEDDED_DATA);\n+\n+\treturn 0;\n+}\n+\n+void PipelineHandlerRPi::mapBuffers(Camera *camera, const std::vector &buffers,\n+\t\t\t\t unsigned int startId)\n+{\n+\tRPiCameraData *data = cameraData(camera);\n+\tstd::vector ipaBuffers;\n+\tunsigned int id = startId;\n \t/*\n \t * Link the FrameBuffers with the index of their position in the vector\n-\t * stored in the RPi stream object.\n+\t * stored in the RPi stream object - along with an identifer mask.\n \t *\n \t * This will allow us to identify buffers passed between the pipeline\n \t * handler and the IPA.\n \t */\n-\tindex = 0;\n-\tfor (auto const &b : data->isp_[Isp::Stats].getBuffers()) {\n-\t\tdata->ipaBuffers_.push_back({ .id = RPiIpaMask::STATS | index,\n-\t\t\t\t\t .planes = b->planes() });\n-\t\tindex++;\n-\t}\n-\n-\tindex = 0;\n-\tfor (auto const &b : data->unicam_[Unicam::Embedded].getBuffers()) {\n-\t\tdata->ipaBuffers_.push_back({ .id = RPiIpaMask::EMBEDDED_DATA | index,\n-\t\t\t\t\t .planes = b->planes() });\n-\t\tindex++;\n+\tfor (auto const &b : buffers) {\n+\t\tipaBuffers.push_back({ .id = id, .planes = b->planes() });\n+\t\tdata->ipaBufferIds_.push_back(id);\n+\t\tid++;\n \t}\n \n-\tdata->ipa_->mapBuffers(data->ipaBuffers_);\n-\n-\treturn 0;\n+\tdata->ipa_->mapBuffers(ipaBuffers);\n }\n \n void PipelineHandlerRPi::freeBuffers(Camera *camera)\n {\n \tRPiCameraData *data = cameraData(camera);\n \n-\tstd::vector ids;\n-\tfor (IPABuffer &ipabuf : data->ipaBuffers_)\n-\t\tids.push_back(ipabuf.id);\n-\n-\tdata->ipa_->unmapBuffers(ids);\n-\tdata->ipaBuffers_.clear();\n+\tdata->ipa_->unmapBuffers(data->ipaBufferIds_);\n+\tdata->ipaBufferIds_.clear();\n \n \tfor (auto const stream : data->streams_)\n \t\tstream->releaseBuffers();\ndiff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp\nindex 61226e94..e4158e3a 100644\n--- a/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.cpp\n@@ -48,6 +48,11 @@ void RPiStream::setExportedBuffers(std::vector> *bu\n \t\t [](std::unique_ptr &b) { return b.get(); });\n }\n \n+void RPiStream::setExternalBuffer(FrameBuffer *buffer)\n+{\n+\tbufferList_.push_back(buffer);\n+}\n+\n const std::vector &RPiStream::getBuffers() const\n {\n \treturn bufferList_;\ndiff --git a/src/libcamera/pipeline/raspberrypi/rpi_stream.h b/src/libcamera/pipeline/raspberrypi/rpi_stream.h\nindex a6fd5c8e..e2534a32 100644\n--- a/src/libcamera/pipeline/raspberrypi/rpi_stream.h\n+++ b/src/libcamera/pipeline/raspberrypi/rpi_stream.h\n@@ -41,6 +41,7 @@ public:\n \tvoid reset();\n \tstd::string name() const;\n \tvoid setExportedBuffers(std::vector> *buffers);\n+\tvoid setExternalBuffer(FrameBuffer *buffer);\n \tconst std::vector &getBuffers() const;\n \tvoid releaseBuffers();\n \tint prepareBuffers(unsigned int count);\n","prefixes":["libcamera-devel","v3","10/10"]}