[{"id":4029,"web_url":"https://patchwork.libcamera.org/comment/4029/","msgid":"<20200316135814.GE2260535@oden.dyn.berto.se>","date":"2020-03-16T13:58:14","subject":"Re: [libcamera-devel] [PATCH 5/9] libcamera: v4l2_videodevice: Add\n\tstandalone buffer export support","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/people/5/","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"content":"Hi Laurent,\n\nThanks for your work.\n\nOn 2020-03-15 01:57:24 +0200, Laurent Pinchart wrote:\n> Add a new exportBuffers() function that only performs buffer allocation\n> and export, but leaves the V4L2 buffer queue unallocated on return. This\n> function will be used to simplify buffer allocation for pipeline\n> handlers. This is made possible by the V4L2 buffer orphaning feature\n> introduced in Linux v5.0, so add a version check to catch and report\n> issues early.\n> \n> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n> ---\n>  src/libcamera/include/v4l2_videodevice.h |   2 +\n>  src/libcamera/v4l2_videodevice.cpp       | 140 ++++++++++++++++++++++-\n>  2 files changed, 140 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/libcamera/include/v4l2_videodevice.h b/src/libcamera/include/v4l2_videodevice.h\n> index 358d89e57414..16fc6b60de02 100644\n> --- a/src/libcamera/include/v4l2_videodevice.h\n> +++ b/src/libcamera/include/v4l2_videodevice.h\n> @@ -193,6 +193,8 @@ public:\n>  \n>  \tint allocateBuffers(unsigned int count,\n>  \t\t\t    std::vector<std::unique_ptr<FrameBuffer>> *buffers);\n> +\tint exportBuffers(unsigned int count,\n> +\t\t\t  std::vector<std::unique_ptr<FrameBuffer>> *buffers);\n>  \tint importBuffers(unsigned int count);\n>  \tint releaseBuffers();\n>  \n> diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp\n> index d02b02ef77d6..e37136af8eb7 100644\n> --- a/src/libcamera/v4l2_videodevice.cpp\n> +++ b/src/libcamera/v4l2_videodevice.cpp\n> @@ -19,6 +19,7 @@\n>  #include <vector>\n>  \n>  #include <linux/drm_fourcc.h>\n> +#include <linux/version.h>\n>  \n>  #include <libcamera/event_notifier.h>\n>  #include <libcamera/file_descriptor.h>\n> @@ -404,6 +405,53 @@ const std::string V4L2DeviceFormat::toString() const\n>   * No API call other than open(), isOpen() and close() shall be called on an\n>   * unopened device instance.\n>   *\n> + * The V4L2VideoDevice class supports the V4L2 MMAP and DMABUF memory types:\n> + *\n> + * - The allocateBuffers() function wraps buffer allocation with the V4L2 MMAP\n> + *   memory type. It requests buffers from the driver, allocating the\n> + *   corresponding memory, and exports them as a set of FrameBuffer objects.\n> + *   Upon successful return the driver's internal buffer management is\n> + *   initialized in MMAP mode, and the video device is ready to accept\n> + *   queueBuffer() calls.\n> + *\n> + *   This is the most traditional V4L2 buffer management, and is mostly useful\n> + *   to support internal buffer pools in pipeline handlers, either for CPU\n> + *   consumption (such as statistics or parameters pools), or for internal\n> + *   image buffers shared between devices.\n> + *\n> + * - The exportBuffers() function operates similarly to allocateBuffers(), but\n> + *   leaves the driver's internal buffer management uninitialized. It uses the\n> + *   V4L2 buffer orphaning support to allocate buffers with the MMAP method,\n> + *   export them as a set of FrameBuffer objects, and reset the driver's\n> + *   internal buffer management. The video device shall be initialized with\n> + *   importBuffers() or allocateBuffers() before it can accept queueBuffer()\n> + *   calls. The exported buffers are directly usable with any V4L2 video device\n> + *   in DMABUF mode, or with other dmabuf importers.\n> + *\n> + *   This method is mostly useful to implement buffer allocation helpers or to\n> + *   allocate ancillary buffers, when a V4L2 video device is used in DMABUF\n> + *   mode but no other source of buffers is available. An example use case\n> + *   would be allocation of scratch buffers to be used in case of buffer\n> + *   underruns on a video device that is otherwise supplied with external\n> + *   buffers.\n> + *\n> + * - The importBuffers() function initializes the driver's buffer management to\n> + *   import buffers in DMABUF mode. It requests buffers from the driver, but\n> + *   doesn't allocate memory. Upon successful return, the video device is ready\n> + *   to accept queueBuffer() calls. The buffers to be imported are provided to\n> + *   queueBuffer(), and may be supplied externally, or come from a previous\n> + *   exportBuffers() call.\n> + *\n> + *   This is the usual buffers initialization method for video devices whose\n> + *   buffers are exposed outside of libcamera. It is also typically used on one\n> + *   of the two video device that participate in buffer sharing inside\n> + *   pipelines, the other video device typically using allocateBuffers().\n> + *\n> + * - The releaseBuffers() function resets the driver's internal buffer\n> + *   management that was initialized by a previous call to allocateBuffers() or\n> + *   importBuffers(). Any memory allocated by allocateBuffers() is freed.\n> + *   Buffer exported by exportBuffers() are not affected by this function.\n> + *\n>   * The V4L2VideoDevice class tracks queued buffers and handles buffer events. It\n>   * automatically dequeues completed buffers and emits the \\ref bufferReady\n>   * signal.\n> @@ -466,6 +514,15 @@ int V4L2VideoDevice::open()\n>  \t\treturn ret;\n>  \t}\n>  \n> +\tif (caps_.version < KERNEL_VERSION(5, 0, 0)) {\n> +\t\tLOG(V4L2, Error)\n> +\t\t\t<< \"V4L2 API v\" << (caps_.version >> 16)\n> +\t\t\t<< \".\" << ((caps_.version >> 8) & 0xff)\n> +\t\t\t<< \".\" << (caps_.version & 0xff)\n> +\t\t\t<< \" too old, v5.0.0 or later is required\";\n> +\t\treturn -EINVAL;\n> +\t}\n> +\n>  \tif (!caps_.hasStreaming()) {\n>  \t\tLOG(V4L2, Error) << \"Device does not support streaming I/O\";\n>  \t\treturn -EINVAL;\n> @@ -1044,11 +1101,27 @@ int V4L2VideoDevice::requestBuffers(unsigned int count,\n>  }\n>  \n>  /**\n> - * \\brief Allocate buffers from the video device\n> + * \\brief Allocate and export buffers from the video device\n>   * \\param[in] count Number of buffers to allocate\n>   * \\param[out] buffers Vector to store allocated buffers\n> + *\n> + * This function wraps buffer allocation with the V4L2 MMAP memory type. It\n> + * requests \\a count buffers from the driver, allocating the corresponding\n> + * memory, and exports them as a set of FrameBuffer objects in \\a buffers. Upon\n> + * successful return the driver's internal buffer management is initialized in\n> + * MMAP mode, and the video device is ready to accept queueBuffer() calls.\n> + *\n> + * The number of planes and the plane sizes for the allocation are determined\n> + * by the currently active format on the device as set by setFormat().\n> + *\n> + * Buffers allocated with this function shall later be free with\n> + * releaseBuffers(). If buffers have already been allocated with\n> + * allocateBuffers() or imported with importBuffers(), this function returns\n> + * -EBUSY.\n> + *\n>   * \\return The number of allocated buffers on success or a negative error code\n>   * otherwise\n> + * \\retval -EBUSY buffers have already been allocated or imported\n>   */\n>  int V4L2VideoDevice::allocateBuffers(unsigned int count,\n>  \t\t\t\t     std::vector<std::unique_ptr<FrameBuffer>> *buffers)\n> @@ -1063,6 +1136,49 @@ int V4L2VideoDevice::allocateBuffers(unsigned int count,\n>  \treturn ret;\n>  }\n>  \n> +/**\n> + * \\brief Export buffers from the video device\n> + * \\param[in] count Number of buffers to allocate\n> + * \\param[out] buffers Vector to store allocated buffers\n> + *\n> + * This function allocates \\a count buffer from the video device and exports\n> + * them as dmabuf objects, stored in \\a buffers. Unlike allocateBuffers(), this\n> + * function leaves the driver's internal buffer management uninitialized. The\n> + * video device shall be initialized with importBuffers() or allocateBuffers()\n> + * before it can accept queueBuffer() calls. The exported buffers are directly\n> + * usable with any V4L2 video device in DMABUF mode, or with other dmabuf\n> + * importers.\n> + *\n> + * The number of planes and the plane sizes for the allocation are determined\n> + * by the currently active format on the device as set by setFormat().\n> + *\n> + * Multiple independent sets of buffers can be allocated with multiple calls to\n> + * this function. Device-specific limitations may apply regarding the minimum\n> + * and maximum number of buffers per set, or to total amount of allocated\n> + * memory. The exported dmabuf lifetime is tied to the returned \\a buffers. To\n> + * free a buffer, the caller shall delete the corresponding FrameBuffer\n> + * instance. No bookkeeping and automatic free is performed by the\n> + * V4L2VideoDevice class.\n> + *\n> + * If buffers have already been allocated with allocateBuffers() or imported\n> + * with importBuffers(), this function returns -EBUSY.\n> + *\n> + * \\return The number of allocated buffers on success or a negative error code\n> + * otherwise\n> + * \\retval -EBUSY buffers have already been allocated or imported\n> + */\n> +int V4L2VideoDevice::exportBuffers(unsigned int count,\n> +\t\t\t\t   std::vector<std::unique_ptr<FrameBuffer>> *buffers)\n> +{\n> +\tint ret = createBuffers(count, buffers);\n> +\tif (ret < 0)\n> +\t\treturn ret;\n> +\n> +\trequestBuffers(0, V4L2_MEMORY_MMAP);\n> +\n> +\treturn ret;\n> +}\n> +\n>  int V4L2VideoDevice::createBuffers(unsigned int count,\n>  \t\t\t\t   std::vector<std::unique_ptr<FrameBuffer>> *buffers)\n>  {\n> @@ -1160,7 +1276,22 @@ FileDescriptor V4L2VideoDevice::exportDmabufFd(unsigned int index,\n>  /**\n>   * \\brief Prepare the device to import \\a count buffers\n>   * \\param[in] count Number of buffers to prepare to import\n> + *\n> + * This function initializes the driver's buffer management to import buffers\n> + * in DMABUF mode. It requests buffers from the driver, but doesn't allocate\n> + * memory.\n> + *\n> + * Upon successful return, the video device is ready to accept queueBuffer()\n> + * calls. The buffers to be imported are provided to queueBuffer(), and may be\n> + * supplied externally, or come from a previous exportBuffers() call.\n> + *\n> + * Device initialization performed by this function shall later be cleaned up\n> + * with releaseBuffers(). If buffers have already been allocated with\n> + * allocateBuffers() or imported with importBuffers(), this function returns\n> + * -EBUSY.\n> + *\n>   * \\return 0 on success or a negative error code otherwise\n> + * \\retval -EBUSY buffers have already been allocated or imported\n>   */\n>  int V4L2VideoDevice::importBuffers(unsigned int count)\n>  {\n> @@ -1183,7 +1314,12 @@ int V4L2VideoDevice::importBuffers(unsigned int count)\n>  }\n>  \n>  /**\n> - * \\brief Release all internally allocated buffers\n> + * \\brief Release resources allocated by importBuffers()\n\nor allocateBuffers(), right?\n\nWith this fixed,\n\nReviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>\n\n> + *\n> + * This function resets the driver's internal buffer management that was\n> + * initialized by a previous call to allocateBuffers() or importBuffers(). Any\n> + * memory allocated by allocateBuffers() is freed. Buffer exported by\n> + * exportBuffers(), if any, are not affected.\n>   */\n>  int V4L2VideoDevice::releaseBuffers()\n>  {\n> -- \n> Regards,\n> \n> Laurent Pinchart\n> \n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"<niklas.soderlund@ragnatech.se>","Received":["from mail-lf1-x143.google.com (mail-lf1-x143.google.com\n\t[IPv6:2a00:1450:4864:20::143])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C67456041A\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 16 Mar 2020 14:58:16 +0100 (CET)","by mail-lf1-x143.google.com with SMTP id u26so1886676lfu.8\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 16 Mar 2020 06:58:16 -0700 (PDT)","from localhost (h-200-138.A463.priv.bahnhof.se. [176.10.200.138])\n\tby smtp.gmail.com with ESMTPSA id\n\tu6sm28044056lff.35.2020.03.16.06.58.15\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 16 Mar 2020 06:58:15 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ragnatech-se.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to;\n\tbh=2wfRv5xoRwURJoO0WgzTQdzmsiyCjHnTmGo7zjJzVjM=;\n\tb=ZxG8GXzd+mxlOyadDscm2VtxWZJASj9Ye6oX6BWlkwUvPiuxGknX1d6zF5yTM+N+jo\n\tsrWqdLUcJ+tOwJ9tStJYnPPyATEfl8/K+9qK1HqNfgzbS7awBw6l4jIRilY2iBx7cZH3\n\tx7GfIrN/s3toZYgnB6HtjdNEYGq6CNmV1eK+TwNtR9EYLZn4LNWgdcp2/Wmj5nxEdRGG\n\t7Y9c5KVFBH8T0g0EUT6/WXyRR/iRZM+rgXHsQBwTkW1gLRAOa4q3MAbcJ6nAbIqNczXC\n\t2NqK+VTuS18hud03lUxWNmp1EXGwN8cBODcWJEBiUNWp3fLsfggGPQ7qJ6bcIUfPyvb0\n\ttp4A==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to;\n\tbh=2wfRv5xoRwURJoO0WgzTQdzmsiyCjHnTmGo7zjJzVjM=;\n\tb=grCqTxNwn9Lkb8f3PbKC8KVWef3Xf3iy2CryIroBImrCbcfwUfRr9FQ1PZXoPKjfXb\n\tV1m2Fdmld93vFlCeNpRIvqDJsmaGaos0qOhg4AMZpbVHQ/AeszZorFqN5GHarnri1Ed/\n\tVfunbJ9W9gdIf17+aTi1NpgXBpANfqrqOioBbjWldBFwa6WrZb8XnuHJbtvsqEI4pyiK\n\tlmihUwMM7SMSalkicILy4whkjzpQjBDhIz35E14oMLc+1Kfe/yH2iCP7geXNUQiEdRSg\n\t7uX9fkU3yGHgjXBYf9xOLM/Kj2HIQYmt0xpiOGi5buIwP04K1/RJ37mSWcSibugEHNos\n\tExSw==","X-Gm-Message-State":"ANhLgQ1TCOClgfLrPVmKc/6l+o5e/cEA90Ed7aLuarpWjMppDNpwFqsj\n\tNxfjZG+iZVFK36XWOjMmgwNh/t3dHH0=","X-Google-Smtp-Source":"ADFU+vtlwlKxDe1bYkhnAEncxM42iGx2S3A/0FFPq4tCgobauKmDTrdHy4+abchGYygWXVpcs3P35A==","X-Received":"by 2002:a19:fc1d:: with SMTP id\n\ta29mr17439441lfi.209.1584367095796; \n\tMon, 16 Mar 2020 06:58:15 -0700 (PDT)","Date":"Mon, 16 Mar 2020 14:58:14 +0100","From":"Niklas =?iso-8859-1?q?S=F6derlund?= <niklas.soderlund@ragnatech.se>","To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200316135814.GE2260535@oden.dyn.berto.se>","References":"<20200314235728.15495-1-laurent.pinchart@ideasonboard.com>\n\t<20200314235728.15495-6-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=iso-8859-1","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200314235728.15495-6-laurent.pinchart@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 5/9] libcamera: v4l2_videodevice: Add\n\tstandalone buffer export support","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","X-List-Received-Date":"Mon, 16 Mar 2020 13:58:17 -0000"}}]