[{"id":3403,"web_url":"https://patchwork.libcamera.org/comment/3403/","msgid":"<20200111000045.GF4859@pendragon.ideasonboard.com>","date":"2020-01-11T00:00:45","subject":"Re: [libcamera-devel] [PATCH v3 08/33] libcamera: buffer: Add\n\tFrameBuffer interface","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Niklas,\n\nThank you for the patch.\n\nOn Fri, Jan 10, 2020 at 08:37:43PM +0100, Niklas Söderlund wrote:\n> Add a new FrameBuffer class to describe memory used to store frames.\n> This change just adds the new interface, future patches will migrate all\n> parts of libcamera to use this and replace the old Buffer interface.\n> \n> This change needs to specify the const keyword for Plane::length() as to\n> not get Doxygen confused with FrameBuffer::Plane::length added in this\n> change.\n> \n> Signed-off-by: Niklas Söderlund \n> ---\n> * Changes since v2\n> - Drop default value for planes argument for FrameBuffer::FrameBuffer(planes, cookie).\n> - Deleted FrameBuffer copy constructor and assignment operator.\n> - Add a Move constructor for FrameBuffer.\n> - Update documentation.\n> \n> Changes since v1:\n> - Rename FrameBuffer::info() to FrameBuffer::metadata()\n> - Fixup commit message\n> - Reorder method order\n> - Rewrite documentation\n> - Add operator==() and operator=!() for FrameBuffer::Plane\n> ---\n> include/libcamera/buffer.h | 36 ++++++++++\n> src/libcamera/buffer.cpp | 130 ++++++++++++++++++++++++++++++++++++-\n> 2 files changed, 165 insertions(+), 1 deletion(-)\n> \n> diff --git a/include/libcamera/buffer.h b/include/libcamera/buffer.h\n> index 0b95e41a2dd205b2..f5522edaf68bbf61 100644\n> --- a/include/libcamera/buffer.h\n> +++ b/include/libcamera/buffer.h\n> @@ -11,6 +11,8 @@\n> #include \n> #include \n> \n> +#include \n> +\n> namespace libcamera {\n> \n> class Request;\n> @@ -33,6 +35,40 @@ struct FrameMetadata {\n> \tstd::vector planes;\n> };\n> \n> +class FrameBuffer final\n> +{\n> +public:\n> +\tstruct Plane {\n> +\t\tFileDescriptor fd;\n> +\t\tunsigned int length;\n> +\t};\n> +\n> +\tFrameBuffer(const std::vector &planes, unsigned int cookie = 0);\n> +\tFrameBuffer(FrameBuffer &&other);\n\nI don't think the move constructor is needed. It's actually dangerous,\nas if a FrameBuffer is added to a request and a new FrameBuffer is later\nmove-constructed from the original FrameBuffer, bad things will happen.\nI think you can delete it (and the corresponding move-assignment\noperator).\n\n> +\n> +\tFrameBuffer(const FrameBuffer &other) = delete;\n> +\tFrameBuffer &operator=(const FrameBuffer &) = delete;\n> +\n> +\tconst std::vector &planes() const { return planes_; }\n> +\n> +\tRequest *request() const { return request_; }\n> +\tconst FrameMetadata &metadata() const { return metadata_; };\n> +\n> +\tunsigned int cookie() const { return cookie_; }\n> +\tvoid setCookie(unsigned int cookie) { cookie_ = cookie; }\n> +\n> +private:\n> +\tfriend class Request; /* Needed to update request_. */\n> +\tfriend class V4L2VideoDevice; /* Needed to update metadata_. */\n> +\n> +\tstd::vector planes_;\n> +\n> +\tRequest *request_;\n> +\tFrameMetadata metadata_;\n> +\n> +\tunsigned int cookie_;\n> +};\n> +\n> class Plane final\n> {\n> public:\n> diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp\n> index b7bc948c80748b18..3e2cc9b94595795e 100644\n> --- a/src/libcamera/buffer.cpp\n> +++ b/src/libcamera/buffer.cpp\n> @@ -240,7 +240,7 @@ void *Plane::mem()\n> }\n> \n> /**\n> - * \\fn Plane::length()\n> + * \\fn Plane::length() const\n> * \\brief Retrieve the length of the memory region\n> * \\return The length of the memory region\n> */\n> @@ -473,4 +473,132 @@ void Buffer::cancel()\n> * The intended callers are Request::prepare() and Request::completeBuffer().\n> */\n> \n> +/**\n> + * \\class FrameBuffer\n> + * \\brief Frame buffer data and its associated dynamic metadata\n> + *\n> + * The FrameBuffer class is the primary interface for applications, IPAs and\n> + * pipelines to interact with frame memory. It contains all the static and\n\ns/pipelines/pipeline handlers/\n\n> + * dynamic information to manage the whole life cycle of a frame capture, from\n> + * buffer creation to consumption.\n> + *\n> + * The static information describes the memory planes that make a frame. The\n> + * planes are specified when creating the FrameBuffer and are expressed as a set\n> + * of dmabuf file descriptors and length.\n> + *\n> + * The dynamic information is grouped in a FrameMetadata instance. It is updated\n> + * during the processing of a queued capture request, and is valid from the\n> + * completion of the buffer as signaled by Camera::bufferComplete() until the\n> + * FrameBuffer is either reused in a new request or deleted.\n> + *\n> + * The creator of a FrameBuffer (application, IPA or pipeline) may associate\n\ns/pipelines/pipeline handler/\n\n> + * to it an integer cookie for any private purpose. The cookie may be set when\n> + * creating the FrameBuffer, and updated at any time with setCookie(). The\n> + * cookie is transparent to the libcamera core and shall only be set by the\n> + * creator of the FrameBuffer. This mechanism supplements the Request cookie.\n> + */\n> +\n> +/**\n> + * \\struct FrameBuffer::Plane\n> + * \\brief A memory region to store a single plane of a frame\n> + *\n> + * Planar pixel formats use multiple memory regions to store the different\n> + * colour components of a frame. The Plane structure describes such a memory\n> + * region by a dmabuf file descriptor and a length. A FrameBuffer then\n> + * contains one or multiple planes, depending on the pixel format of the\n> + * frames it is meant to store.\n> + *\n> + * To support DMA access, planes are associated with dmabuf objects represented\n> + * by FileDescriptor handles. The Plane class doesn't handle mapping of the\n> + * memory to the CPU, but applications and IPAs may use the dmabuf file\n> + * descriptors to map the plane memory with mmap() and access its contents.\n> + *\n> + * \\todo Once we have a Kernel API which can express offsets within a plane\n> + * this structure shall be extended to contain this information. See commit\n> + * 83148ce8be55e for initial documentation of this feature.\n> + */\n> +\n> +/**\n> + * \\var FrameBuffer::Plane::fd\n> + * \\brief The dmabuf file descriptor\n> + */\n> +\n> +/**\n> + * \\var FrameBuffer::Plane::length\n> + * \\brief The plane length in bytes\n> + */\n> +\n> +/**\n> + * \\brief Construct a FrameBuffer with an array of planes\n> + * \\param[in] planes The frame memory planes\n> + * \\param[in] cookie Cookie\n> + *\n> + * The file descriptors associated with each plane are duplicated and can be\n> + * closed by the caller after the FrameBuffer has been constructed.\n\nThis isn't true anymore, now that the FileDescriptor can be cheaply\ncopied, is it ? You could explicitly state that the file descriptor must\nnot be closed by the caller, but FileDescriptor doesn't allow explicit\nclose. I think you can thus just drop this sentence.\n\n> + */\n> +FrameBuffer::FrameBuffer(const std::vector &planes, unsigned int cookie)\n> +\t: planes_(planes), request_(nullptr), cookie_(cookie)\n> +{\n> +}\n> +\n> +/**\n> + * \\brief Move constructor, create a FrameBuffer by taking over \\a other\n> + * \\param[in] other The other FrameBuffer\n> + *\n> + * The \\a other FrameBuffer will be invalidated.\n> + */\n> +FrameBuffer::FrameBuffer(FrameBuffer &&other)\n> +\t: planes_(std::move(other.planes_)),\n> +\t request_(utils::exchange(other.request_, nullptr)),\n> +\t metadata_(std::move(other.metadata_)),\n> +\t cookie_(utils::exchange(other.cookie_, 0))\n\nNow that FileDescriptor can be cheaply copied we can drop the move\nconstructor. Sorry for having asked you to add one in the review of the\nprevious version.\n\n> +{\n> +}\n> +\n> +/**\n> + * \\fn FrameBuffer::planes()\n> + * \\brief Retrieve the static plane descriptors\n> + * \\return Array of plane descriptors\n> + */\n> +\n> +/**\n> + * \\fn FrameBuffer::request()\n> + * \\brief Retrieve the request this buffer belongs to\n> + *\n> + * The intended callers of this method are buffer completion handlers that\n> + * need to associate a buffer to the request it belongs to.\n\nYou've dropped the following paragraph present in v2:\n\n * A Buffer is associated to a request by Request::prepare() and the\n * association is valid until the buffer completes. The returned request\n * pointer is valid only during that interval.\n\nWhile I had comments about the documentation, I don't think this should\nbe dropped, it's important to document the validaty of the association.\nMore importantly I asked why the buffer wasn't associated with the\nrequest at Request::addBuffer() time.\n\n> + *\n> + * \\return The Request the Buffer belongs to, or nullptr if the buffer is\n> + * not associated with a request\n> + */\n> +\n> +/**\n> + * \\fn FrameBuffer::metadata()\n> + * \\brief Retrieve the dynamic metadata\n> + * \\return Dynamic metadata for the frame contained in the buffer\n> + */\n> +\n> +/**\n> + * \\fn FrameBuffer::cookie()\n> + * \\brief Retrieve the cookie\n> + *\n> + * The cookie belongs to the creator of the FrameBuffer, which controls its\n> + * lifetime and value.\n> + *\n> + * \\sa setCookie()\n> + *\n> + * \\return The cookie\n> + */\n> +\n> +/**\n> + * \\fn FrameBuffer::setCookie()\n> + * \\brief Set the cookie\n> + * \\param[in] cookie Cookie to set\n> + *\n> + * The cookie belongs to the creator of the FrameBuffer. Its value may be\n> + * modified at any time with this method. Applications and IPAs shall not modify\n> + * the cookie value of buffers they haven't created themselves. The libcamera\n> + * core never modifies the buffer cookie.\n> + */\n> +\n> } /* namespace libcamera */","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1A16B6067C\n\tfor ;\n\tSat, 11 Jan 2020 01:00:59 +0100 (CET)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 87A1552F;\n\tSat, 11 Jan 2020 01:00:58 +0100 (CET)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1578700858;\n\tbh=tyuT1UGQ0HZXhuo5vLYb0Pz2U4skGSN/AWq0maL1cpM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=cDV2y28QSgnGiwJ1yBFz6WJLim42gfVcfHhfOv1Ru5OUWZrgYHAEB9H/Y6Uoxri3a\n\tBpC2R1+qaOcAKTojhODmYR4S4zZvAsAWlR/WYXo+3/phJU2SzFdHpHMg7FwA6vjgyO\n\t6TpznjwQri49cfMjYiZBUq/XUr0o4F76mZMLdgFk=","Date":"Sat, 11 Jan 2020 02:00:45 +0200","From":"Laurent Pinchart ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Cc":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200111000045.GF4859@pendragon.ideasonboard.com>","References":"<20200110193808.2266294-1-niklas.soderlund@ragnatech.se>\n\t<20200110193808.2266294-9-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200110193808.2266294-9-niklas.soderlund@ragnatech.se>","User-Agent":"Mutt/1.10.1 (2018-07-13)","Subject":"Re: [libcamera-devel] [PATCH v3 08/33] libcamera: buffer: Add\n\tFrameBuffer interface","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Sat, 11 Jan 2020 00:00:59 -0000"}}]