{"id":2751,"url":"https://patchwork.libcamera.org/api/patches/2751/?format=json","web_url":"https://patchwork.libcamera.org/patch/2751/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200129033210.278800-18-nicolas@ndufresne.ca>","date":"2020-01-29T03:32:04","name":"[libcamera-devel,v1,17/23] gst: Add a pool and an allocator implementation","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"25f622654a11b0f20f4303e3dd0b4ff9d62f5dac","submitter":{"id":30,"url":"https://patchwork.libcamera.org/api/people/30/?format=json","name":"Nicolas Dufresne","email":"nicolas@ndufresne.ca"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/2751/mbox/","series":[{"id":648,"url":"https://patchwork.libcamera.org/api/series/648/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=648","date":"2020-01-29T03:31:47","name":"GStreamer Element for libcamera","version":1,"mbox":"https://patchwork.libcamera.org/series/648/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/2751/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/2751/checks/","tags":{},"headers":{"Return-Path":"","Received":["from bhuna.collabora.co.uk (bhuna.collabora.co.uk\n\t[IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E026608BE\n\tfor ;\n\tWed, 29 Jan 2020 04:35:39 +0100 (CET)","from nicolas-tpx395.localdomain (unknown\n\t[IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits))\n\t(No client certificate requested) (Authenticated sender: nicolas)\n\tby bhuna.collabora.co.uk (Postfix) with ESMTPSA id 5791F2942F4;\n\tWed, 29 Jan 2020 03:35:37 +0000 (GMT)"],"From":"Nicolas Dufresne ","To":"libcamera-devel@lists.libcamera.org","Cc":"Nicolas Dufresne ","Date":"Tue, 28 Jan 2020 22:32:04 -0500","Message-Id":"<20200129033210.278800-18-nicolas@ndufresne.ca>","X-Mailer":"git-send-email 2.24.1","In-Reply-To":"<20200129033210.278800-1-nicolas@ndufresne.ca>","References":"<20200129033210.278800-1-nicolas@ndufresne.ca>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v1 17/23] gst: Add a pool and an allocator\n\timplementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Wed, 29 Jan 2020 03:35:39 -0000"},"content":"From: Nicolas Dufresne \n\nThis is needed to track the livetime of the FrameBufferAllocator in relation to\nthe GstBuffer/GstMemory objects travelling inside GStreamer.\n\nSigned-off-by: Nicolas Dufresne \n---\n src/gstreamer/gstlibcameraallocator.cpp | 240 ++++++++++++++++++++++++\n src/gstreamer/gstlibcameraallocator.h | 29 +++\n src/gstreamer/gstlibcamerapool.cpp | 109 +++++++++++\n src/gstreamer/gstlibcamerapool.h | 27 +++\n src/gstreamer/meson.build | 11 +-\n 5 files changed, 413 insertions(+), 3 deletions(-)\n create mode 100644 src/gstreamer/gstlibcameraallocator.cpp\n create mode 100644 src/gstreamer/gstlibcameraallocator.h\n create mode 100644 src/gstreamer/gstlibcamerapool.cpp\n create mode 100644 src/gstreamer/gstlibcamerapool.h","diff":"diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp\nnew file mode 100644\nindex 0000000..0f248b4\n--- /dev/null\n+++ b/src/gstreamer/gstlibcameraallocator.cpp\n@@ -0,0 +1,240 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Collabora Ltd.\n+ * Author: Nicolas Dufresne \n+ *\n+ * gstlibcameraallocator.cpp - GStreamer Custom Allocator\n+ */\n+\n+#include \"gstlibcameraallocator.h\"\n+#include \"gstlibcamera-utils.h\"\n+\n+#include \n+#include \n+#include \n+\n+using namespace libcamera;\n+\n+/***********************************************************************/\n+/* Internal object for tracking memories associated with a FrameBuffer */\n+/***********************************************************************/\n+\n+static gboolean gst_libcamera_allocator_release(GstMiniObject *mini_object);\n+\n+/* This internal object is used to track the outstanding GstMemory object that\n+ * are part of a FrameBuffer. The allocator will only re-use a FrameBuffer when\n+ * al all outstranging GstMemory have returned.\n+ */\n+struct FrameWrap {\n+\tFrameWrap(GstAllocator *allocator, FrameBuffer *buffer,\n+\t\t gpointer stream);\n+\t~FrameWrap();\n+\n+\tvoid AcquirePlane() { ++outstanding_planes_; }\n+\tbool ReleasePlane() { return --outstanding_planes_ == 0; }\n+\n+\tgpointer stream_;\n+\tFrameBuffer *buffer_;\n+\tstd::vector planes_;\n+\tgint outstanding_planes_;\n+};\n+\n+static GQuark\n+gst_libcamera_frame_quark(void)\n+{\n+\tstatic gsize frame_quark = 0;\n+\n+\tif (g_once_init_enter(&frame_quark)) {\n+\t\tGQuark quark = g_quark_from_string(\"GstLibCameraFrameWrap\");\n+\t\tg_once_init_leave(&frame_quark, quark);\n+\t}\n+\n+\treturn frame_quark;\n+}\n+\n+FrameWrap::FrameWrap(GstAllocator *allocator, FrameBuffer *buffer,\n+\t\t gpointer stream)\n+\n+\t: stream_(stream),\n+\t buffer_(buffer),\n+\t outstanding_planes_(0)\n+{\n+\tfor (const FrameBuffer::Plane &plane : buffer->planes()) {\n+\t\tGstMemory *mem = gst_fd_allocator_alloc(allocator,\n+\t\t\t\t\t\t\tplane.fd.fd(),\n+\t\t\t\t\t\t\tplane.length,\n+\t\t\t\t\t\t\tGST_FD_MEMORY_FLAG_DONT_CLOSE);\n+\t\tgst_mini_object_set_qdata(GST_MINI_OBJECT(mem),\n+\t\t\t\t\t gst_libcamera_frame_quark(),\n+\t\t\t\t\t this, NULL);\n+\t\tGST_MINI_OBJECT(mem)->dispose = gst_libcamera_allocator_release;\n+\t\tg_object_unref(mem->allocator);\n+\t\tplanes_.push_back(mem);\n+\t}\n+}\n+\n+FrameWrap::~FrameWrap()\n+{\n+\tfor (GstMemory *mem : planes_) {\n+\t\tGST_MINI_OBJECT(mem)->dispose = nullptr;\n+\t\tg_object_ref(mem->allocator);\n+\t\tgst_memory_unref(mem);\n+\t}\n+}\n+\n+/***********************************/\n+/* The GstAllocator implementation */\n+/***********************************/\n+struct _GstLibcameraAllocator {\n+\tGstDmaBufAllocator parent;\n+\tFrameBufferAllocator *fb_allocator;\n+\t/* A hash table using Stream pointer as key and returning a GQueue of\n+\t * FrameWrap. */\n+\tGHashTable *pools;\n+};\n+\n+G_DEFINE_TYPE(GstLibcameraAllocator, gst_libcamera_allocator,\n+\t GST_TYPE_DMABUF_ALLOCATOR);\n+\n+static gboolean\n+gst_libcamera_allocator_release(GstMiniObject *mini_object)\n+{\n+\tGstMemory *mem = GST_MEMORY_CAST(mini_object);\n+\tGstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(mem->allocator);\n+\tGST_OBJECT_LOCKER(self);\n+\tgpointer data = gst_mini_object_get_qdata(mini_object,\n+\t\t\t\t\t\t gst_libcamera_frame_quark());\n+\tauto *frame = (FrameWrap *)data;\n+\n+\tgst_memory_ref(mem);\n+\n+\tif (frame->ReleasePlane()) {\n+\t\tauto *pool = (GQueue *)g_hash_table_lookup(self->pools, frame->stream_);\n+\t\tg_return_val_if_fail(pool, TRUE);\n+\t\tg_queue_push_tail(pool, frame);\n+\t}\n+\n+\t/* Keep last in case we are holding on the last allocator ref */\n+\tg_object_unref(mem->allocator);\n+\n+\t/* Rreturns FALSE so that our mini object isn't freed */\n+\treturn FALSE;\n+}\n+\n+static void\n+gst_libcamera_allocator_free_pool(gpointer data)\n+{\n+\tGQueue *queue = (GQueue *)data;\n+\tFrameWrap *frame;\n+\n+\twhile ((frame = (FrameWrap *)g_queue_pop_head(queue))) {\n+\t\tg_warn_if_fail(frame->outstanding_planes_ == 0);\n+\t\tdelete frame;\n+\t}\n+\n+\tg_queue_free(queue);\n+}\n+\n+static void\n+gst_libcamera_allocator_init(GstLibcameraAllocator *self)\n+{\n+\tself->pools = g_hash_table_new_full(NULL, NULL, NULL,\n+\t\t\t\t\t gst_libcamera_allocator_free_pool);\n+\tGST_OBJECT_FLAG_SET(self, GST_ALLOCATOR_FLAG_CUSTOM_ALLOC);\n+}\n+\n+static void\n+gst_libcamera_allocator_dispose(GObject *object)\n+{\n+\tGstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object);\n+\n+\tif (self->pools) {\n+\t\tg_hash_table_unref(self->pools);\n+\t\tself->pools = NULL;\n+\t}\n+\n+\tG_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->dispose(object);\n+}\n+\n+static void\n+gst_libcamera_allocator_finalize(GObject *object)\n+{\n+\tGstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object);\n+\n+\tdelete self->fb_allocator;\n+\n+\tG_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->finalize(object);\n+}\n+\n+static void\n+gst_libcamera_allocator_class_init(GstLibcameraAllocatorClass *klass)\n+{\n+\tauto *allocator_class = GST_ALLOCATOR_CLASS(klass);\n+\tauto *object_class = G_OBJECT_CLASS(klass);\n+\n+\tobject_class->dispose = gst_libcamera_allocator_dispose;\n+\tobject_class->finalize = gst_libcamera_allocator_finalize;\n+\tallocator_class->alloc = NULL;\n+}\n+\n+GstLibcameraAllocator *\n+gst_libcamera_allocator_new(std::shared_ptr camera)\n+{\n+\tauto *self = (GstLibcameraAllocator *)g_object_new(GST_TYPE_LIBCAMERA_ALLOCATOR,\n+\t\t\t\t\t\t\t nullptr);\n+\n+\tself->fb_allocator = FrameBufferAllocator::create(camera);\n+\tfor (Stream *stream : camera->streams()) {\n+\t\tgint ret;\n+\n+\t\tret = self->fb_allocator->allocate(stream);\n+\t\tif (ret == 0)\n+\t\t\treturn nullptr;\n+\n+\t\tGQueue *pool = g_queue_new();\n+\t\tfor (const std::unique_ptr &buffer :\n+\t\t self->fb_allocator->buffers(stream)) {\n+\t\t\tauto *fb = new FrameWrap(GST_ALLOCATOR(self),\n+\t\t\t\t\t\t buffer.get(), stream);\n+\t\t\tg_queue_push_tail(pool, fb);\n+\t\t}\n+\n+\t\tg_hash_table_insert(self->pools, stream, pool);\n+\t}\n+\n+\treturn self;\n+}\n+\n+bool\n+gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self,\n+\t\t\t\t Stream *stream, GstBuffer *buffer)\n+{\n+\tGST_OBJECT_LOCKER(self);\n+\n+\tauto *pool = (GQueue *)g_hash_table_lookup(self->pools, stream);\n+\tg_return_val_if_fail(pool, false);\n+\n+\tauto *frame = (FrameWrap *)g_queue_pop_head(pool);\n+\tif (!frame)\n+\t\treturn false;\n+\n+\tfor (GstMemory *mem : frame->planes_) {\n+\t\tframe->AcquirePlane();\n+\t\tgst_buffer_append_memory(buffer, mem);\n+\t\tg_object_ref(mem->allocator);\n+\t}\n+\n+\treturn true;\n+}\n+\n+gsize\n+gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self,\n+\t\t\t\t Stream *stream)\n+{\n+\tGST_OBJECT_LOCKER(self);\n+\n+\tauto *pool = (GQueue *)g_hash_table_lookup(self->pools, stream);\n+\tg_return_val_if_fail(pool, false);\n+\n+\treturn pool->length;\n+}\ndiff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h\nnew file mode 100644\nindex 0000000..f2a0f58\n--- /dev/null\n+++ b/src/gstreamer/gstlibcameraallocator.h\n@@ -0,0 +1,29 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Collabora Ltd.\n+ * Author: Nicolas Dufresne \n+ *\n+ * gstlibcameraallocator.h - GStreamer Custom Allocator\n+ */\n+\n+#include \n+#include \n+#include \n+\n+#ifndef __GST_LIBCAMERA_ALLOCATOR_H__\n+#define __GST_LIBCAMERA_ALLOCATOR_H__\n+\n+#define GST_TYPE_LIBCAMERA_ALLOCATOR gst_libcamera_allocator_get_type()\n+G_DECLARE_FINAL_TYPE(GstLibcameraAllocator, gst_libcamera_allocator,\n+\t\t GST_LIBCAMERA, ALLOCATOR, GstDmaBufAllocator)\n+\n+GstLibcameraAllocator *gst_libcamera_allocator_new(std::shared_ptr camera);\n+\n+bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self,\n+\t\t\t\t\t libcamera::Stream *stream,\n+\t\t\t\t\t GstBuffer *buffer);\n+\n+gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator,\n+\t\t\t\t\t libcamera::Stream *stream);\n+\n+#endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */\ndiff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp\nnew file mode 100644\nindex 0000000..f84d1d6\n--- /dev/null\n+++ b/src/gstreamer/gstlibcamerapool.cpp\n@@ -0,0 +1,109 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Collabora Ltd.\n+ * Author: Nicolas Dufresne \n+ *\n+ * gstlibcamerapool.cpp - GStreamer Buffer Pool\n+ */\n+\n+#include \"gstlibcamerapool.h\"\n+#include \"gstlibcamera-utils.h\"\n+\n+#include \n+\n+using namespace libcamera;\n+\n+struct _GstLibcameraPool {\n+\tGstBufferPool parent;\n+\n+\tGstAtomicQueue *queue;\n+\tGstLibcameraAllocator *allocator;\n+\tStream *stream;\n+};\n+\n+G_DEFINE_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_TYPE_BUFFER_POOL);\n+\n+static GstFlowReturn\n+gst_libcamera_pool_acquire_buffer(GstBufferPool *pool, GstBuffer **buffer,\n+\t\t\t\t GstBufferPoolAcquireParams *params)\n+{\n+\tGstLibcameraPool *self = GST_LIBCAMERA_POOL(pool);\n+\tGstBuffer *buf = (GstBuffer *)gst_atomic_queue_pop(self->queue);\n+\tif (!buf)\n+\t\treturn GST_FLOW_ERROR;\n+\n+\tif (!gst_libcamera_allocator_prepare_buffer(self->allocator, self->stream, buf))\n+\t\treturn GST_FLOW_ERROR;\n+\n+\t*buffer = buf;\n+\treturn GST_FLOW_OK;\n+}\n+\n+static void\n+gst_libcamera_pool_reset_buffer(GstBufferPool *pool, GstBuffer *buffer)\n+{\n+\tGstBufferPoolClass *klass = GST_BUFFER_POOL_CLASS(gst_libcamera_pool_parent_class);\n+\n+\t/* Clears all the memories and only pool the GstBuffer objects */\n+\tgst_buffer_remove_all_memory(buffer);\n+\tklass->reset_buffer(pool, buffer);\n+\tGST_BUFFER_FLAGS(buffer) = 0;\n+}\n+\n+static void\n+gst_libcamera_pool_release_buffer(GstBufferPool *pool, GstBuffer *buffer)\n+{\n+\tGstLibcameraPool *self = GST_LIBCAMERA_POOL(pool);\n+\tgst_atomic_queue_push(self->queue, (gpointer)buffer);\n+}\n+\n+static void\n+gst_libcamera_pool_init(GstLibcameraPool *self)\n+{\n+\tself->queue = gst_atomic_queue_new(4);\n+}\n+\n+static void\n+gst_libcamera_pool_finalize(GObject *object)\n+{\n+\tGstLibcameraPool *self = GST_LIBCAMERA_POOL(object);\n+\tGstBuffer *buf;\n+\n+\twhile ((buf = (GstBuffer *)gst_atomic_queue_pop(self->queue)))\n+\t\tgst_buffer_unref(buf);\n+\n+\tgst_atomic_queue_unref(self->queue);\n+\tg_object_unref(self->allocator);\n+\n+\tG_OBJECT_CLASS(gst_libcamera_pool_parent_class)->finalize(object);\n+}\n+\n+static void\n+gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass)\n+{\n+\tauto *object_class = G_OBJECT_CLASS(klass);\n+\tauto *pool_class = GST_BUFFER_POOL_CLASS(klass);\n+\n+\tobject_class->finalize = gst_libcamera_pool_finalize;\n+\tpool_class->start = nullptr;\n+\tpool_class->acquire_buffer = gst_libcamera_pool_acquire_buffer;\n+\tpool_class->reset_buffer = gst_libcamera_pool_reset_buffer;\n+\tpool_class->release_buffer = gst_libcamera_pool_release_buffer;\n+}\n+\n+GstLibcameraPool *\n+gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream)\n+{\n+\tauto *pool = (GstLibcameraPool *)g_object_new(GST_TYPE_LIBCAMERA_POOL, nullptr);\n+\n+\tpool->allocator = (GstLibcameraAllocator *)g_object_ref(allocator);\n+\tpool->stream = stream;\n+\n+\tgsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream);\n+\tfor (gsize i = 0; i < pool_size; i++) {\n+\t\tGstBuffer *buffer = gst_buffer_new();\n+\t\tgst_atomic_queue_push(pool->queue, buffer);\n+\t}\n+\n+\treturn pool;\n+}\ndiff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h\nnew file mode 100644\nindex 0000000..ca6b299\n--- /dev/null\n+++ b/src/gstreamer/gstlibcamerapool.h\n@@ -0,0 +1,27 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2020, Collabora Ltd.\n+ * Author: Nicolas Dufresne \n+ *\n+ * gstlibcamerapool.h - GStreamer Buffer Pool\n+ *\n+ * This is a partial implementation of GstBufferPool intended for internal use\n+ * only. This pool cannot be configured or activated.\n+ */\n+\n+#include \n+#include \n+\n+#include \"gstlibcameraallocator.h\"\n+\n+#ifndef __GST_LIBCAMERA_POOL_H__\n+#define __GST_LIBCAMERA_POOL_H__\n+\n+#define GST_TYPE_LIBCAMERA_POOL gst_libcamera_pool_get_type()\n+G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool,\n+\t\t GST_LIBCAMERA, POOL, GstBufferPool)\n+\n+GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator,\n+\t\t\t\t\t libcamera::Stream *stream);\n+\n+#endif /* __GST_LIBCAMERA_POOL_H__ */\ndiff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build\nindex e497bf4..346910f 100644\n--- a/src/gstreamer/meson.build\n+++ b/src/gstreamer/meson.build\n@@ -4,6 +4,8 @@ libcamera_gst_sources = [\n 'gstlibcamerasrc.cpp',\n 'gstlibcameraprovider.cpp',\n 'gstlibcamerapad.cpp',\n+ 'gstlibcameraallocator.cpp',\n+ 'gstlibcamerapool.cpp'\n ]\n \n libcamera_gst_c_args = [\n@@ -11,15 +13,18 @@ libcamera_gst_c_args = [\n '-DPACKAGE=\"@0@\"'.format(meson.project_name()),\n ]\n \n-gst_dep = dependency('gstreamer-video-1.0', version : '>=1.16.1',\n+gst_req = '>=1.16.1'\n+gstvideo_dep = dependency('gstreamer-video-1.0', version : gst_req,\n+ required : get_option('gstreamer'))\n+gstallocator_dep = dependency('gstreamer-allocators-1.0', version : gst_req,\n required : get_option('gstreamer'))\n \n-if gst_dep.found()\n+if gstvideo_dep.found() and gstallocator_dep.found()\n libcamera_gst = shared_library('gstlibcamera',\n libcamera_gst_sources,\n c_args : libcamera_gst_c_args,\n include_directories : [],\n- dependencies : [libcamera_dep, gst_dep],\n+ dependencies : [libcamera_dep, gstvideo_dep, gstallocator_dep],\n install: true,\n install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')),\n )\n","prefixes":["libcamera-devel","v1","17/23"]}