From patchwork Thu Feb 27 20:03:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2883 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 60EE2626BE for ; Thu, 27 Feb 2020 21:04:18 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id DFA3029654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:41 -0500 Message-Id: <20200227200407.490616-2-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/27] Add GStreamer plugin and element skeleton X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:18 -0000 This implements the GStreamer plugin interface and adds libcamerasrc element feature to it. This is just enough to allow plugin introspection. gst-inspect-1.0 build/src/gstreamer/libgstlibcamera.so Plugin Details: Name libcamera Description libcamera capture plugin Filename build/src/gstreamer/libgstlibcamera.so Version 0.0.0+1042-6c9f16d3-dirty License LGPL Source module libcamera Binary package libcamera Origin URL https://libcamera.org libcamerasrc: libcamera Source 1 features: GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer gst-inspect-1.0 libcamerasrc Factory Details: Rank primary (256) Long-name libcamera Source Klass Source/Video Description Linux Camera source using libcamera Author Nicolas Dufresne Reviewed-by: Laurent Pinchart --- meson_options.txt | 5 +++++ src/gstreamer/gstlibcamera.c | 21 +++++++++++++++++++++ src/gstreamer/gstlibcamerasrc.cpp | 31 +++++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerasrc.h | 22 ++++++++++++++++++++++ src/gstreamer/meson.build | 22 ++++++++++++++++++++++ src/meson.build | 2 ++ 6 files changed, 103 insertions(+) create mode 100644 src/gstreamer/gstlibcamera.c create mode 100644 src/gstreamer/gstlibcamerasrc.cpp create mode 100644 src/gstreamer/gstlibcamerasrc.h create mode 100644 src/gstreamer/meson.build diff --git a/meson_options.txt b/meson_options.txt index 79ee4de..b17b6de 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -7,6 +7,11 @@ option('documentation', type : 'boolean', description : 'Generate the project documentation') +option('gstreamer', + type : 'feature', + value : 'auto', + description : 'Compile libcamera GStreamer plugin') + option('test', type : 'boolean', description: 'Compile and include the tests') diff --git a/src/gstreamer/gstlibcamera.c b/src/gstreamer/gstlibcamera.c new file mode 100644 index 0000000..7dd94ca --- /dev/null +++ b/src/gstreamer/gstlibcamera.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera.c - GStreamer plugin + */ + +#include "gstlibcamerasrc.h" + +static gboolean +plugin_init(GstPlugin *plugin) +{ + return gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_SRC); + return TRUE; +} + +GST_PLUGIN_DEFINE(GST_VERSION_MAJOR, GST_VERSION_MINOR, + libcamera, "libcamera capture plugin", + plugin_init, VERSION, "LGPL", PACKAGE, "https://libcamera.org"); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp new file mode 100644 index 0000000..3807503 --- /dev/null +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerasrc.cpp - GStreamer Capture Element + */ + +#include "gstlibcamerasrc.h" + +struct _GstLibcameraSrc { + GstElement parent; +}; + +G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); + +static void +gst_libcamera_src_init(GstLibcameraSrc *self) +{ +} + +static void +gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) +{ + GstElementClass *element_class = (GstElementClass *)klass; + + gst_element_class_set_metadata(element_class, + "libcamera Source", "Source/Video", + "Linux Camera source using libcamera", + "Nicolas Dufresne + * + * gstlibcamerasrc.h - GStreamer Capture Element + */ + +#ifndef __GST_LIBCAMERA_SRC_H__ +#define __GST_LIBCAMERA_SRC_H__ + +#include + +G_BEGIN_DECLS + +#define GST_TYPE_LIBCAMERA_SRC gst_libcamera_src_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraSrc, gst_libcamera_src, + GST_LIBCAMERA, SRC, GstElement) + +G_END_DECLS + +#endif /* __GST_LIBCAMERA_SRC_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build new file mode 100644 index 0000000..832b8a5 --- /dev/null +++ b/src/gstreamer/meson.build @@ -0,0 +1,22 @@ +libcamera_gst_sources = [ + 'gstlibcamera.c', + 'gstlibcamerasrc.cpp', +] + +libcamera_gst_c_args = [ + '-DVERSION="@0@"'.format(libcamera_git_version), + '-DPACKAGE="@0@"'.format(meson.project_name()), +] + +gst_dep = dependency('gstreamer-video-1.0', version : '>=1.16.1', + required : get_option('gstreamer')) + +if gst_dep.found() + libcamera_gst = shared_library('gstlibcamera', + libcamera_gst_sources, + c_args : libcamera_gst_c_args, + dependencies : [libcamera_dep, gst_dep], + install: true, + install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')), + ) +endif diff --git a/src/meson.build b/src/meson.build index 5adcd61..d818d8b 100644 --- a/src/meson.build +++ b/src/meson.build @@ -10,3 +10,5 @@ subdir('qcam') if get_option('v4l2') subdir('v4l2') endif + +subdir('gstreamer') From patchwork Thu Feb 27 20:03:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2884 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C8E9C626D0 for ; Thu, 27 Feb 2020 21:04:18 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 576652949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:42 -0500 Message-Id: <20200227200407.490616-3-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/27] gst: Add utility to convert StreamFormats to GstCaps X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:19 -0000 This transforms the basic information found in StreamFormats to GstCaps. This can be handy to reply to early caps query or inside a device provider. Note that we ignored generated range as they are harmful to caps negotiation. We also don't simplify the caps for readability reasons, so some of the discrete value may be included in a range. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamera-utils.cpp | 98 ++++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 18 +++++ src/gstreamer/meson.build | 1 + 3 files changed, 117 insertions(+) create mode 100644 src/gstreamer/gstlibcamera-utils.cpp create mode 100644 src/gstreamer/gstlibcamera-utils.h diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp new file mode 100644 index 0000000..dc129c3 --- /dev/null +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera-utils.c - GStreamer libcamera Utility Function + */ + +#include "gstlibcamera-utils.h" +#include + +using namespace libcamera; + +static struct { + GstVideoFormat gst_format; + guint drm_fourcc; +} format_map[] = { + { GST_VIDEO_FORMAT_ENCODED, DRM_FORMAT_MJPEG }, + { GST_VIDEO_FORMAT_RGB, DRM_FORMAT_BGR888 }, + { GST_VIDEO_FORMAT_BGR, DRM_FORMAT_RGB888 }, + { GST_VIDEO_FORMAT_ARGB, DRM_FORMAT_BGRA8888 }, + { GST_VIDEO_FORMAT_NV12, DRM_FORMAT_NV12 }, + { GST_VIDEO_FORMAT_NV21, DRM_FORMAT_NV21 }, + { GST_VIDEO_FORMAT_NV16, DRM_FORMAT_NV16 }, + { GST_VIDEO_FORMAT_NV61, DRM_FORMAT_NV61 }, + { GST_VIDEO_FORMAT_NV24, DRM_FORMAT_NV24 }, + { GST_VIDEO_FORMAT_UYVY, DRM_FORMAT_UYVY }, + { GST_VIDEO_FORMAT_VYUY, DRM_FORMAT_VYUY }, + { GST_VIDEO_FORMAT_YUY2, DRM_FORMAT_YUYV }, + { GST_VIDEO_FORMAT_YVYU, DRM_FORMAT_YVYU }, + /* \todo NV42 is used in libcamera but is not mapped in GStreamer yet. */ +}; + +static inline GstVideoFormat +drm_to_gst_format(guint drm_fourcc) +{ + for (const auto &item : format_map) + if (item.drm_fourcc == drm_fourcc) + return item.gst_format; + return GST_VIDEO_FORMAT_UNKNOWN; +} + +static GstStructure * +bare_structure_from_fourcc(guint fourcc) +{ + GstVideoFormat gst_format = drm_to_gst_format(fourcc); + + if (gst_format == GST_VIDEO_FORMAT_UNKNOWN) + return nullptr; + + if (gst_format != GST_VIDEO_FORMAT_ENCODED) + return gst_structure_new("video/x-raw", "format", G_TYPE_STRING, + gst_video_format_to_string(gst_format), nullptr); + + switch (fourcc) { + case DRM_FORMAT_MJPEG: + return gst_structure_new_empty("image/jpeg"); + default: + return nullptr; + } +} + +GstCaps * +gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) +{ + GstCaps *caps = gst_caps_new_empty(); + + for (unsigned int fourcc : formats.pixelformats()) { + g_autoptr(GstStructure) bare_s = bare_structure_from_fourcc(fourcc); + + if (!bare_s) { + GST_WARNING("Unsupported DRM format %" GST_FOURCC_FORMAT, + GST_FOURCC_ARGS(fourcc)); + continue; + } + + for (const Size &size : formats.sizes(fourcc)) { + GstStructure *s = gst_structure_copy(bare_s); + gst_structure_set(s, + "width", G_TYPE_INT, size.width, + "height", G_TYPE_INT, size.height, + nullptr); + gst_caps_append_structure(caps, s); + } + + const SizeRange &range = formats.range(fourcc); + if (range.hStep && range.vStep) { + GstStructure *s = gst_structure_copy(bare_s); + gst_structure_set(s, + "width", GST_TYPE_INT_RANGE, range.min.width, range.max.width, range.hStep, + "height", GST_TYPE_INT_RANGE, range.min.height, range.max.height, range.vStep, + nullptr); + gst_caps_append_structure(caps, s); + } + } + + return caps; +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h new file mode 100644 index 0000000..33160b8 --- /dev/null +++ b/src/gstreamer/gstlibcamera-utils.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera-utils.h - GStreamer libcamera Utility Functions + */ + +#ifndef __GST_LIBCAMERA_UTILS_H_ + +#include +#include + +#include + +GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); + +#endif /* __GST_LIBCAMERA_UTILS_H_ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 832b8a5..f409107 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,4 +1,5 @@ libcamera_gst_sources = [ + 'gstlibcamera-utils.cpp', 'gstlibcamera.c', 'gstlibcamerasrc.cpp', ] From patchwork Thu Feb 27 20:03:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2885 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40C40626D0 for ; Thu, 27 Feb 2020 21:04:19 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C2CEF29654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:43 -0500 Message-Id: <20200227200407.490616-4-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/27] gst: Add initial device provider X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:19 -0000 This feature is used with GstDeviceMonitor in order to enumerate and monitor devices to be used with the source element. The resulting GstDevice implementation is also used by application to abstract the configuration of the source element. Implementation notes: - libcamera does not support polling yet - The device ID isn't unique in libcamera yet - The "name" property does not yet exist in libcamerasrc yet Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera.c | 10 +- src/gstreamer/gstlibcameraprovider.cpp | 235 +++++++++++++++++++++++++ src/gstreamer/gstlibcameraprovider.h | 23 +++ src/gstreamer/meson.build | 1 + 4 files changed, 267 insertions(+), 2 deletions(-) create mode 100644 src/gstreamer/gstlibcameraprovider.cpp create mode 100644 src/gstreamer/gstlibcameraprovider.h diff --git a/src/gstreamer/gstlibcamera.c b/src/gstreamer/gstlibcamera.c index 7dd94ca..81c7bb1 100644 --- a/src/gstreamer/gstlibcamera.c +++ b/src/gstreamer/gstlibcamera.c @@ -6,13 +6,19 @@ * gstlibcamera.c - GStreamer plugin */ +#include "gstlibcameraprovider.h" #include "gstlibcamerasrc.h" static gboolean plugin_init(GstPlugin *plugin) { - return gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, - GST_TYPE_LIBCAMERA_SRC); + if (!gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_SRC) || + !gst_device_provider_register(plugin, "libcameraprovider", + GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_PROVIDER)) + return FALSE; + return TRUE; } diff --git a/src/gstreamer/gstlibcameraprovider.cpp b/src/gstreamer/gstlibcameraprovider.cpp new file mode 100644 index 0000000..3dcac59 --- /dev/null +++ b/src/gstreamer/gstlibcameraprovider.cpp @@ -0,0 +1,235 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraprovider.c - GStreamer Device Provider + */ + +#include "gstlibcamera-utils.h" +#include "gstlibcameraprovider.h" +#include "gstlibcamerasrc.h" + +#include +#include + +using namespace libcamera; + +GST_DEBUG_CATEGORY_STATIC(provider_debug); +#define GST_CAT_DEFAULT provider_debug + +/** + * \struct _GstLibcameraDevice + * \brief libcamera GstDevice implementation + * + * This object will be used by GstLibcameraProder to abstract a libcamera + * device. It also provides helpers to create and configure the + * libcamerasrc GstElement to be used with this device. The implementation is + * private to the plugin. + */ + +enum { + PROP_DEVICE_NAME = 1, +}; + +#define GST_TYPE_LIBCAMERA_DEVICE gst_libcamera_device_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraDevice, gst_libcamera_device, + GST_LIBCAMERA, DEVICE, GstDevice); + +struct _GstLibcameraDevice { + GstDevice parent; + gchar *name; +}; + +G_DEFINE_TYPE(GstLibcameraDevice, gst_libcamera_device, GST_TYPE_DEVICE); + +static GstElement * +gst_libcamera_device_create_element(GstDevice *device, const gchar *name) +{ + GstElement *source = gst_element_factory_make("libcamerasrc", name); + + /* Provider and source lives in the same plugin, so making the source + * should never fail. */ + g_assert(source); + + g_object_set(source, "camera-name", GST_LIBCAMERA_DEVICE(device)->name, nullptr); + + return source; +} + +static gboolean +gst_libcamera_device_reconfigure_element(GstDevice *device, + GstElement *element) +{ + if (!GST_LIBCAMERA_IS_SRC(element)) + return FALSE; + + g_object_set(element, "camera-name", GST_LIBCAMERA_DEVICE(device)->name, nullptr); + + return TRUE; +} + +static void +gst_libcamera_device_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + GstLibcameraDevice *device = GST_LIBCAMERA_DEVICE(object); + + switch (prop_id) { + case PROP_DEVICE_NAME: + device->name = g_value_dup_string(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_device_init(GstLibcameraDevice *self) +{ +} + +static void +gst_libcamera_device_finalize(GObject *object) +{ + GstLibcameraDevice *self = GST_LIBCAMERA_DEVICE(object); + gpointer klass = gst_libcamera_device_parent_class; + + g_free(self->name); + + G_OBJECT_GET_CLASS(klass)->finalize(object); +} + +static void +gst_libcamera_device_class_init(GstLibcameraDeviceClass *klass) +{ + GstDeviceClass *device_class = GST_DEVICE_CLASS(klass); + GObjectClass *object_class = G_OBJECT_CLASS(klass); + + device_class->create_element = gst_libcamera_device_create_element; + device_class->reconfigure_element = gst_libcamera_device_reconfigure_element; + + object_class->set_property = gst_libcamera_device_set_property; + object_class->finalize = gst_libcamera_device_finalize; + + GParamSpec *pspec = g_param_spec_string("name", "Name", + "The name of the camera device", "", + (GParamFlags)(G_PARAM_STATIC_STRINGS | G_PARAM_WRITABLE | + G_PARAM_CONSTRUCT_ONLY)); + g_object_class_install_property(object_class, PROP_DEVICE_NAME, pspec); +} + +static GstDevice * +gst_libcamera_device_new(const std::shared_ptr &camera) +{ + g_autoptr(GstCaps) caps = gst_caps_new_empty(); + const gchar *name = camera->name().c_str(); + StreamRoles roles; + + roles.push_back(StreamRole::VideoRecording); + std::unique_ptr config = camera->generateConfiguration(roles); + + for (const StreamConfiguration &stream_cfg : *config) { + GstCaps *sub_caps = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); + if (sub_caps) + gst_caps_append(caps, sub_caps); + } + + return GST_DEVICE(g_object_new(GST_TYPE_LIBCAMERA_DEVICE, + /* \todo Use a unique identifier instead of camera name. */ + "name", name, + "display-name", name, + "caps", caps, + "device-class", "Source/Video", + nullptr)); +} + +/** + * \struct _GstLibcameraProvider + * \brief libcamera GstDeviceProvider implementation + * + * This GstFeature will be used by GstDeviceMonitor to probe the available + * libcamera devices. The implementation is private to the plugin. + */ + +struct _GstLibcameraProvider { + GstDeviceProvider parent; + CameraManager *cm; +}; + +G_DEFINE_TYPE_WITH_CODE(GstLibcameraProvider, gst_libcamera_provider, + GST_TYPE_DEVICE_PROVIDER, + GST_DEBUG_CATEGORY_INIT(provider_debug, "libcamera-provider", 0, + "libcamera Device Provider")); + +static GList * +gst_libcamera_provider_probe(GstDeviceProvider *provider) +{ + GstLibcameraProvider *self = GST_LIBCAMERA_PROVIDER(provider); + CameraManager *cm = self->cm; + GList *devices = nullptr; + gint ret; + + GST_INFO_OBJECT(self, "Probing cameras using libcamera"); + + /* \todo Move the CameraMananger start()/stop() calls into + * GstDeviceProfiler start()/stop() virtual function when CameraMananger + * gains monitoring support. Meanwhile we need to cycle start()/stop() + * to ensure every probe() calls return the latest list. + */ + ret = cm->start(); + if (ret) { + GST_ERROR_OBJECT(self, "Failed to retrieve device list: %s", + g_strerror(-ret)); + return nullptr; + } + + for (const std::shared_ptr &camera : cm->cameras()) { + GST_INFO_OBJECT(self, "Found camera '%s'", camera->name().c_str()); + devices = g_list_append(devices, + g_object_ref_sink(gst_libcamera_device_new(camera))); + } + + cm->stop(); + + return devices; +} + +static void +gst_libcamera_provider_init(GstLibcameraProvider *self) +{ + GstDeviceProvider *provider = GST_DEVICE_PROVIDER(self); + + self->cm = new CameraManager(); + + /* Avoid devices being duplicated. */ + gst_device_provider_hide_provider(provider, "v4l2deviceprovider"); +} + +static void +gst_libcamera_provider_finalize(GObject *object) +{ + GstLibcameraProvider *self = GST_LIBCAMERA_PROVIDER(object); + gpointer klass = gst_libcamera_provider_parent_class; + + delete self->cm; + + return G_OBJECT_GET_CLASS(klass)->finalize(object); +} + +static void +gst_libcamera_provider_class_init(GstLibcameraProviderClass *klass) +{ + GstDeviceProviderClass *provider_class = GST_DEVICE_PROVIDER_CLASS(klass); + GObjectClass *object_class = G_OBJECT_CLASS(klass); + + provider_class->probe = gst_libcamera_provider_probe; + object_class->finalize = gst_libcamera_provider_finalize; + + gst_device_provider_class_set_metadata(provider_class, + "libcamera Device Provider", + "Source/Video", + "List camera device using libcamera", + "Nicolas Dufresne "); +} diff --git a/src/gstreamer/gstlibcameraprovider.h b/src/gstreamer/gstlibcameraprovider.h new file mode 100644 index 0000000..bdd19db --- /dev/null +++ b/src/gstreamer/gstlibcameraprovider.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraprovider.h - GStreamer Device Provider + */ + +#ifndef __GST_LIBCAMERA_PROVIDER_H__ +#define __GST_LIBCAMERA_PROVIDER_H__ + +#include + +G_BEGIN_DECLS + +#define GST_TYPE_LIBCAMERA_PROVIDER gst_libcamera_provider_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraProvider, gst_libcamera_provider, + GST_LIBCAMERA, PROVIDER, GstDeviceProvider) + +G_END_DECLS + +#endif /* __GST_LIBCAMERA_PROVIDER_H__ */ + diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index f409107..a57dd85 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,6 +1,7 @@ libcamera_gst_sources = [ 'gstlibcamera-utils.cpp', 'gstlibcamera.c', + 'gstlibcameraprovider.cpp', 'gstlibcamerasrc.cpp', ] From patchwork Thu Feb 27 20:03:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2886 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B845762708 for ; Thu, 27 Feb 2020 21:04:19 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 3A1342949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:44 -0500 Message-Id: <20200227200407.490616-5-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/27] gst: utils: Add simple scoped lockers for GMutex and GRectMutex X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:21 -0000 While GLib has locker implementation already using g_autoptr(), recursive mutext locker was onmly introduced in recent GLib. Implement a simple locker for GMutex and GRectMutex in order to allow making locking simplier and safer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.h | 51 ++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 33160b8..737ca63 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -15,4 +15,55 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); +/** + * \class GLibLocker + * \brief A simple scoped mutex locker for GMutex + */ +class GLibLocker +{ +public: + GLibLocker(GMutex *mutex) + : mutex_(mutex) + { + g_mutex_lock(mutex_); + } + + GLibLocker(GstObject *object) + : mutex_(GST_OBJECT_GET_LOCK(object)) + { + g_mutex_lock(mutex_); + } + + ~GLibLocker() + { + g_mutex_unlock(mutex_); + } + +private: + GMutex *mutex_; +}; + +/** + * \class GLibRecLocker + * \brief A simple scoped mutex locker for GRecMutex + */ +class GLibRecLocker +{ +public: + GLibRecLocker(GRecMutex *mutex) + : mutex_(mutex) + { + g_rec_mutex_lock(mutex_); + } + + ~GLibRecLocker() + { + g_rec_mutex_unlock(mutex_); + } + +private: + GRecMutex *mutex_; +}; + + #endif /* __GST_LIBCAMERA_UTILS_H_ */ From patchwork Thu Feb 27 20:03:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2887 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 23274626FF for ; Thu, 27 Feb 2020 21:04:20 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A513529654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:45 -0500 Message-Id: <20200227200407.490616-6-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/27] gst: Add pads to the source X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:21 -0000 This simply adds the boiler plate for pads on the source element. The design is that we have one pad, called "src", that will always be present, and then more pads can be requested prior in READY or less state. Initially pads have one property "stream-role" that let you decide which role this pad will have. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 101 ++++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 19 ++++++ src/gstreamer/gstlibcamerasrc.cpp | 19 ++++++ src/gstreamer/meson.build | 1 + 4 files changed, 140 insertions(+) create mode 100644 src/gstreamer/gstlibcamerapad.cpp create mode 100644 src/gstreamer/gstlibcamerapad.h diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp new file mode 100644 index 0000000..6212c3b --- /dev/null +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapad.cpp - GStreamer Capture Pad + */ + +#include "gstlibcamerapad.h" +#include "gstlibcamera-utils.h" + +#include + +using namespace libcamera; + +struct _GstLibcameraPad { + GstPad parent; + StreamRole role; +}; + +enum { + PROP_0, + PROP_STREAM_ROLE +}; + +G_DEFINE_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_TYPE_PAD); + +static void +gst_libcamera_pad_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + auto *self = GST_LIBCAMERA_PAD(object); + GLibLocker lock(GST_OBJECT(self)); + + switch (prop_id) { + case PROP_STREAM_ROLE: + self->role = (StreamRole)g_value_get_enum(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_pad_get_property(GObject *object, guint prop_id, GValue *value, + GParamSpec *pspec) +{ + auto *self = GST_LIBCAMERA_PAD(object); + GLibLocker lock(GST_OBJECT(self)); + + switch (prop_id) { + case PROP_STREAM_ROLE: + g_value_set_enum(value, self->role); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_pad_init(GstLibcameraPad *self) +{ +} + +static GType +gst_libcamera_stream_role_get_type(void) +{ + static GType type = 0; + static const GEnumValue values[] = { + { StillCapture, "libcamera::StillCapture", "still-capture" }, + { VideoRecording, "libcamera::VideoRecording", "video-recording" }, + { Viewfinder, "libcamera::Viewfinder", "view-finder" }, + { 0, NULL, NULL } + }; + + if (!type) + type = g_enum_register_static("GstLibcameraStreamRole", values); + + return type; +} + +static void +gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) +{ + auto *object_class = G_OBJECT_CLASS(klass); + + object_class->set_property = gst_libcamera_pad_set_property; + object_class->get_property = gst_libcamera_pad_get_property; + + auto *spec = g_param_spec_enum("stream-role", "Stream Role", + "The selected stream role", + gst_libcamera_stream_role_get_type(), + VideoRecording, + (GParamFlags)(GST_PARAM_MUTABLE_READY + | G_PARAM_CONSTRUCT + | G_PARAM_READWRITE + | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_STREAM_ROLE, spec); +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h new file mode 100644 index 0000000..2e745f1 --- /dev/null +++ b/src/gstreamer/gstlibcamerapad.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapad.h - GStreamer Capture Element + */ + +#include + +#ifndef __GST_LIBCAMERA_PAD_H__ +#define __GST_LIBCAMERA_PAD_H__ + +#define GST_TYPE_LIBCAMERA_PAD gst_libcamera_pad_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, + GST_LIBCAMERA, PAD, GstPad) + + +#endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 3807503..085c489 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -6,6 +6,7 @@ * gstlibcamerasrc.cpp - GStreamer Capture Element */ +#include "gstlibcamerapad.h" #include "gstlibcamerasrc.h" struct _GstLibcameraSrc { @@ -14,6 +15,18 @@ struct _GstLibcameraSrc { G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); +#define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw; image/jpeg") + +/* For the simple case, we have a src pad that is always present. */ +GstStaticPadTemplate src_template = { + "src", GST_PAD_SRC, GST_PAD_ALWAYS, TEMPLATE_CAPS +}; + +/* More pads can be requested in state < PAUSED */ +GstStaticPadTemplate request_src_template = { + "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS +}; + static void gst_libcamera_src_init(GstLibcameraSrc *self) { @@ -28,4 +41,10 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) "libcamera Source", "Source/Video", "Linux Camera source using libcamera", "Nicolas Dufresne X-Patchwork-Id: 2888 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 993AA626F6 for ; Thu, 27 Feb 2020 21:04:20 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1C4F02949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:46 -0500 Message-Id: <20200227200407.490616-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/27] gst: libcamerasrc: Allocate and add static pad X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:22 -0000 This pad will always be present and will allow simple pipeline to be used to stream from the camera. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 085c489..fb403cf 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -11,6 +11,7 @@ struct _GstLibcameraSrc { GstElement parent; + GstPad *srcpad; }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); @@ -30,6 +31,10 @@ GstStaticPadTemplate request_src_template = { static void gst_libcamera_src_init(GstLibcameraSrc *self) { + GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); + + self->srcpad = gst_pad_new_from_template(templ, "src"); + gst_element_add_pad(GST_ELEMENT(self), self->srcpad); } static void From patchwork Thu Feb 27 20:03:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2889 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 156896272D for ; Thu, 27 Feb 2020 21:04:21 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 9FB1529654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:47 -0500 Message-Id: <20200227200407.490616-8-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/27] gst: libcamerasrc: Add camera-name property X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:22 -0000 This property will be used to select by name the camera to use. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 68 ++++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index fb403cf..55ed174 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -6,12 +6,19 @@ * gstlibcamerasrc.cpp - GStreamer Capture Element */ +#include "gstlibcamera-utils.h" #include "gstlibcamerapad.h" #include "gstlibcamerasrc.h" struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; + gchar *camera_name; +}; + +enum { + PROP_0, + PROP_CAMERA_NAME }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); @@ -28,6 +35,52 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +static void +gst_libcamera_src_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + GLibLocker lock(GST_OBJECT(object)); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + switch (prop_id) { + case PROP_CAMERA_NAME: + g_free(self->camera_name); + self->camera_name = g_value_dup_string(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, + GParamSpec *pspec) +{ + GLibLocker lock(GST_OBJECT(object)); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + switch (prop_id) { + case PROP_CAMERA_NAME: + g_value_set_string(value, self->camera_name); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_src_finalize(GObject *object) +{ + GObjectClass *klass = G_OBJECT_CLASS(gst_libcamera_src_parent_class); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + g_free(self->camera_name); + + return klass->finalize(object); +} + static void gst_libcamera_src_init(GstLibcameraSrc *self) { @@ -40,7 +93,12 @@ gst_libcamera_src_init(GstLibcameraSrc *self) static void gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) { - GstElementClass *element_class = (GstElementClass *)klass; + GstElementClass *element_class = GST_ELEMENT_CLASS(klass); + GObjectClass *object_class = G_OBJECT_CLASS(klass); + + object_class->set_property = gst_libcamera_src_set_property; + object_class->get_property = gst_libcamera_src_get_property; + object_class->finalize = gst_libcamera_src_finalize; gst_element_class_set_metadata(element_class, "libcamera Source", "Source/Video", @@ -52,4 +110,12 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) gst_element_class_add_static_pad_template_with_gtype(element_class, &request_src_template, GST_TYPE_LIBCAMERA_PAD); + + GParamSpec *spec = g_param_spec_string("camera-name", "Camera Name", + "Select by name which camera to use.", nullptr, + (GParamFlags)(GST_PARAM_MUTABLE_READY + | G_PARAM_CONSTRUCT + | G_PARAM_READWRITE + | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_CAMERA_NAME, spec); } From patchwork Thu Feb 27 20:03:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2890 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9C7B626DF for ; Thu, 27 Feb 2020 21:04:21 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 176D429654B From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:48 -0500 Message-Id: <20200227200407.490616-9-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/27] gst: libcamerasrc: Add a debug category X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:23 -0000 This will allow selecting libcamerasrc traces with the following environment: GST_DEBUG=libcamerasrc:7 Or all libcamera GStreamer element traces using GST_DEBUG="libcamera*:7" Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 55ed174..c993143 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -10,6 +10,9 @@ #include "gstlibcamerapad.h" #include "gstlibcamerasrc.h" +GST_DEBUG_CATEGORY_STATIC(source_debug); +#define GST_CAT_DEFAULT source_debug + struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; @@ -21,7 +24,9 @@ enum { PROP_CAMERA_NAME }; -G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); +G_DEFINE_TYPE_WITH_CODE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT, + GST_DEBUG_CATEGORY_INIT(source_debug, "libcamerasrc", 0, + "libcamera Source")); #define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw; image/jpeg") From patchwork Thu Feb 27 20:03:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2891 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EA0C62730 for ; Thu, 27 Feb 2020 21:04:22 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 9854B29654C From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:49 -0500 Message-Id: <20200227200407.490616-10-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/27] gst: libcamerasrc: Implement selection and acquisition X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:23 -0000 This add code to select and acquire a camera. With this, it is now possible to run pipeline like: gst-launch-1.0 libcamerasrc ! fakesink Though no buffer will be streamed yet. In this function, we implement the change_state() virtual method to trigger actions on specific state transitions. Note that we also return GST_STATE_CHANGE_NO_PREROLL in GST_STATE_CHANGE_READY_TO_PAUSED and GST_STATE_CHANGE_PLAYING_TO_PAUSED transitions as this is required for all live sources. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 125 ++++++++++++++++++++++++++++++ 1 file changed, 125 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index c993143..0c60478 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -10,13 +10,26 @@ #include "gstlibcamerapad.h" #include "gstlibcamerasrc.h" +#include +#include + +using namespace libcamera; + GST_DEBUG_CATEGORY_STATIC(source_debug); #define GST_CAT_DEFAULT source_debug +/* Used for C++ object with destructors. */ +struct GstLibcameraSrcState { + std::unique_ptr cm; + std::shared_ptr cam; +}; + struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; gchar *camera_name; + + GstLibcameraSrcState *state; }; enum { @@ -40,6 +53,83 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +static bool +gst_libcamera_src_open(GstLibcameraSrc *self) +{ + std::unique_ptr cm = std::make_unique(); + std::shared_ptr cam; + gint ret = 0; + + GST_DEBUG_OBJECT(self, "Opening camera device ..."); + + ret = cm->start(); + if (ret) { + GST_ELEMENT_ERROR(self, LIBRARY, INIT, + ("Failed listing cameras."), + ("libcamera::CameraMananger::start() failed: %s", g_strerror(-ret))); + return false; + } + + g_autofree gchar *camera_name = nullptr; + { + GLibLocker lock(GST_OBJECT(self)); + if (self->camera_name) + camera_name = g_strdup(self->camera_name); + } + + if (camera_name) { + cam = cm->get(self->camera_name); + if (!cam) { + GST_ELEMENT_ERROR(self, RESOURCE, NOT_FOUND, + ("Could not find a camera named '%s'.", self->camera_name), + ("libcamera::CameraMananger::get() returned nullptr")); + return false; + } + } else { + if (cm->cameras().empty()) { + GST_ELEMENT_ERROR(self, RESOURCE, NOT_FOUND, + ("Could not find any supported camera on this system."), + ("libcamera::CameraMananger::cameras() is empty")); + return false; + } + cam = cm->cameras()[0]; + } + + GST_INFO_OBJECT(self, "Using camera named '%s'", cam->name().c_str()); + + ret = cam->acquire(); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, BUSY, + ("Camera name '%s' is already in use.", cam->name().c_str()), + ("libcamera::Camera::acquire() failed: %s", g_strerror(ret))); + return false; + } + + /* No need to lock here, we didn't start our threads yet. */ + self->state->cm = std::move(cm); + self->state->cam = cam; + + return true; +} + +static void +gst_libcamera_src_close(GstLibcameraSrc *self) +{ + GstLibcameraSrcState *state = self->state; + gint ret; + + ret = state->cam->release(); + if (ret) { + GST_ELEMENT_WARNING(self, RESOURCE, BUSY, + ("Camera name '%s' is still in use.", state->cam->name().c_str()), + ("libcamera::Camera.release() failed: %s", g_strerror(-ret))); + } + + state->cam.reset(); + state->cm->stop(); + state->cm.reset(); +} + static void gst_libcamera_src_set_property(GObject *object, guint prop_id, const GValue *value, GParamSpec *pspec) @@ -75,6 +165,36 @@ gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, } } +static GstStateChangeReturn +gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(element); + GstStateChangeReturn ret = GST_STATE_CHANGE_SUCCESS; + GstElementClass *klass = GST_ELEMENT_CLASS(gst_libcamera_src_parent_class); + + ret = klass->change_state(element, transition); + if (ret == GST_STATE_CHANGE_FAILURE) + return ret; + + switch (transition) { + case GST_STATE_CHANGE_NULL_TO_READY: + if (!gst_libcamera_src_open(self)) + return GST_STATE_CHANGE_FAILURE; + break; + case GST_STATE_CHANGE_READY_TO_PAUSED: + case GST_STATE_CHANGE_PLAYING_TO_PAUSED: + ret = GST_STATE_CHANGE_NO_PREROLL; + break; + case GST_STATE_CHANGE_READY_TO_NULL: + gst_libcamera_src_close(self); + break; + default: + break; + } + + return ret; +} + static void gst_libcamera_src_finalize(GObject *object) { @@ -82,6 +202,7 @@ gst_libcamera_src_finalize(GObject *object) GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); g_free(self->camera_name); + delete self->state; return klass->finalize(object); } @@ -89,10 +210,12 @@ gst_libcamera_src_finalize(GObject *object) static void gst_libcamera_src_init(GstLibcameraSrc *self) { + GstLibcameraSrcState *state = new GstLibcameraSrcState(); GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); self->srcpad = gst_pad_new_from_template(templ, "src"); gst_element_add_pad(GST_ELEMENT(self), self->srcpad); + self->state = state; } static void @@ -105,6 +228,8 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) object_class->get_property = gst_libcamera_src_get_property; object_class->finalize = gst_libcamera_src_finalize; + element_class->change_state = gst_libcamera_src_change_state; + gst_element_class_set_metadata(element_class, "libcamera Source", "Source/Video", "Linux Camera source using libcamera", From patchwork Thu Feb 27 20:03:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2892 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F7926271D for ; Thu, 27 Feb 2020 21:04:22 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 127EF29654F From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:50 -0500 Message-Id: <20200227200407.490616-11-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/27] gst: libcamerasrc: Add a task for the streaming thread X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:23 -0000 Use a GstTask as our internal streaming thread. Unlike GstBaseSrc, we will be running a streaming thread at the element level rather than per pad. This is needed to combine buffer request for multiple pads. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 49 +++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 0c60478..53ece26 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -26,7 +26,11 @@ struct GstLibcameraSrcState { struct _GstLibcameraSrc { GstElement parent; + + GRecMutex stream_lock; + GstTask *task; GstPad *srcpad; + gchar *camera_name; GstLibcameraSrcState *state; @@ -112,6 +116,30 @@ gst_libcamera_src_open(GstLibcameraSrc *self) return true; } +static void +gst_libcamera_src_task_run(gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread is now capturing"); +} + +static void +gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread has started"); +} + +static void +gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); +} + static void gst_libcamera_src_close(GstLibcameraSrc *self) { @@ -182,9 +210,22 @@ gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) return GST_STATE_CHANGE_FAILURE; break; case GST_STATE_CHANGE_READY_TO_PAUSED: + /* This needs to be called after pads activation.*/ + if (!gst_task_pause(self->task)) + return GST_STATE_CHANGE_FAILURE; + ret = GST_STATE_CHANGE_NO_PREROLL; + break; case GST_STATE_CHANGE_PLAYING_TO_PAUSED: ret = GST_STATE_CHANGE_NO_PREROLL; break; + case GST_STATE_CHANGE_PAUSED_TO_READY: + /* \todo this might requires some thread unblocking in the future + * if the streaming thread starts doing any kind of blocking + * operations. If this was the case, we would need to do so + * before pad deactivation, so before chaining to the parent + * change_state function. */ + gst_task_join(self->task); + break; case GST_STATE_CHANGE_READY_TO_NULL: gst_libcamera_src_close(self); break; @@ -201,6 +242,8 @@ gst_libcamera_src_finalize(GObject *object) GObjectClass *klass = G_OBJECT_CLASS(gst_libcamera_src_parent_class); GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + g_rec_mutex_clear(&self->stream_lock); + g_clear_object(&self->task); g_free(self->camera_name); delete self->state; @@ -213,6 +256,12 @@ gst_libcamera_src_init(GstLibcameraSrc *self) GstLibcameraSrcState *state = new GstLibcameraSrcState(); GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); + g_rec_mutex_init(&self->stream_lock); + self->task = gst_task_new(gst_libcamera_src_task_run, self, nullptr); + gst_task_set_enter_callback(self->task, gst_libcamera_src_task_enter, self, nullptr); + gst_task_set_leave_callback(self->task, gst_libcamera_src_task_leave, self, nullptr); + gst_task_set_lock(self->task, &self->stream_lock); + self->srcpad = gst_pad_new_from_template(templ, "src"); gst_element_add_pad(GST_ELEMENT(self), self->srcpad); self->state = state; From patchwork Thu Feb 27 20:03:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2893 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 110B0626FF for ; Thu, 27 Feb 2020 21:04:23 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7E6EA29654B From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:51 -0500 Message-Id: <20200227200407.490616-12-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 11/27] gst: libcamerapad: Add a method to access the role X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:23 -0000 Each pad can have a different roles. Users will have to request and configure their pads roles before moving higher state. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 8 ++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ 2 files changed, 10 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 6212c3b..d4f5b15 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -99,3 +99,11 @@ gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) | G_PARAM_STATIC_STRINGS)); g_object_class_install_property(object_class, PROP_STREAM_ROLE, spec); } + +StreamRole +gst_libcamera_pad_get_role(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + return self->role; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 2e745f1..3dea0e7 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -7,6 +7,7 @@ */ #include +#include #ifndef __GST_LIBCAMERA_PAD_H__ #define __GST_LIBCAMERA_PAD_H__ @@ -15,5 +16,6 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_LIBCAMERA, PAD, GstPad) +libcamera::StreamRole gst_libcamera_pad_get_role(GstPad *pad); #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Thu Feb 27 20:03:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2894 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 922E8626F6 for ; Thu, 27 Feb 2020 21:04:23 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 0A34D29654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:52 -0500 Message-Id: <20200227200407.490616-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 12/27] gst: libcamerasrc: Store the srcpad in a vector X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:24 -0000 This will allow implementing generic algorithm even if we cannot request pads yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 53ece26..5a86a6d 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -12,6 +12,7 @@ #include #include +#include using namespace libcamera; @@ -22,6 +23,7 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); struct GstLibcameraSrcState { std::unique_ptr cm; std::shared_ptr cam; + std::vector srcpads; }; struct _GstLibcameraSrc { @@ -29,7 +31,6 @@ struct _GstLibcameraSrc { GRecMutex stream_lock; GstTask *task; - GstPad *srcpad; gchar *camera_name; @@ -262,8 +263,8 @@ gst_libcamera_src_init(GstLibcameraSrc *self) gst_task_set_leave_callback(self->task, gst_libcamera_src_task_leave, self, nullptr); gst_task_set_lock(self->task, &self->stream_lock); - self->srcpad = gst_pad_new_from_template(templ, "src"); - gst_element_add_pad(GST_ELEMENT(self), self->srcpad); + state->srcpads.push_back(gst_pad_new_from_template(templ, "src")); + gst_element_add_pad(GST_ELEMENT(self), state->srcpads[0]); self->state = state; } From patchwork Thu Feb 27 20:03:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2895 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1FAD7626F6 for ; Thu, 27 Feb 2020 21:04:24 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 8A9AB29654B From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:53 -0500 Message-Id: <20200227200407.490616-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 13/27] gst: libcamerasrc: Send stream start event X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:24 -0000 Prior to sending caps, we need to send a stream-start event. This requires generating a stream and a group id. The stream id is random for live sources and the group id is shared across all pads. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 5a86a6d..e2c63d1 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -129,8 +129,19 @@ static void gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GLibRecLocker(&self->stream_lock); + GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread has started"); + + guint group_id = gst_util_group_id_next(); + for (GstPad *srcpad : state->srcpads) { + /* Create stream-id and push stream-start */ + g_autofree gchar *stream_id = gst_pad_create_stream_id(srcpad, GST_ELEMENT(self), nullptr); + GstEvent *event = gst_event_new_stream_start(stream_id); + gst_event_set_group_id(event, group_id); + gst_pad_push_event(srcpad, event); + } } static void From patchwork Thu Feb 27 20:03:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2896 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8A28F626F6 for ; Thu, 27 Feb 2020 21:04:24 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 16F1A2949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:54 -0500 Message-Id: <20200227200407.490616-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 14/27] gst: utils: Add StreamConfiguration helpers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:24 -0000 This add helpers to deal with the conversion from StreamConfiguration to caps and vis-versa. This is needed to implement caps negotiation. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.cpp | 64 ++++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 3 ++ 2 files changed, 67 insertions(+) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index dc129c3..231f3c5 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -40,6 +40,18 @@ drm_to_gst_format(guint drm_fourcc) return GST_VIDEO_FORMAT_UNKNOWN; } +static inline guint +gst_format_to_drm(GstVideoFormat gst_format) +{ + if (gst_format == GST_VIDEO_FORMAT_ENCODED) + return DRM_FORMAT_INVALID; + + for (const auto &item : format_map) + if (item.gst_format == gst_format) + return item.drm_fourcc; + return DRM_FORMAT_INVALID; +} + static GstStructure * bare_structure_from_fourcc(guint fourcc) { @@ -96,3 +108,55 @@ gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) return caps; } + +GstCaps * +gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg) +{ + GstCaps *caps = gst_caps_new_empty(); + GstStructure *s = bare_structure_from_fourcc(stream_cfg.pixelFormat); + + gst_structure_set(s, + "width", G_TYPE_INT, stream_cfg.size.width, + "height", G_TYPE_INT, stream_cfg.size.height, + nullptr); + gst_caps_append_structure(caps, s); + + return caps; +} + +void +gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, + GstCaps *caps) +{ + GstVideoFormat gst_format = drm_to_gst_format(stream_cfg.pixelFormat); + + /* First fixate the caps using default configuration value */ + g_assert(gst_caps_is_writable(caps)); + caps = gst_caps_truncate(caps); + GstStructure *s = gst_caps_get_structure(caps, 0); + + gst_structure_fixate_field_nearest_int(s, "width", stream_cfg.size.width); + gst_structure_fixate_field_nearest_int(s, "height", stream_cfg.size.height); + + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_video_format_to_string(gst_format); + gst_structure_fixate_field_string(s, "format", format); + } + + /* Then configure the stream with the result. */ + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_structure_get_string(s, "format"); + gst_format = gst_video_format_from_string(format); + stream_cfg.pixelFormat = gst_format_to_drm(gst_format); + } else if (gst_structure_has_name(s, "image/jpeg")) { + stream_cfg.pixelFormat = DRM_FORMAT_MJPEG; + } else { + g_critical("Unsupported media type: %s", gst_structure_get_name(s)); + } + + gint width, height; + gst_structure_get_int(s, "width", &width); + gst_structure_get_int(s, "height", &height); + stream_cfg.size.width = width; + stream_cfg.size.height = height; +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 737ca63..51898a0 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -14,6 +14,9 @@ #include GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); +GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); +void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, + GstCaps *caps); /** * \class GLibLocker From patchwork Thu Feb 27 20:03:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2897 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EC1A8626F6 for ; Thu, 27 Feb 2020 21:04:24 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 81D9329654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:55 -0500 Message-Id: <20200227200407.490616-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 15/27] gst: libcamerasrc: Implement minimal caps negotiation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:25 -0000 This is not expected to work in every possible cases, but should be sufficient as an initial implementation. What is does is that it turns the StreamFormats into caps and queries downstream caps with that as a filter. The result is the subset of caps that can be used. We then keep the first structure in that result and fixate using the default values found in StreamConfiguration as a default in case a range is available. We then validate this configuration and turn the potentially modified configuration into caps that we push downstream. Note that we trust the order in StreamFormats as being sorted best first, but this is not currently in libcamera. A todo has been added in the head of this file as a reminder to fix that in the core. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 82 ++++++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index e2c63d1..2c5c34c 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -6,6 +6,12 @@ * gstlibcamerasrc.cpp - GStreamer Capture Element */ +/** + * \todo libcamera UVC drivers picks the lowest possible resolution first, this + * should be fixed so that we get a decent resolution and framerate for the + * role by default. + */ + #include "gstlibcamera-utils.h" #include "gstlibcamerapad.h" #include "gstlibcamerasrc.h" @@ -23,6 +29,7 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); struct GstLibcameraSrcState { std::unique_ptr cm; std::shared_ptr cam; + std::unique_ptr config; std::vector srcpads; }; @@ -131,16 +138,89 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); GLibRecLocker(&self->stream_lock); GstLibcameraSrcState *state = self->state; + GstFlowReturn flow_ret = GST_FLOW_OK; GST_DEBUG_OBJECT(self, "Streaming thread has started"); guint group_id = gst_util_group_id_next(); + StreamRoles roles; for (GstPad *srcpad : state->srcpads) { - /* Create stream-id and push stream-start */ + /* Create stream-id and push stream-start .*/ g_autofree gchar *stream_id = gst_pad_create_stream_id(srcpad, GST_ELEMENT(self), nullptr); GstEvent *event = gst_event_new_stream_start(stream_id); gst_event_set_group_id(event, group_id); gst_pad_push_event(srcpad, event); + + /* Collect the streams roles for the next iteration .*/ + roles.push_back(gst_libcamera_pad_get_role(srcpad)); + } + + /* Generate the stream configurations, there should be one per pad .*/ + state->config = state->cam->generateConfiguration(roles); + /* \todo Check if camera may increase or decrease the number of streams + * regardless of the number of roles.*/ + g_assert(state->config->size() == state->srcpads.size()); + + for (gsize i = 0; i < state->srcpads.size(); i++) { + GstPad *srcpad = state->srcpads[i]; + StreamConfiguration &stream_cfg = state->config->at(i); + + /* Retrieve the supported caps. */ + g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); + g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad, filter); + if (gst_caps_is_empty(caps)) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + break; + } + + /* Fixate caps and configure the stream. */ + caps = gst_caps_make_writable(caps); + gst_libcamera_configure_stream_from_caps(stream_cfg, caps); + } + + if (flow_ret != GST_FLOW_OK) + goto done; + + /* Validate the configuration. */ + if (state->config->validate() == CameraConfiguration::Invalid) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + goto done; + } + + /* + * Regardless if it has been modified, create clean caps and push the + * caps event. Downstream will decide if the caps are acceptable. + */ + for (gsize i = 0; i < state->srcpads.size(); i++) { + GstPad *srcpad = state->srcpads[i]; + const StreamConfiguration &stream_cfg = state->config->at(i); + + g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg); + if (!gst_pad_push_event(srcpad, gst_event_new_caps(caps))) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + break; + } + } + + { + gint ret = state->cam->configure(state->config.get()); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Failed to configure camera: %s", g_strerror(-ret)), + ("Camera::configure() failed with error code %i", ret)); + gst_task_stop(task); + return; + } + } + +done: + switch (flow_ret) { + case GST_FLOW_NOT_NEGOTIATED: + GST_ELEMENT_FLOW_ERROR(self, flow_ret); + gst_task_stop(task); + return; + default: + break; } } From patchwork Thu Feb 27 20:03:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2898 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 779A4626F6 for ; Thu, 27 Feb 2020 21:04:25 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id ED5092949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:56 -0500 Message-Id: <20200227200407.490616-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 16/27] gst: libcamerasrc: Push segment event X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:25 -0000 Now that we have stream-start and caps, we can now push a segment event to announce what time will our buffer correlate to. For live sources this is just an open segment in time format. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 2c5c34c..4003a89 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -200,6 +200,11 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) flow_ret = GST_FLOW_NOT_NEGOTIATED; break; } + + /* Send an open segment event with time format. */ + GstSegment segment; + gst_segment_init(&segment, GST_FORMAT_TIME); + gst_pad_push_event(srcpad, gst_event_new_segment(&segment)); } { From patchwork Thu Feb 27 20:03:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2899 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 077E762727 for ; Thu, 27 Feb 2020 21:04:26 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7A0D429654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:57 -0500 Message-Id: <20200227200407.490616-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 17/27] gst: Add a pool and an allocator implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:26 -0000 This is needed to track the lifetime of the FrameBufferAllocator in relation to the GstBuffer/GstMemory objects travelling inside GStreamer. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcameraallocator.cpp | 244 ++++++++++++++++++++++++ src/gstreamer/gstlibcameraallocator.h | 29 +++ src/gstreamer/gstlibcamerapool.cpp | 109 +++++++++++ src/gstreamer/gstlibcamerapool.h | 26 +++ src/gstreamer/meson.build | 13 +- 5 files changed, 417 insertions(+), 4 deletions(-) create mode 100644 src/gstreamer/gstlibcameraallocator.cpp create mode 100644 src/gstreamer/gstlibcameraallocator.h create mode 100644 src/gstreamer/gstlibcamerapool.cpp create mode 100644 src/gstreamer/gstlibcamerapool.h diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp new file mode 100644 index 0000000..f268561 --- /dev/null +++ b/src/gstreamer/gstlibcameraallocator.cpp @@ -0,0 +1,244 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraallocator.cpp - GStreamer Custom Allocator + */ + +#include "gstlibcameraallocator.h" +#include "gstlibcamera-utils.h" + +#include +#include +#include + +using namespace libcamera; + +static gboolean gst_libcamera_allocator_release(GstMiniObject *mini_object); + +/** + * \struct FrameWrap + * \brief An internal wrapper to track the relation between FrameBuffer and + * GstMemory(s) + * + * This wrapper maintains a count of the outstanding GstMemory (there may be + * multiple GstMemory per FrameBuffer), and give back the FrameBuffer to the + * allocator pool when all memory objects have returned. + */ + +struct FrameWrap { + FrameWrap(GstAllocator *allocator, FrameBuffer *buffer, + gpointer stream); + ~FrameWrap(); + + void acquirePlane() { ++outstandingPlanes_; } + bool releasePlane() { return --outstandingPlanes_ == 0; } + + static GQuark getQuark(); + + gpointer stream_; + FrameBuffer *buffer_; + std::vector planes_; + gint outstandingPlanes_; +}; + +FrameWrap::FrameWrap(GstAllocator *allocator, FrameBuffer *buffer, + gpointer stream) + + : stream_(stream), + buffer_(buffer), + outstandingPlanes_(0) +{ + for (const FrameBuffer::Plane &plane : buffer->planes()) { + GstMemory *mem = gst_fd_allocator_alloc(allocator, plane.fd.fd(), plane.length, + GST_FD_MEMORY_FLAG_DONT_CLOSE); + gst_mini_object_set_qdata(GST_MINI_OBJECT(mem), FrameWrap::getQuark(), this, nullptr); + GST_MINI_OBJECT(mem)->dispose = gst_libcamera_allocator_release; + g_object_unref(mem->allocator); + planes_.push_back(mem); + } +} + +FrameWrap::~FrameWrap() +{ + for (GstMemory *mem : planes_) { + GST_MINI_OBJECT(mem)->dispose = nullptr; + g_object_ref(mem->allocator); + gst_memory_unref(mem); + } +} + +GQuark FrameWrap::getQuark(void) +{ + static gsize frame_quark = 0; + + if (g_once_init_enter(&frame_quark)) { + GQuark quark = g_quark_from_string("GstLibcameraFrameWrap"); + g_once_init_leave(&frame_quark, quark); + } + + return frame_quark; +} + + + +/** + * \struct _GstLibcameraAllocator + * \brief A pooling GstDmaBufAllocator for libcamera + * + * This is a pooling GstDmaBufAllocator implementation. This implementation override + * the dispose function of memory object in order to keep them alive when they + * are disposed by downstream elements. + */ +struct _GstLibcameraAllocator { + GstDmaBufAllocator parent; + FrameBufferAllocator *fb_allocator; + /* A hash table using Stream pointer as key and returning a GQueue of + * FrameWrap. */ + GHashTable *pools; +}; + +G_DEFINE_TYPE(GstLibcameraAllocator, gst_libcamera_allocator, + GST_TYPE_DMABUF_ALLOCATOR); + +static gboolean +gst_libcamera_allocator_release(GstMiniObject *mini_object) +{ + GstMemory *mem = GST_MEMORY_CAST(mini_object); + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(mem->allocator); + GLibLocker lock(GST_OBJECT(self)); + auto *frame = reinterpret_cast(gst_mini_object_get_qdata(mini_object, FrameWrap::getQuark())); + + gst_memory_ref(mem); + + if (frame->releasePlane()) { + auto *pool = reinterpret_cast(g_hash_table_lookup(self->pools, frame->stream_)); + g_return_val_if_fail(pool, TRUE); + g_queue_push_tail(pool, frame); + } + + /* Keep last in case we are holding on the last allocator ref. */ + g_object_unref(mem->allocator); + + /* Return FALSE so that our mini object isn't freed. */ + return FALSE; +} + +static void +gst_libcamera_allocator_free_pool(gpointer data) +{ + GQueue *queue = reinterpret_cast(data); + FrameWrap *frame; + + while ((frame = reinterpret_cast(g_queue_pop_head(queue)))) { + g_warn_if_fail(frame->outstandingPlanes_ == 0); + delete frame; + } + + g_queue_free(queue); +} + +static void +gst_libcamera_allocator_init(GstLibcameraAllocator *self) +{ + self->pools = g_hash_table_new_full(nullptr, nullptr, nullptr, + gst_libcamera_allocator_free_pool); + GST_OBJECT_FLAG_SET(self, GST_ALLOCATOR_FLAG_CUSTOM_ALLOC); +} + +static void +gst_libcamera_allocator_dispose(GObject *object) +{ + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object); + + if (self->pools) { + g_hash_table_unref(self->pools); + self->pools = nullptr; + } + + G_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->dispose(object); +} + +static void +gst_libcamera_allocator_finalize(GObject *object) +{ + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object); + + delete self->fb_allocator; + + G_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->finalize(object); +} + +static void +gst_libcamera_allocator_class_init(GstLibcameraAllocatorClass *klass) +{ + auto *allocator_class = GST_ALLOCATOR_CLASS(klass); + auto *object_class = G_OBJECT_CLASS(klass); + + object_class->dispose = gst_libcamera_allocator_dispose; + object_class->finalize = gst_libcamera_allocator_finalize; + allocator_class->alloc = nullptr; +} + +GstLibcameraAllocator * +gst_libcamera_allocator_new(std::shared_ptr camera) +{ + auto *self = (GstLibcameraAllocator *)g_object_new(GST_TYPE_LIBCAMERA_ALLOCATOR, + nullptr); + + self->fb_allocator = FrameBufferAllocator::create(camera); + for (Stream *stream : camera->streams()) { + gint ret; + + ret = self->fb_allocator->allocate(stream); + if (ret == 0) + return nullptr; + + GQueue *pool = g_queue_new(); + for (const std::unique_ptr &buffer : + self->fb_allocator->buffers(stream)) { + auto *fb = new FrameWrap(GST_ALLOCATOR(self), + buffer.get(), stream); + g_queue_push_tail(pool, fb); + } + + g_hash_table_insert(self->pools, stream, pool); + } + + return self; +} + +bool +gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, + Stream *stream, GstBuffer *buffer) +{ + GLibLocker lock(GST_OBJECT(self)); + + auto *pool = reinterpret_cast(g_hash_table_lookup(self->pools, stream)); + g_return_val_if_fail(pool, false); + + auto *frame = reinterpret_cast(g_queue_pop_head(pool)); + if (!frame) + return false; + + for (GstMemory *mem : frame->planes_) { + frame->acquirePlane(); + gst_buffer_append_memory(buffer, mem); + g_object_ref(mem->allocator); + } + + return true; +} + +gsize +gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self, + Stream *stream) +{ + GLibLocker lock(GST_OBJECT(self)); + + auto *pool = reinterpret_cast(g_hash_table_lookup(self->pools, stream)); + g_return_val_if_fail(pool, false); + + return pool->length; +} diff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h new file mode 100644 index 0000000..25cbf85 --- /dev/null +++ b/src/gstreamer/gstlibcameraallocator.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraallocator.h - GStreamer Custom Allocator + */ + +#ifndef __GST_LIBCAMERA_ALLOCATOR_H__ +#define __GST_LIBCAMERA_ALLOCATOR_H__ + +#include +#include +#include + +#define GST_TYPE_LIBCAMERA_ALLOCATOR gst_libcamera_allocator_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraAllocator, gst_libcamera_allocator, + GST_LIBCAMERA, ALLOCATOR, GstDmaBufAllocator) + +GstLibcameraAllocator *gst_libcamera_allocator_new(std::shared_ptr camera); + +bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, + libcamera::Stream *stream, + GstBuffer *buffer); + +gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator, + libcamera::Stream *stream); + +#endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */ diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp new file mode 100644 index 0000000..ee106a7 --- /dev/null +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapool.cpp - GStreamer Buffer Pool + */ + +#include "gstlibcamera-utils.h" +#include "gstlibcamerapool.h" + +#include + +using namespace libcamera; + +struct _GstLibcameraPool { + GstBufferPool parent; + + GstAtomicQueue *queue; + GstLibcameraAllocator *allocator; + Stream *stream; +}; + +G_DEFINE_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_TYPE_BUFFER_POOL); + +static GstFlowReturn +gst_libcamera_pool_acquire_buffer(GstBufferPool *pool, GstBuffer **buffer, + GstBufferPoolAcquireParams *params) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + GstBuffer *buf = GST_BUFFER(gst_atomic_queue_pop(self->queue)); + if (!buf) + return GST_FLOW_ERROR; + + if (!gst_libcamera_allocator_prepare_buffer(self->allocator, self->stream, buf)) + return GST_FLOW_ERROR; + + *buffer = buf; + return GST_FLOW_OK; +} + +static void +gst_libcamera_pool_reset_buffer(GstBufferPool *pool, GstBuffer *buffer) +{ + GstBufferPoolClass *klass = GST_BUFFER_POOL_CLASS(gst_libcamera_pool_parent_class); + + /* Clears all the memories and only pool the GstBuffer objects */ + gst_buffer_remove_all_memory(buffer); + klass->reset_buffer(pool, buffer); + GST_BUFFER_FLAGS(buffer) = 0; +} + +static void +gst_libcamera_pool_release_buffer(GstBufferPool *pool, GstBuffer *buffer) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + gst_atomic_queue_push(self->queue, buffer); +} + +static void +gst_libcamera_pool_init(GstLibcameraPool *self) +{ + self->queue = gst_atomic_queue_new(4); +} + +static void +gst_libcamera_pool_finalize(GObject *object) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(object); + GstBuffer *buf; + + while ((buf = GST_BUFFER(gst_atomic_queue_pop(self->queue)))) + gst_buffer_unref(buf); + + gst_atomic_queue_unref(self->queue); + g_object_unref(self->allocator); + + G_OBJECT_CLASS(gst_libcamera_pool_parent_class)->finalize(object); +} + +static void +gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass) +{ + auto *object_class = G_OBJECT_CLASS(klass); + auto *pool_class = GST_BUFFER_POOL_CLASS(klass); + + object_class->finalize = gst_libcamera_pool_finalize; + pool_class->start = nullptr; + pool_class->acquire_buffer = gst_libcamera_pool_acquire_buffer; + pool_class->reset_buffer = gst_libcamera_pool_reset_buffer; + pool_class->release_buffer = gst_libcamera_pool_release_buffer; +} + +GstLibcameraPool * +gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) +{ + auto *pool = GST_LIBCAMERA_POOL(g_object_new(GST_TYPE_LIBCAMERA_POOL, nullptr)); + + pool->allocator = GST_LIBCAMERA_ALLOCATOR(g_object_ref(allocator)); + pool->stream = stream; + + gsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream); + for (gsize i = 0; i < pool_size; i++) { + GstBuffer *buffer = gst_buffer_new(); + gst_atomic_queue_push(pool->queue, buffer); + } + + return pool; +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h new file mode 100644 index 0000000..a764c75 --- /dev/null +++ b/src/gstreamer/gstlibcamerapool.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapool.h - GStreamer Buffer Pool + * + * This is a partial implementation of GstBufferPool intended for internal use + * only. This pool cannot be configured or activated. + */ + +#ifndef __GST_LIBCAMERA_POOL_H__ +#define __GST_LIBCAMERA_POOL_H__ + +#include +#include + +#include "gstlibcameraallocator.h" + +#define GST_TYPE_LIBCAMERA_POOL gst_libcamera_pool_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstBufferPool) + +GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, + libcamera::Stream *stream); + +#endif /* __GST_LIBCAMERA_POOL_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 1c4a2e3..90773af 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,7 +1,9 @@ libcamera_gst_sources = [ 'gstlibcamera-utils.cpp', 'gstlibcamera.c', + 'gstlibcameraallocator.cpp', 'gstlibcamerapad.cpp', + 'gstlibcamerapool.cpp', 'gstlibcameraprovider.cpp', 'gstlibcamerasrc.cpp', ] @@ -11,14 +13,17 @@ libcamera_gst_c_args = [ '-DPACKAGE="@0@"'.format(meson.project_name()), ] -gst_dep = dependency('gstreamer-video-1.0', version : '>=1.16.1', - required : get_option('gstreamer')) +gst_req = '>=1.16.1' +gstvideo_dep = dependency('gstreamer-video-1.0', version : gst_req, + required : get_option('gstreamer')) +gstallocator_dep = dependency('gstreamer-allocators-1.0', version : gst_req, + required : get_option('gstreamer')) -if gst_dep.found() +if gstvideo_dep.found() and gstallocator_dep.found() libcamera_gst = shared_library('gstlibcamera', libcamera_gst_sources, c_args : libcamera_gst_c_args, - dependencies : [libcamera_dep, gst_dep], + dependencies : [libcamera_dep, gstvideo_dep, gstallocator_dep], install: true, install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')), ) From patchwork Thu Feb 27 20:03:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2900 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BD4A6274B for ; Thu, 27 Feb 2020 21:04:26 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 016FD2949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:58 -0500 Message-Id: <20200227200407.490616-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 18/27] gst: libcamerapad: Allow storing a pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:28 -0000 This adds get/set helper to store a pool on the pad. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 18 ++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 6 ++++++ 2 files changed, 24 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index d4f5b15..c4c466d 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -16,6 +16,7 @@ using namespace libcamera; struct _GstLibcameraPad { GstPad parent; StreamRole role; + GstLibcameraPool *pool; }; enum { @@ -107,3 +108,20 @@ gst_libcamera_pad_get_role(GstPad *pad) GLibLocker lock(GST_OBJECT(self)); return self->role; } + +GstLibcameraPool * +gst_libcamera_pad_get_pool(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + return self->pool; +} + +void +gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (self->pool) + g_object_unref(self->pool); + self->pool = pool; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 3dea0e7..4570c0c 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -9,6 +9,8 @@ #include #include +#include "gstlibcamerapool.h" + #ifndef __GST_LIBCAMERA_PAD_H__ #define __GST_LIBCAMERA_PAD_H__ @@ -18,4 +20,8 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, libcamera::StreamRole gst_libcamera_pad_get_role(GstPad *pad); +GstLibcameraPool *gst_libcamera_pad_get_pool(GstPad *pad); + +void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); + #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Thu Feb 27 20:03:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2901 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 02EA362732 for ; Thu, 27 Feb 2020 21:04:27 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 8150829654B From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:03:59 -0500 Message-Id: <20200227200407.490616-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 19/27] gst: libcamerasrc: Allocate and release buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:29 -0000 Setup the allocation and the release of buffers in the element. We have one pooling GstAllocator that wraps the FrameBufferAllocator and tracks the lifetime of FrameBuffer object. Then, for each pads we have a GstBufferPool object which is only used to avoid re-allocating the GstBuffer structure everytime we push a buffer. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerasrc.cpp | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 4003a89..fa9ff5b 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -13,7 +13,9 @@ */ #include "gstlibcamera-utils.h" +#include "gstlibcameraallocator.h" #include "gstlibcamerapad.h" +#include "gstlibcamerapool.h" #include "gstlibcamerasrc.h" #include @@ -42,6 +44,7 @@ struct _GstLibcameraSrc { gchar *camera_name; GstLibcameraSrcState *state; + GstLibcameraAllocator *allocator; }; enum { @@ -218,6 +221,23 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) } } + self->allocator = gst_libcamera_allocator_new(state->cam); + if (!self->allocator) { + GST_ELEMENT_ERROR(self, RESOURCE, NO_SPACE_LEFT, + ("Failed to allocate memory"), + ("gst_libcamera_allocator_new() failed.")); + gst_task_stop(task); + return; + } + + for (gsize i = 0; i < state->srcpads.size(); i++) { + GstPad *srcpad = state->srcpads[i]; + const StreamConfiguration &stream_cfg = state->config->at(i); + GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, + stream_cfg.stream()); + gst_libcamera_pad_set_pool(srcpad, pool); + } + done: switch (flow_ret) { case GST_FLOW_NOT_NEGOTIATED: @@ -233,8 +253,15 @@ static void gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); + state->cam->stop(); + + for (GstPad *srcpad : state->srcpads) + gst_libcamera_pad_set_pool(srcpad, NULL); + + g_clear_object(&self->allocator); } static void @@ -243,6 +270,8 @@ gst_libcamera_src_close(GstLibcameraSrc *self) GstLibcameraSrcState *state = self->state; gint ret; + GST_DEBUG_OBJECT(self, "Releasing resources"); + ret = state->cam->release(); if (ret) { GST_ELEMENT_WARNING(self, RESOURCE, BUSY, From patchwork Thu Feb 27 20:04:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2902 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A5946273C for ; Thu, 27 Feb 2020 21:04:27 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id EE40729654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:04:00 -0500 Message-Id: <20200227200407.490616-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 20/27] gst: Add getters for Stream and FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:30 -0000 This adds getters on pad/pool/allocator so that we can retrieve the Stream or FrameBuffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcameraallocator.cpp | 8 ++++++++ src/gstreamer/gstlibcameraallocator.h | 2 ++ src/gstreamer/gstlibcamerapad.cpp | 11 +++++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ src/gstreamer/gstlibcamerapool.cpp | 20 ++++++++++++++++++++ src/gstreamer/gstlibcamerapool.h | 7 +++++++ src/gstreamer/gstlibcamerasrc.cpp | 1 - 7 files changed, 50 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp index f268561..f87b52c 100644 --- a/src/gstreamer/gstlibcameraallocator.cpp +++ b/src/gstreamer/gstlibcameraallocator.cpp @@ -242,3 +242,11 @@ gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self, return pool->length; } + +FrameBuffer * +gst_libcamera_memory_get_frame_buffer(GstMemory *mem) +{ + auto *frame = reinterpret_cast(gst_mini_object_get_qdata(GST_MINI_OBJECT_CAST(mem), + FrameWrap::getQuark())); + return frame->buffer_; +} diff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h index 25cbf85..8839040 100644 --- a/src/gstreamer/gstlibcameraallocator.h +++ b/src/gstreamer/gstlibcameraallocator.h @@ -26,4 +26,6 @@ bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +libcamera::FrameBuffer *gst_libcamera_memory_get_frame_buffer(GstMemory *mem); + #endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */ diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index c4c466d..4f96546 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -125,3 +125,14 @@ gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool) g_object_unref(self->pool); self->pool = pool; } + +Stream * +gst_libcamera_pad_get_stream(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (self->pool) + return gst_libcamera_pool_get_stream(self->pool); + + return nullptr; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 4570c0c..81d0d58 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -24,4 +24,6 @@ GstLibcameraPool *gst_libcamera_pad_get_pool(GstPad *pad); void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); +libcamera::Stream *gst_libcamera_pad_get_stream(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp index ee106a7..f85c3aa 100644 --- a/src/gstreamer/gstlibcamerapool.cpp +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -107,3 +107,23 @@ gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) return pool; } + +Stream * +gst_libcamera_pool_get_stream(GstLibcameraPool *self) +{ + return self->stream; +} + +Stream * +gst_libcamera_buffer_get_stream(GstBuffer *buffer) +{ + auto *self = (GstLibcameraPool *)buffer->pool; + return self->stream; +} + +FrameBuffer * +gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer) +{ + GstMemory *mem = gst_buffer_peek_memory(buffer, 0); + return gst_libcamera_memory_get_frame_buffer(mem); +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h index a764c75..6fd2913 100644 --- a/src/gstreamer/gstlibcamerapool.h +++ b/src/gstreamer/gstlibcamerapool.h @@ -23,4 +23,11 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +libcamera::Stream *gst_libcamera_pool_get_stream(GstLibcameraPool *self); + +libcamera::Stream *gst_libcamera_buffer_get_stream(GstBuffer *buffer); + +libcamera::FrameBuffer *gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer); + + #endif /* __GST_LIBCAMERA_POOL_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index fa9ff5b..83f93cc 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -256,7 +256,6 @@ gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); - state->cam->stop(); for (GstPad *srcpad : state->srcpads) gst_libcamera_pad_set_pool(srcpad, NULL); From patchwork Thu Feb 27 20:04:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2903 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E96262730 for ; Thu, 27 Feb 2020 21:04:28 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7A8A529654B From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:04:01 -0500 Message-Id: <20200227200407.490616-22-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 21/27] gst: pad: Add method to store retrieve pending buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:31 -0000 These will be useful for streaming. The requestComplete callback will store the buffers on each pads so that the _run() can pick them up and push them through the pads from a streaming thread. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 27 +++++++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 4 ++++ 2 files changed, 31 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 4f96546..8662c92 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -17,6 +17,7 @@ struct _GstLibcameraPad { GstPad parent; StreamRole role; GstLibcameraPool *pool; + GQueue pending_buffers; }; enum { @@ -136,3 +137,29 @@ gst_libcamera_pad_get_stream(GstPad *pad) return nullptr; } + +void +gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + + g_queue_push_head(&self->pending_buffers, buffer); +} + +GstFlowReturn +gst_libcamera_pad_push_pending(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GstBuffer *buffer; + + { + GLibLocker lock(GST_OBJECT(self)); + buffer = GST_BUFFER(g_queue_pop_tail(&self->pending_buffers)); + } + + if (!buffer) + return GST_FLOW_CUSTOM_SUCCESS; + + return gst_pad_push(pad, buffer); +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 81d0d58..d928570 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -26,4 +26,8 @@ void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); libcamera::Stream *gst_libcamera_pad_get_stream(GstPad *pad); +void gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer); + +GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Thu Feb 27 20:04:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2904 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7541F6274B for ; Thu, 27 Feb 2020 21:04:28 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 074DE29654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:04:02 -0500 Message-Id: <20200227200407.490616-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 22/27] gst: libcamerasrc: Implement initial streaming X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:31 -0000 With this patch, the element is now able to push buffers to the next element in the graph. The buffers are currently missing any metadata like timestamp, sequence number. This will be added in the next commit. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerapad.cpp | 11 +- src/gstreamer/gstlibcamerapad.h | 2 + src/gstreamer/gstlibcamerasrc.cpp | 192 +++++++++++++++++++++++++++++- 3 files changed, 202 insertions(+), 3 deletions(-) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 8662c92..2cf1630 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -18,6 +18,7 @@ struct _GstLibcameraPad { StreamRole role; GstLibcameraPool *pool; GQueue pending_buffers; + GstClockTime latency; }; enum { @@ -159,7 +160,15 @@ gst_libcamera_pad_push_pending(GstPad *pad) } if (!buffer) - return GST_FLOW_CUSTOM_SUCCESS; + return GST_FLOW_OK; return gst_pad_push(pad, buffer); } + +bool +gst_libcamera_pad_has_pending(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + return (self->pending_buffers.length > 0); +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index d928570..eb24000 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -30,4 +30,6 @@ void gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer); GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); +bool gst_libcamera_pad_has_pending(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 83f93cc..70f2048 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -18,8 +18,10 @@ #include "gstlibcamerapool.h" #include "gstlibcamerasrc.h" +#include #include #include +#include #include using namespace libcamera; @@ -27,12 +29,71 @@ using namespace libcamera; GST_DEBUG_CATEGORY_STATIC(source_debug); #define GST_CAT_DEFAULT source_debug +struct RequestWrap { + RequestWrap(Request *request); + ~RequestWrap(); + + void attachBuffer(GstBuffer *buffer); + GstBuffer *detachBuffer(Stream *stream); + + /* For ptr comparision only. */ + Request *request_; + std::map buffers_; +}; + +RequestWrap::RequestWrap(Request *request) + : request_(request) +{ +} + +RequestWrap::~RequestWrap() +{ + for (std::pair &item : buffers_) { + if (item.second) + gst_buffer_unref(item.second); + } +} + +void RequestWrap::attachBuffer(GstBuffer *buffer) +{ + FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); + Stream *stream = gst_libcamera_buffer_get_stream(buffer); + + request_->addBuffer(stream, fb); + + auto item = buffers_.find(stream); + if (item != buffers_.end()) { + gst_buffer_unref(item->second); + item->second = buffer; + } else { + buffers_[stream] = buffer; + } +} + +GstBuffer *RequestWrap::detachBuffer(Stream *stream) +{ + GstBuffer *buffer = nullptr; + + auto item = buffers_.find(stream); + if (item != buffers_.end()) { + buffer = item->second; + item->second = nullptr; + } + + return buffer; +} + /* Used for C++ object with destructors. */ struct GstLibcameraSrcState { + GstLibcameraSrc *src; + std::unique_ptr cm; std::shared_ptr cam; std::unique_ptr config; std::vector srcpads; + std::queue> requests; + + void requestCompleted(Request *request); }; struct _GstLibcameraSrc { @@ -45,6 +106,7 @@ struct _GstLibcameraSrc { GstLibcameraSrcState *state; GstLibcameraAllocator *allocator; + GstFlowCombiner *flow_combiner; }; enum { @@ -68,6 +130,41 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +void +GstLibcameraSrcState::requestCompleted(Request *request) +{ + GLibLocker lock(GST_OBJECT(this->src)); + + GST_DEBUG_OBJECT(this->src, "buffers are ready"); + + std::unique_ptr wrap = std::move(this->requests.front()); + this->requests.pop(); + + g_return_if_fail(wrap->request_ == request); + + if ((request->status() == Request::RequestCancelled)) { + GST_DEBUG_OBJECT(this->src, "Request was cancelled"); + return; + } + + GstBuffer *buffer; + for (GstPad *srcpad : this->srcpads) { + Stream *stream = gst_libcamera_pad_get_stream(srcpad); + buffer = wrap->detachBuffer(stream); + gst_libcamera_pad_queue_buffer(srcpad, buffer); + } + + { + /* We only want to resume the task if it's paused. */ + GstTask *task = this->src->task; + GLibLocker lock(GST_OBJECT(task)); + if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { + GST_TASK_STATE(task) = GST_TASK_STARTED; + GST_TASK_SIGNAL(task); + } + } +} + static bool gst_libcamera_src_open(GstLibcameraSrc *self) { @@ -120,6 +217,8 @@ gst_libcamera_src_open(GstLibcameraSrc *self) return false; } + cam->requestCompleted.connect(self->state, &GstLibcameraSrcState::requestCompleted); + /* No need to lock here, we didn't start our threads yet. */ self->state->cm = std::move(cm); self->state->cam = cam; @@ -131,17 +230,85 @@ static void gst_libcamera_src_task_run(gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; + + Request *request = new Request(state->cam.get()); + auto wrap = std::make_unique(request); + for (GstPad *srcpad : state->srcpads) { + GstLibcameraPool *pool = gst_libcamera_pad_get_pool(srcpad); + GstBuffer *buffer; + GstFlowReturn ret; + + ret = gst_buffer_pool_acquire_buffer(GST_BUFFER_POOL(pool), + &buffer, nullptr); + if (ret != GST_FLOW_OK) { + /* RequestWrap does not take ownership, and we won't be + * queueing this one due to lack of buffers. */ + delete request; + request = NULL; + break; + } + + wrap->attachBuffer(buffer); + } + + if (request) { + GLibLocker lock(GST_OBJECT(self)); + GST_TRACE_OBJECT(self, "Requesting buffers"); + state->cam->queueRequest(request); + state->requests.push(std::move(wrap)); + } + + GstFlowReturn ret = GST_FLOW_OK; + gst_flow_combiner_reset(self->flow_combiner); + for (GstPad *srcpad : state->srcpads) { + ret = gst_libcamera_pad_push_pending(srcpad); + ret = gst_flow_combiner_update_pad_flow(self->flow_combiner, + srcpad, ret); + } - GST_DEBUG_OBJECT(self, "Streaming thread is now capturing"); + { + /* + * Here we need to decide if we want to pause or stop the task. This + * needs to happen in lock step with the callback thread which may want + * to resume the task. + */ + GLibLocker lock(GST_OBJECT(self)); + if (ret != GST_FLOW_OK) { + if (ret == GST_FLOW_EOS) { + g_autoptr(GstEvent) eos = gst_event_new_eos(); + guint32 seqnum = gst_util_seqnum_next(); + gst_event_set_seqnum(eos, seqnum); + for (GstPad *srcpad : state->srcpads) + gst_pad_push_event(srcpad, gst_event_ref(eos)); + } else if (ret != GST_FLOW_FLUSHING) { + GST_ELEMENT_FLOW_ERROR(self, ret); + } + gst_task_stop(self->task); + return; + } + + gboolean do_pause = true; + for (GstPad *srcpad : state->srcpads) { + if (gst_libcamera_pad_has_pending(srcpad)) { + do_pause = false; + break; + } + } + + if (do_pause) + gst_task_pause(self->task); + } } static void gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); - GLibRecLocker(&self->stream_lock); + GLibRecLocker lock(&self->stream_lock); GstLibcameraSrcState *state = self->state; GstFlowReturn flow_ret = GST_FLOW_OK; + gint ret = 0; GST_DEBUG_OBJECT(self, "Streaming thread has started"); @@ -230,12 +397,23 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) return; } + self->flow_combiner = gst_flow_combiner_new(); for (gsize i = 0; i < state->srcpads.size(); i++) { GstPad *srcpad = state->srcpads[i]; const StreamConfiguration &stream_cfg = state->config->at(i); GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, stream_cfg.stream()); gst_libcamera_pad_set_pool(srcpad, pool); + gst_flow_combiner_add_pad(self->flow_combiner, srcpad); + } + + ret = state->cam->start(); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Failed to start the camera: %s", g_strerror(-ret)), + ("Camera.start() failed with error code %i", ret)); + gst_task_stop(task); + return; } done: @@ -257,10 +435,14 @@ gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); + state->cam->stop(); + for (GstPad *srcpad : state->srcpads) gst_libcamera_pad_set_pool(srcpad, NULL); g_clear_object(&self->allocator); + g_clear_pointer(&self->flow_combiner, + (GDestroyNotify)gst_flow_combiner_free); } static void @@ -340,6 +522,9 @@ gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) return GST_STATE_CHANGE_FAILURE; ret = GST_STATE_CHANGE_NO_PREROLL; break; + case GST_STATE_CHANGE_PAUSED_TO_PLAYING: + gst_task_start(self->task); + break; case GST_STATE_CHANGE_PLAYING_TO_PAUSED: ret = GST_STATE_CHANGE_NO_PREROLL; break; @@ -389,6 +574,9 @@ gst_libcamera_src_init(GstLibcameraSrc *self) state->srcpads.push_back(gst_pad_new_from_template(templ, "src")); gst_element_add_pad(GST_ELEMENT(self), state->srcpads[0]); + + /* C-style friend. */ + state->src = self; self->state = state; } From patchwork Thu Feb 27 20:04:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2905 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E621E6273D for ; Thu, 27 Feb 2020 21:04:28 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 71D1F2949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:04:03 -0500 Message-Id: <20200227200407.490616-24-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 23/27] gst: libcamerasrc: Implement timestamp support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:31 -0000 This is an experimental patch adding timestamp support to the libcamerasrc element. This patch currently assume that the driver timestamp are relative to the system monotonic clock. Without a reference clock source, the timestamp are otherwise unusable, and without timestamp only minor use case can be achieved. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 23 +++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ src/gstreamer/gstlibcamerasrc.cpp | 20 ++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 2cf1630..31d3edd 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -62,9 +62,24 @@ gst_libcamera_pad_get_property(GObject *object, guint prop_id, GValue *value, } } +static gboolean +gst_libcamera_pad_query(GstPad *pad, GstObject *parent, GstQuery *query) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (query->type != GST_QUERY_LATENCY) + return gst_pad_query_default(pad, parent, query); + + /* TRUE here means live, we assumes that max latency is the same as min + * as we have no idea that duration of frames. */ + gst_query_set_latency(query, TRUE, self->latency, self->latency); + return TRUE; +} + static void gst_libcamera_pad_init(GstLibcameraPad *self) { + GST_PAD_QUERYFUNC(self) = gst_libcamera_pad_query; } static GType @@ -172,3 +187,11 @@ gst_libcamera_pad_has_pending(GstPad *pad) GLibLocker lock(GST_OBJECT(self)); return (self->pending_buffers.length > 0); } + +void +gst_libcamera_pad_set_latency(GstPad *pad, GstClockTime latency) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + self->latency = latency; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index eb24000..a3291e8 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -32,4 +32,6 @@ GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); bool gst_libcamera_pad_has_pending(GstPad *pad); +void gst_libcamera_pad_set_latency(GstPad *pad, GstClockTime latency); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 70f2048..73a1360 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -151,6 +151,26 @@ GstLibcameraSrcState::requestCompleted(Request *request) for (GstPad *srcpad : this->srcpads) { Stream *stream = gst_libcamera_pad_get_stream(srcpad); buffer = wrap->detachBuffer(stream); + + FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); + + if (GST_ELEMENT_CLOCK(this->src)) { + GstClockTime gst_base_time = GST_ELEMENT(this->src)->base_time; + GstClockTime gst_now = gst_clock_get_time(GST_ELEMENT_CLOCK(this->src)); + /* \todo Need to expose which reference clock the timestamp relates to. */ + GstClockTime sys_now = g_get_monotonic_time() * 1000; + + /* Deduced from: sys_now - sys_base_time == gst_now - gst_bae_time */ + GstClockTime sys_base_time = sys_now - (gst_now - gst_base_time); + GST_BUFFER_PTS(buffer) = fb->metadata().timestamp - sys_base_time; + gst_libcamera_pad_set_latency(srcpad, sys_now - fb->metadata().timestamp); + } else { + GST_BUFFER_PTS(buffer) = 0; + } + + GST_BUFFER_OFFSET(buffer) = fb->metadata().sequence; + GST_BUFFER_OFFSET_END(buffer) = fb->metadata().sequence; + gst_libcamera_pad_queue_buffer(srcpad, buffer); } From patchwork Thu Feb 27 20:04:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2906 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 605636273F for ; Thu, 27 Feb 2020 21:04:29 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id DD9DE29654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:04:04 -0500 Message-Id: <20200227200407.490616-25-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 24/27] gst: libcamerasrc: Add a TODO comment X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:31 -0000 This is to guide upcoming contributors toward what is left to do to get toward a production ready element. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 73a1360..1e55916 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -7,6 +7,25 @@ */ /** + * \todo The following is a list of items that needs implementation in the GStreamer plugin + * - Implement GstElement::send_event + * + Allowing application to send EOS + * + Allowing application to use FLUSH/FLUSH_STOP + * + Prevent the main thread from accessing streaming thread + * - Implement renegotiation (even if slow) + * - Implement GstElement::request-new-pad (multi stream) + * + Evaluate if a single streaming thread is fine + * - Add application driven request (snapshot) + * - Add framerate control + * - Add buffer importation support + * + * Requires new libcamera API: + * - Add framerate negotiation support + * - Add colorimetry support + * - Add timestamp support + * - Use unique names to select the camera devices + * - Add GstVideoMeta support (strides and offsets) + * * \todo libcamera UVC drivers picks the lowest possible resolution first, this * should be fixed so that we get a decent resolution and framerate for the * role by default. From patchwork Thu Feb 27 20:04:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2907 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EA5F962746 for ; Thu, 27 Feb 2020 21:04:29 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 59CB229654B From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Jakub Adam Date: Thu, 27 Feb 2020 15:04:05 -0500 Message-Id: <20200227200407.490616-26-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 25/27] gst: utils: Factor-out the task resume helper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:31 -0000 From: Jakub Adam Task resume will be added in the core GStreamer API in the future and we will need to call this in another location in the following patches. Signed-off-by: Jakub Adam Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.cpp | 11 +++++++++++ src/gstreamer/gstlibcamera-utils.h | 1 + src/gstreamer/gstlibcamerasrc.cpp | 10 +--------- 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 231f3c5..11c2d56 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -160,3 +160,14 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, stream_cfg.size.width = width; stream_cfg.size.height = height; } + +void +gst_libcamera_resume_task(GstTask *task) +{ + /* We only want to resume the task if it's paused. */ + GLibLocker lock(GST_OBJECT(task)); + if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { + GST_TASK_STATE(task) = GST_TASK_STARTED; + GST_TASK_SIGNAL(task); + } +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 51898a0..440f196 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -17,6 +17,7 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &fo GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, GstCaps *caps); +void gst_libcamera_resume_task(GstTask *task); /** * \class GLibLocker diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 1e55916..fe60584 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -193,15 +193,7 @@ GstLibcameraSrcState::requestCompleted(Request *request) gst_libcamera_pad_queue_buffer(srcpad, buffer); } - { - /* We only want to resume the task if it's paused. */ - GstTask *task = this->src->task; - GLibLocker lock(GST_OBJECT(task)); - if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { - GST_TASK_STATE(task) = GST_TASK_STARTED; - GST_TASK_SIGNAL(task); - } - } + gst_libcamera_resume_task(this->src->task); } static bool From patchwork Thu Feb 27 20:04:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2908 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6F4EE62755 for ; Thu, 27 Feb 2020 21:04:30 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id D9C662949DF From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Jakub Adam Date: Thu, 27 Feb 2020 15:04:06 -0500 Message-Id: <20200227200407.490616-27-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 26/27] gst: libcamerasrc: Prevent src task deadlock on exhausted buffer pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:31 -0000 From: Jakub Adam Allow GstLibcameraPool to notify the source when a new buffer has become available in a previously exhausted buffer pool. This can be used to resume a src task that got paused because it couldn't acquire a buffer. Without this change the src task will never resume from pause once the pool gets exhausted. To trigger the deadlock (it doesn't happen every time), run: gst-launch-1.0 libcamerasrc ! queue ! glimagesink Signed-off-by: Jakub Adam --- src/gstreamer/gstlibcamerapool.cpp | 16 ++++++++++++++++ src/gstreamer/gstlibcamerapool.h | 6 ++++++ src/gstreamer/gstlibcamerasrc.cpp | 14 ++++++++++++-- 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp index f85c3aa..5aa0eeb 100644 --- a/src/gstreamer/gstlibcamerapool.cpp +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -18,6 +18,8 @@ struct _GstLibcameraPool { GstAtomicQueue *queue; GstLibcameraAllocator *allocator; + GstLibcameraBufferNotify buffer_notify; + gpointer buffer_notify_data; Stream *stream; }; @@ -54,7 +56,12 @@ static void gst_libcamera_pool_release_buffer(GstBufferPool *pool, GstBuffer *buffer) { GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + bool do_notify = gst_atomic_queue_length(self->queue) == 0; + gst_atomic_queue_push(self->queue, buffer); + + if (do_notify && self->buffer_notify) + self->buffer_notify(self->buffer_notify_data); } static void @@ -108,6 +115,15 @@ gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) return pool; } +void +gst_libcamera_pool_set_buffer_notify(GstLibcameraPool *self, + GstLibcameraBufferNotify notify, + gpointer data) +{ + self->buffer_notify = notify; + self->buffer_notify_data = data; +} + Stream * gst_libcamera_pool_get_stream(GstLibcameraPool *self) { diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h index 6fd2913..545be01 100644 --- a/src/gstreamer/gstlibcamerapool.h +++ b/src/gstreamer/gstlibcamerapool.h @@ -20,9 +20,15 @@ #define GST_TYPE_LIBCAMERA_POOL gst_libcamera_pool_get_type() G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstBufferPool) +typedef void (*GstLibcameraBufferNotify)(gpointer data); + GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +void gst_libcamera_pool_set_buffer_notify(GstLibcameraPool *self, + GstLibcameraBufferNotify notify, + gpointer data); + libcamera::Stream *gst_libcamera_pool_get_stream(GstLibcameraPool *self); libcamera::Stream *gst_libcamera_buffer_get_stream(GstBuffer *buffer); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index fe60584..ecbfe37 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -434,6 +434,10 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) const StreamConfiguration &stream_cfg = state->config->at(i); GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, stream_cfg.stream()); + gst_libcamera_pool_set_buffer_notify(pool, + (GstLibcameraBufferNotify)gst_libcamera_resume_task, + task); + gst_libcamera_pad_set_pool(srcpad, pool); gst_flow_combiner_add_pad(self->flow_combiner, srcpad); } @@ -468,8 +472,14 @@ gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) state->cam->stop(); - for (GstPad *srcpad : state->srcpads) - gst_libcamera_pad_set_pool(srcpad, NULL); + for (GstPad *srcpad : state->srcpads) { + auto *pool = gst_libcamera_pad_get_pool(srcpad); + + if (pool) { + gst_libcamera_pool_set_buffer_notify(pool, NULL, NULL); + gst_libcamera_pad_set_pool(srcpad, NULL); + } + } g_clear_object(&self->allocator); g_clear_pointer(&self->flow_combiner, From patchwork Thu Feb 27 20:04:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2909 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CFE8F62737 for ; Thu, 27 Feb 2020 21:04:30 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 677A529654A From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Date: Thu, 27 Feb 2020 15:04:07 -0500 Message-Id: <20200227200407.490616-28-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200227200407.490616-1-nicolas.dufresne@collabora.com> References: <20200227200407.490616-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 27/27] gst: Reduce GStreamer requirement to 1.14 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2020 20:04:32 -0000 This has been tested to build and run on 1.14, except for the device monitor. Mostly, all device monitors that are non-monitoring have been broken for a long time. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 90773af..269b8b6 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -13,7 +13,7 @@ libcamera_gst_c_args = [ '-DPACKAGE="@0@"'.format(meson.project_name()), ] -gst_req = '>=1.16.1' +gst_req = '>=1.14.0' gstvideo_dep = dependency('gstreamer-video-1.0', version : gst_req, required : get_option('gstreamer')) gstallocator_dep = dependency('gstreamer-allocators-1.0', version : gst_req,