From patchwork Wed Jan 29 03:31:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2735 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 919806044C for ; Wed, 29 Jan 2020 04:35:12 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 93DE1292988; Wed, 29 Jan 2020 03:35:11 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:48 -0500 Message-Id: <20200129033210.278800-2-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 01/23] Add GStreamer plugin and element skeleton X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:12 -0000 From: Nicolas Dufresne Signed-off-by: Nicolas Dufresne --- meson_options.txt | 5 +++++ src/gstreamer/gstlibcamera.c | 21 +++++++++++++++++++++ src/gstreamer/gstlibcamerasrc.cpp | 31 +++++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerasrc.h | 22 ++++++++++++++++++++++ src/gstreamer/meson.build | 23 +++++++++++++++++++++++ src/meson.build | 2 ++ 6 files changed, 104 insertions(+) create mode 100644 src/gstreamer/gstlibcamera.c create mode 100644 src/gstreamer/gstlibcamerasrc.cpp create mode 100644 src/gstreamer/gstlibcamerasrc.h create mode 100644 src/gstreamer/meson.build diff --git a/meson_options.txt b/meson_options.txt index 79ee4de..1b78ed8 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -15,3 +15,8 @@ option('v4l2', type : 'boolean', value : false, description : 'Compile the V4L2 compatibility layer') + +option('gstreamer', + type : 'feature', + value : 'auto', + description : 'Compile libcamera GStreamer plugin') diff --git a/src/gstreamer/gstlibcamera.c b/src/gstreamer/gstlibcamera.c new file mode 100644 index 0000000..953fb29 --- /dev/null +++ b/src/gstreamer/gstlibcamera.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera.c - GStreamer plugin + */ + +#include "gstlibcamerasrc.h" + +static gboolean +plugin_init(GstPlugin *plugin) +{ + return gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_SRC); + return TRUE; +} + +GST_PLUGIN_DEFINE(GST_VERSION_MAJOR, GST_VERSION_MINOR, + libcamera, "LibCamera capture plugin", + plugin_init, VERSION, "LGPL", PACKAGE, "https://libcamera.org"); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp new file mode 100644 index 0000000..39a06a4 --- /dev/null +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerasrc.cpp - GStreamer Capture Element + */ + +#include "gstlibcamerasrc.h" + +struct _GstLibcameraSrc { + GstElement parent; +}; + +G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); + +static void +gst_libcamera_src_init(GstLibcameraSrc *self) +{ +} + +static void +gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) +{ + GstElementClass *element_class = (GstElementClass *)klass; + + gst_element_class_set_metadata(element_class, + "LibCamera Source", "Source/Video", + "Linux Camera source using libcamera", + "Nicolas Dufresne + * + * gstlibcamerasrc.h - GStreamer Capture Element + */ + +#include + +#ifndef __GST_LIBCAMERA_SRC_H__ +#define __GST_LIBCAMERA_SRC_H__ + +G_BEGIN_DECLS + +#define GST_TYPE_LIBCAMERA_SRC gst_libcamera_src_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraSrc, gst_libcamera_src, + GST_LIBCAMERA, SRC, GstElement) + +G_END_DECLS + +#endif /* __GST_LIBCAMERA_SRC_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build new file mode 100644 index 0000000..32d4233 --- /dev/null +++ b/src/gstreamer/meson.build @@ -0,0 +1,23 @@ +libcamera_gst_sources = [ + 'gstlibcamera.c', + 'gstlibcamerasrc.cpp', +] + +libcamera_gst_c_args = [ + '-DVERSION="@0@"'.format(libcamera_git_version), + '-DPACKAGE="@0@"'.format(meson.project_name()), +] + +gst_dep = dependency('gstreamer-video-1.0', version : '>=1.16.1', + required : get_option('gstreamer')) + +if gst_dep.found() + libcamera_gst = shared_library('gstlibcamera', + libcamera_gst_sources, + c_args : libcamera_gst_c_args, + include_directories : [], + dependencies : [libcamera_dep, gst_dep], + install: true, + install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')), + ) +endif diff --git a/src/meson.build b/src/meson.build index 5adcd61..d818d8b 100644 --- a/src/meson.build +++ b/src/meson.build @@ -10,3 +10,5 @@ subdir('qcam') if get_option('v4l2') subdir('v4l2') endif + +subdir('gstreamer') From patchwork Wed Jan 29 03:31:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2736 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 81EC960447 for ; Wed, 29 Jan 2020 04:35:16 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id A8E6528EA9F; Wed, 29 Jan 2020 03:35:12 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:49 -0500 Message-Id: <20200129033210.278800-3-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 02/23] gst: Add utility to convert StreamFormats to GstCaps X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:16 -0000 From: Nicolas Dufresne This transform the basic information found in StreamFormats to GstCaps. This can be handy to reply to early caps query or inside a device provider. Note that we ignored generated range as they are harmful to caps negotiation. We also don't simplify the caps for readability reasons, so some of the discrete value may be included in a range. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamera-utils.cpp | 102 +++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 18 +++++ src/gstreamer/meson.build | 1 + 3 files changed, 121 insertions(+) create mode 100644 src/gstreamer/gstlibcamera-utils.cpp create mode 100644 src/gstreamer/gstlibcamera-utils.h diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp new file mode 100644 index 0000000..2540be0 --- /dev/null +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera-utils.c - GStreamer Libcamera Utility Function + */ + +#include "gstlibcamera-utils.h" +#include + +using namespace libcamera; + +static struct { + GstVideoFormat gst_format; + guint drm_fourcc; +} format_map[] = { + { GST_VIDEO_FORMAT_ENCODED, DRM_FORMAT_MJPEG }, + { GST_VIDEO_FORMAT_RGB, DRM_FORMAT_BGR888 }, + { GST_VIDEO_FORMAT_BGR, DRM_FORMAT_RGB888 }, + { GST_VIDEO_FORMAT_ARGB, DRM_FORMAT_BGRA8888 }, + { GST_VIDEO_FORMAT_NV12, DRM_FORMAT_NV12 }, + { GST_VIDEO_FORMAT_NV21, DRM_FORMAT_NV21 }, + { GST_VIDEO_FORMAT_NV16, DRM_FORMAT_NV16 }, + { GST_VIDEO_FORMAT_NV61, DRM_FORMAT_NV61 }, + { GST_VIDEO_FORMAT_NV24, DRM_FORMAT_NV24 }, + /* { GST_VIDEO_FORMAT_NV42, DRM_FORMAT_NV42 }, */ + { GST_VIDEO_FORMAT_UYVY, DRM_FORMAT_UYVY }, + { GST_VIDEO_FORMAT_VYUY, DRM_FORMAT_VYUY }, + { GST_VIDEO_FORMAT_YUY2, DRM_FORMAT_YUYV }, + { GST_VIDEO_FORMAT_YVYU, DRM_FORMAT_YVYU }, +}; +#define FORMAT_MAP_SIZE G_N_ELEMENTS(format_map) + +static inline GstVideoFormat +drm_to_gst_format(guint drm_fourcc) +{ + for (guint i = 0; i < FORMAT_MAP_SIZE; i++) + if (format_map[i].drm_fourcc == drm_fourcc) + return format_map[i].gst_format; + return GST_VIDEO_FORMAT_UNKNOWN; +} + +static GstStructure * +bare_structure_from_fourcc(guint fourcc) +{ + GstVideoFormat gst_format = drm_to_gst_format(fourcc); + + if (gst_format == GST_VIDEO_FORMAT_UNKNOWN) + return NULL; + + if (gst_format != GST_VIDEO_FORMAT_ENCODED) + return gst_structure_new("video/x-raw", "format", G_TYPE_STRING, + gst_video_format_to_string(gst_format), NULL); + + switch (fourcc) { + case DRM_FORMAT_MJPEG: + return gst_structure_new_empty("image/jpeg"); + default: + break; + } + + return NULL; +} + +GstCaps * +gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) +{ + GstCaps *caps = gst_caps_new_empty(); + + for (unsigned int fourcc : formats.pixelformats()) { + g_autoptr(GstStructure) bare_s = bare_structure_from_fourcc(fourcc); + + if (!bare_s) { + GST_WARNING("Unsupported DRM format %" GST_FOURCC_FORMAT, + GST_FOURCC_ARGS(fourcc)); + continue; + } + + for (const Size &size : formats.sizes(fourcc)) { + GstStructure *s = gst_structure_copy(bare_s); + gst_structure_set(s, + "width", G_TYPE_INT, size.width, + "height", G_TYPE_INT, size.height, + NULL); + gst_caps_append_structure(caps, s); + } + + const SizeRange &range = formats.range(fourcc); + if (range.hStep && range.vStep) { + GstStructure *s = gst_structure_copy(bare_s); + + gst_structure_set(s, + "width", GST_TYPE_INT_RANGE, range.min.width, range.max.width, range.hStep, + "height", GST_TYPE_INT_RANGE, range.min.height, range.max.height, range.vStep, + NULL); + gst_caps_append_structure(caps, s); + } + } + + return caps; +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h new file mode 100644 index 0000000..4545512 --- /dev/null +++ b/src/gstreamer/gstlibcamera-utils.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera-utils.h - GStreamer Libcamera Utility Functions + */ + +#include +#include + +#include + +#ifndef __GST_LIBCAMERA_UTILS_H_ + +GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); + +#endif /* __GST_LIBCAMERA_UTILS_H_ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 32d4233..39a34e7 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,5 +1,6 @@ libcamera_gst_sources = [ 'gstlibcamera.c', + 'gstlibcamera-utils.cpp', 'gstlibcamerasrc.cpp', ] From patchwork Wed Jan 29 03:31:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2737 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5FC7960447 for ; Wed, 29 Jan 2020 04:35:19 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id B11C028EA9F; Wed, 29 Jan 2020 03:35:16 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:50 -0500 Message-Id: <20200129033210.278800-4-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 03/23] gst: Add initial device provider X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:19 -0000 From: Nicolas Dufresne This feature is used with GstDeviceMonitor in order to enumerate and monitor devices to be used with the source element. The resulting GstDevice implementation is also used by application to abstract the configuration of the source element. Implementations notes: - libcamera does not support polling yet - The device ID isn't unique in libcamera - The "name" property does not yet exist in libcamerasrc Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamera.c | 10 +- src/gstreamer/gstlibcameraprovider.cpp | 227 +++++++++++++++++++++++++ src/gstreamer/gstlibcameraprovider.h | 23 +++ src/gstreamer/meson.build | 1 + 4 files changed, 259 insertions(+), 2 deletions(-) create mode 100644 src/gstreamer/gstlibcameraprovider.cpp create mode 100644 src/gstreamer/gstlibcameraprovider.h diff --git a/src/gstreamer/gstlibcamera.c b/src/gstreamer/gstlibcamera.c index 953fb29..7356374 100644 --- a/src/gstreamer/gstlibcamera.c +++ b/src/gstreamer/gstlibcamera.c @@ -7,12 +7,18 @@ */ #include "gstlibcamerasrc.h" +#include "gstlibcameraprovider.h" static gboolean plugin_init(GstPlugin *plugin) { - return gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, - GST_TYPE_LIBCAMERA_SRC); + if (!gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_SRC) || + !gst_device_provider_register(plugin, "libcameraprovider", + GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_PROVIDER)) + return FALSE; + return TRUE; } diff --git a/src/gstreamer/gstlibcameraprovider.cpp b/src/gstreamer/gstlibcameraprovider.cpp new file mode 100644 index 0000000..3e27912 --- /dev/null +++ b/src/gstreamer/gstlibcameraprovider.cpp @@ -0,0 +1,227 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraprovider.c - GStreamer Device Provider + */ + +#include "gstlibcamera-utils.h" +#include "gstlibcameraprovider.h" +#include "gstlibcamerasrc.h" + +#include +#include + +using namespace libcamera; + +GST_DEBUG_CATEGORY_STATIC(provider_debug); +#define GST_CAT_DEFAULT provider_debug + +/**************************************/ +/* GstLibcameraDevice internal Object */ +/**************************************/ + +enum { + PROP_DEVICE_NAME = 1, +}; + +#define GST_TYPE_LIBCAMERA_DEVICE gst_libcamera_device_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraDevice, gst_libcamera_device, + GST_LIBCAMERA, DEVICE, GstDevice); + +struct _GstLibcameraDevice { + GstDevice parent; + gchar *name; +}; + +G_DEFINE_TYPE(GstLibcameraDevice, gst_libcamera_device, GST_TYPE_DEVICE); + +static GstElement * +gst_libcamera_device_create_element(GstDevice *device, const gchar *name) +{ + GstElement *source; + + source = gst_element_factory_make("libcamerasrc", name); + g_assert(source); + + g_object_set(source, "camera-name", GST_LIBCAMERA_DEVICE(device)->name, NULL); + + return source; +} + +static gboolean +gst_libcamera_device_reconfigure_element(GstDevice *device, + GstElement *element) +{ + if (!GST_LIBCAMERA_IS_SRC(element)) + return FALSE; + + g_object_set(element, "camera-name", GST_LIBCAMERA_DEVICE(device)->name, NULL); + + return TRUE; +} + +static void +gst_libcamera_device_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + GstLibcameraDevice *device; + + device = GST_LIBCAMERA_DEVICE(object); + + switch (prop_id) { + case PROP_DEVICE_NAME: + device->name = g_value_dup_string(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_device_init(GstLibcameraDevice *self) +{ +} + +static void +gst_libcamera_device_finalize(GObject *object) +{ + GstLibcameraDevice *self = GST_LIBCAMERA_DEVICE(object); + gpointer klass = gst_libcamera_device_parent_class; + + g_free(self->name); + + G_OBJECT_GET_CLASS(klass)->finalize(object); +} + +static void +gst_libcamera_device_class_init(GstLibcameraDeviceClass *klass) +{ + GParamSpec *pspec; + GstDeviceClass *device_class = (GstDeviceClass *)klass; + GObjectClass *object_class = (GObjectClass *)klass; + + device_class->create_element = gst_libcamera_device_create_element; + device_class->reconfigure_element = gst_libcamera_device_reconfigure_element; + + object_class->set_property = gst_libcamera_device_set_property; + object_class->finalize = gst_libcamera_device_finalize; + + pspec = g_param_spec_string("name", "Name", + "The name of the camera device", "", + (GParamFlags)(G_PARAM_STATIC_STRINGS | G_PARAM_WRITABLE | + G_PARAM_CONSTRUCT_ONLY)); + g_object_class_install_property(object_class, PROP_DEVICE_NAME, pspec); +} + +static GstDevice * +gst_libcamera_device_new(const std::shared_ptr &camera) +{ + g_autoptr(GstCaps) caps = gst_caps_new_empty(); + g_autoptr(GstStructure) props = NULL; + const gchar *name = camera->name().c_str(); + StreamRoles roles; + + roles.push_back(StreamRole::VideoRecording); + std::unique_ptr config = camera->generateConfiguration(roles); + + for (const StreamConfiguration &stream_cfg : *config) { + GstCaps *sub_caps = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); + if (sub_caps) + gst_caps_append(caps, sub_caps); + } + + return (GstDevice *)g_object_new(GST_TYPE_LIBCAMERA_DEVICE, + /* FIXME not sure name is unique */ + "name", name, + "display-name", name, + "caps", caps, + "device-class", "Source/Video", + "properties", props, + NULL); +} + +/*************************************/ +/* GstLibcameraDeviceProvider Object */ +/*************************************/ + +struct _GstLibcameraProvider { + GstDeviceProvider parent; + CameraManager *cm; +}; + +G_DEFINE_TYPE_WITH_CODE(GstLibcameraProvider, gst_libcamera_provider, + GST_TYPE_DEVICE_PROVIDER, + GST_DEBUG_CATEGORY_INIT(provider_debug, "libcamera-provider", 0, + "LibCamera Device Provider")); + +static GList * +gst_libcamera_provider_probe(GstDeviceProvider *provider) +{ + GstLibcameraProvider *self = GST_LIBCAMERA_PROVIDER(provider); + CameraManager *cm = self->cm; + GList *devices = NULL; + gint ret; + + GST_INFO_OBJECT(self, "Probing cameras using LibCamera"); + + /* FIXME as long as the manager isn't able to handle hot-plug, we need to + * cycle start/stop here in order to ensure wthat we have an up to date + * list */ + ret = cm->start(); + if (ret) { + GST_ERROR_OBJECT(self, "Failed to retrieve device list: %s", + g_strerror(-ret)); + return NULL; + } + + for (const std::shared_ptr &camera : cm->cameras()) { + GST_INFO_OBJECT(self, "Found camera '%s'", camera->name().c_str()); + devices = g_list_append(devices, + g_object_ref_sink(gst_libcamera_device_new(camera))); + } + + cm->stop(); + + return devices; +} + +static void +gst_libcamera_provider_init(GstLibcameraProvider *self) +{ + GstDeviceProvider *provider = GST_DEVICE_PROVIDER(self); + + self->cm = new CameraManager(); + + /* Avoid devices being duplicated */ + gst_device_provider_hide_provider(provider, "v4l2deviceprovider"); +} + +static void +gst_libcamera_provider_finalize(GObject *object) +{ + GstLibcameraProvider *self = GST_LIBCAMERA_PROVIDER(object); + gpointer klass = gst_libcamera_provider_parent_class; + + delete self->cm; + + return G_OBJECT_GET_CLASS(klass)->finalize(object); +} + +static void +gst_libcamera_provider_class_init(GstLibcameraProviderClass *klass) +{ + GstDeviceProviderClass *provider_class = (GstDeviceProviderClass *)klass; + GObjectClass *object_class = (GObjectClass *)klass; + + provider_class->probe = gst_libcamera_provider_probe; + object_class->finalize = gst_libcamera_provider_finalize; + + gst_device_provider_class_set_metadata(provider_class, + "LibCamera Device Provider", + "Source/Video", + "List camera device using LibCamera", + "Nicolas Dufresne "); +} diff --git a/src/gstreamer/gstlibcameraprovider.h b/src/gstreamer/gstlibcameraprovider.h new file mode 100644 index 0000000..6dd232d --- /dev/null +++ b/src/gstreamer/gstlibcameraprovider.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraprovider.h - GStreamer Device Provider + */ + +#include + +#ifndef __GST_LIBCAMERA_PROVIDER_H__ +#define __GST_LIBCAMERA_PROVIDER_H__ + +G_BEGIN_DECLS + +#define GST_TYPE_LIBCAMERA_PROVIDER gst_libcamera_provider_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraProvider, gst_libcamera_provider, + GST_LIBCAMERA, PROVIDER, GstDeviceProvider) + +G_END_DECLS + +#endif /* __GST_LIBCAMERA_PROVIDER_H__ */ + diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 39a34e7..7769b78 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -2,6 +2,7 @@ libcamera_gst_sources = [ 'gstlibcamera.c', 'gstlibcamera-utils.cpp', 'gstlibcamerasrc.cpp', + 'gstlibcameraprovider.cpp', ] libcamera_gst_c_args = [ From patchwork Wed Jan 29 03:31:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2738 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 401D5608BF for ; Wed, 29 Jan 2020 04:35:20 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 79D9E28EA9F; Wed, 29 Jan 2020 03:35:19 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:51 -0500 Message-Id: <20200129033210.278800-5-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 04/23] gst: utils: Add a macro to use a GMutexLocker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:20 -0000 From: Nicolas Dufresne The GMutexLocker uses GCC/CLANG C feature to implement a smart lock, that unlocks the mutex when the GMutexLocker goes out of scope. This could have been implemented in C++, but as this is already implemented I decided to just reuse it. This is particularly handy to avoid gotos in error handling cases and to prevent unbalanced locking. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamera-utils.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 4545512..528dc2c 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -13,6 +13,8 @@ #ifndef __GST_LIBCAMERA_UTILS_H_ +#define GST_OBJECT_LOCKER(obj) g_autoptr(GMutexLocker) locker = g_mutex_locker_new(GST_OBJECT_GET_LOCK(obj)) + GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); #endif /* __GST_LIBCAMERA_UTILS_H_ */ From patchwork Wed Jan 29 03:31:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2739 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 23B70608C8 for ; Wed, 29 Jan 2020 04:35:23 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 4E7BC28EA9F; Wed, 29 Jan 2020 03:35:20 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:52 -0500 Message-Id: <20200129033210.278800-6-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 05/23] gst: Add pads to the source X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:23 -0000 From: Nicolas Dufresne This simply add the boiler plate for pads on the source element. The design is that we have one pad, called "src", that will always be present, and then more pads can be requested prior in READY or less state. Initially pads have one property "stream-role" that let you decide which role this pad will have. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerapad.cpp | 101 ++++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 19 ++++++ src/gstreamer/gstlibcamerasrc.cpp | 19 ++++++ src/gstreamer/meson.build | 1 + 4 files changed, 140 insertions(+) create mode 100644 src/gstreamer/gstlibcamerapad.cpp create mode 100644 src/gstreamer/gstlibcamerapad.h diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp new file mode 100644 index 0000000..c6c22e4 --- /dev/null +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapad.cpp - GStreamer Capture Pad + */ + +#include "gstlibcamerapad.h" +#include "gstlibcamera-utils.h" + +#include + +using namespace libcamera; + +struct _GstLibcameraPad { + GstPad parent; + StreamRole role; +}; + +enum { + PROP_0, + PROP_STREAM_ROLE +}; + +G_DEFINE_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_TYPE_PAD); + +static void +gst_libcamera_pad_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + auto *self = GST_LIBCAMERA_PAD(object); + GST_OBJECT_LOCKER(self); + + switch (prop_id) { + case PROP_STREAM_ROLE: + self->role = (StreamRole)g_value_get_enum(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_pad_get_property(GObject *object, guint prop_id, GValue *value, + GParamSpec *pspec) +{ + auto *self = GST_LIBCAMERA_PAD(object); + GST_OBJECT_LOCKER(self); + + switch (prop_id) { + case PROP_STREAM_ROLE: + g_value_set_enum(value, self->role); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_pad_init(GstLibcameraPad *self) +{ +} + +static GType +gst_libcamera_stream_role_get_type(void) +{ + static GType type = 0; + static const GEnumValue values[] = { + { StillCapture, "libcamera::StillCapture", "still-capture" }, + { VideoRecording, "libcamera::VideoRecording", "video-recording" }, + { Viewfinder, "libcamera::Viewfinder", "view-finder" }, + { 0, NULL, NULL } + }; + + if (!type) + type = g_enum_register_static("GstLibcameraStreamRole", values); + + return type; +} + +static void +gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) +{ + auto *object_class = G_OBJECT_CLASS(klass); + + object_class->set_property = gst_libcamera_pad_set_property; + object_class->get_property = gst_libcamera_pad_get_property; + + auto *spec = g_param_spec_enum("stream-role", "Stream Role", + "The selected stream role", + gst_libcamera_stream_role_get_type(), + VideoRecording, + (GParamFlags)(GST_PARAM_MUTABLE_READY + | G_PARAM_CONSTRUCT + | G_PARAM_READWRITE + | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_STREAM_ROLE, spec); +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h new file mode 100644 index 0000000..2e745f1 --- /dev/null +++ b/src/gstreamer/gstlibcamerapad.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapad.h - GStreamer Capture Element + */ + +#include + +#ifndef __GST_LIBCAMERA_PAD_H__ +#define __GST_LIBCAMERA_PAD_H__ + +#define GST_TYPE_LIBCAMERA_PAD gst_libcamera_pad_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, + GST_LIBCAMERA, PAD, GstPad) + + +#endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 39a06a4..a8bcede 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -7,6 +7,7 @@ */ #include "gstlibcamerasrc.h" +#include "gstlibcamerapad.h" struct _GstLibcameraSrc { GstElement parent; @@ -14,6 +15,18 @@ struct _GstLibcameraSrc { G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); +#define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw;image/jpeg") + +/* For the simple case, we have a src pad that is always present. */ +GstStaticPadTemplate src_template = { + "src", GST_PAD_SRC, GST_PAD_ALWAYS, TEMPLATE_CAPS +}; + +/* More pads can be requested in state < PAUSED */ +GstStaticPadTemplate request_src_template = { + "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS +}; + static void gst_libcamera_src_init(GstLibcameraSrc *self) { @@ -28,4 +41,10 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) "LibCamera Source", "Source/Video", "Linux Camera source using libcamera", "Nicolas Dufresne X-Patchwork-Id: 2740 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 02F7D608C8 for ; Wed, 29 Jan 2020 04:35:24 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 5202F2942F8; Wed, 29 Jan 2020 03:35:23 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:53 -0500 Message-Id: <20200129033210.278800-7-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 06/23] gst: libcamerasrc: Allocate and add static pad X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:24 -0000 From: Nicolas Dufresne Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index a8bcede..50bba39 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -11,6 +11,7 @@ struct _GstLibcameraSrc { GstElement parent; + GstPad *srcpad; }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); @@ -30,6 +31,10 @@ GstStaticPadTemplate request_src_template = { static void gst_libcamera_src_init(GstLibcameraSrc *self) { + GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); + + self->srcpad = gst_pad_new_from_template(templ, "src"); + gst_element_add_pad(GST_ELEMENT(self), self->srcpad); } static void From patchwork Wed Jan 29 03:31:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2741 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 62A96608D3 for ; Wed, 29 Jan 2020 04:35:25 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 211E328EA9F; Wed, 29 Jan 2020 03:35:23 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:54 -0500 Message-Id: <20200129033210.278800-8-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 07/23] gst: libcamerasrc: Add camera-name property X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:25 -0000 From: Nicolas Dufresne This property will be used to select by name the camera to use. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 68 +++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 50bba39..74e1d7e 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -8,10 +8,17 @@ #include "gstlibcamerasrc.h" #include "gstlibcamerapad.h" +#include "gstlibcamera-utils.h" struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; + gchar *camera_name; +}; + +enum { + PROP_0, + PROP_CAMERA_NAME }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); @@ -28,6 +35,53 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +static void +gst_libcamera_src_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + GST_OBJECT_LOCKER(object); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + switch (prop_id) { + case PROP_CAMERA_NAME: + g_free (self->camera_name); + self->camera_name = g_value_dup_string(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, + GParamSpec *pspec) +{ + GST_OBJECT_LOCKER(object); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + switch (prop_id) { + case PROP_CAMERA_NAME: + g_value_set_string(value, self->camera_name); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } + +} + +static void +gst_libcamera_src_finalize(GObject *object) +{ + GObjectClass *klass = G_OBJECT_CLASS(gst_libcamera_src_parent_class); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + g_free(self->camera_name); + + return klass->finalize(object); +} + static void gst_libcamera_src_init(GstLibcameraSrc *self) { @@ -41,6 +95,12 @@ static void gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) { GstElementClass *element_class = (GstElementClass *)klass; + GObjectClass *object_class = (GObjectClass *)klass; + GParamSpec *spec; + + object_class->set_property = gst_libcamera_src_set_property; + object_class->get_property = gst_libcamera_src_get_property; + object_class->finalize = gst_libcamera_src_finalize; gst_element_class_set_metadata(element_class, "LibCamera Source", "Source/Video", @@ -52,4 +112,12 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) gst_element_class_add_static_pad_template_with_gtype(element_class, &request_src_template, GST_TYPE_LIBCAMERA_PAD); + + spec = g_param_spec_string("camera-name", "Camera Name", + "Select by name which camera to use.", nullptr, + (GParamFlags)(GST_PARAM_MUTABLE_READY + | G_PARAM_CONSTRUCT + | G_PARAM_READWRITE + | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_CAMERA_NAME, spec); } From patchwork Wed Jan 29 03:31:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2742 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E04B608D3 for ; Wed, 29 Jan 2020 04:35:26 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 8339528EA9F; Wed, 29 Jan 2020 03:35:25 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:55 -0500 Message-Id: <20200129033210.278800-9-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 08/23] gst: libcamerasrc: Add a debug category X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:26 -0000 From: Nicolas Dufresne This will allow selecting libcamerasrc traces with the following environment: GST_DEBUG=libcamerasrc:7 Or all libcamera GStreamer element traces using GST_DEBUG="libcamera*:7" Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 74e1d7e..2177a8d 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -10,6 +10,9 @@ #include "gstlibcamerapad.h" #include "gstlibcamera-utils.h" +GST_DEBUG_CATEGORY_STATIC(source_debug); +#define GST_CAT_DEFAULT source_debug + struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; @@ -21,7 +24,9 @@ enum { PROP_CAMERA_NAME }; -G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); +G_DEFINE_TYPE_WITH_CODE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT, + GST_DEBUG_CATEGORY_INIT(source_debug, "libcamerasrc", 0, + "LibCamera Source")); #define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw;image/jpeg") From patchwork Wed Jan 29 03:31:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2743 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A7AA608BF for ; Wed, 29 Jan 2020 04:35:28 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id BA46328EA9F; Wed, 29 Jan 2020 03:35:26 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:56 -0500 Message-Id: <20200129033210.278800-10-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 09/23] gst: libcamerasrc: Implement selection and acquisition X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:29 -0000 From: Nicolas Dufresne This add code to select and acquire a camera. With this, it is now possible to run pipeline like: gst-launch-1.0 libcamerasrc ! fakesink Though no buffer will be streamed yet. In this function, we implement the change_state() virtual method to trigger actions on specific state transitions. Note that we also return GST_STATE_CHANGE_NO_PREROLL in GST_STATE_CHANGE_READY_TO_PAUSED and GST_STATE_CHANGE_PLAYING_TO_PAUSED transitions as this is required for all live sources. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerasrc.cpp | 124 ++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 2177a8d..abb376b 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -10,13 +10,26 @@ #include "gstlibcamerapad.h" #include "gstlibcamera-utils.h" +#include +#include + +using namespace libcamera; + GST_DEBUG_CATEGORY_STATIC(source_debug); #define GST_CAT_DEFAULT source_debug +/* Used for C++ object with destructors */ +struct GstLibcameraSrcState { + std::shared_ptr cm; + std::shared_ptr cam; +}; + struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; gchar *camera_name; + + GstLibcameraSrcState *state; }; enum { @@ -40,6 +53,83 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +static bool +gst_libcamera_src_open(GstLibcameraSrc *self) +{ + std::shared_ptr cm = std::make_shared(); + std::shared_ptr cam = nullptr; + gint ret = 0; + g_autofree gchar *camera_name = nullptr; + + GST_DEBUG_OBJECT(self, "Opening camera device ..."); + + ret = cm->start(); + if (ret) { + GST_ELEMENT_ERROR(self, LIBRARY, INIT, + ("Failed listing cameras."), + ("libcamera::CameraMananger.start() failed: %s", g_strerror(-ret))); + return false; + } + + { + GST_OBJECT_LOCKER(self); + if (self->camera_name) + camera_name = g_strdup(self->camera_name); + } + + if (camera_name) { + cam = cm->get(self->camera_name); + if (!cam) { + GST_ELEMENT_ERROR(self, RESOURCE, NOT_FOUND, + ("Could not find a camera named '%s'.", self->camera_name), + ("libcamera::CameraMananger.get() returned NULL")); + return false; + } + } else { + if (cm->cameras().empty()) { + GST_ELEMENT_ERROR(self, RESOURCE, NOT_FOUND, + ("Could not find any supported camera on this system."), + ("libcamera::CameraMananger.cameras() is empty")); + return false; + } + cam = cm->cameras()[0]; + } + + GST_INFO_OBJECT(self, "Using camera named '%s'", cam->name().c_str()); + + ret = cam->acquire(); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, BUSY, + ("Camera name '%s' is already in use.", cam->name().c_str()), + ("libcamera::Camera.acquire() failed: %s", g_strerror(ret))); + return false; + } + + /* no need to lock here we didn't start our threads */ + self->state->cm = cm; + self->state->cam = cam; + + return true; +} + +static void +gst_libcamera_src_close(GstLibcameraSrc *self) +{ + GstLibcameraSrcState *state = self->state; + gint ret; + + ret = state->cam->release(); + if (ret) { + GST_ELEMENT_WARNING(self, RESOURCE, BUSY, + ("Camera name '%s' is still in use.", state->cam->name().c_str()), + ("libcamera::Camera.release() failed: %s", g_strerror(-ret))); + } + + state->cam.reset(); + state->cm->stop(); + state->cm.reset(); +} + static void gst_libcamera_src_set_property(GObject *object, guint prop_id, const GValue *value, GParamSpec *pspec) @@ -73,7 +163,36 @@ gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); break; } +} +static GstStateChangeReturn +gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(element); + GstStateChangeReturn ret = GST_STATE_CHANGE_SUCCESS; + GstElementClass *klass = GST_ELEMENT_CLASS(gst_libcamera_src_parent_class); + + ret = klass->change_state(element, transition); + if (ret == GST_STATE_CHANGE_FAILURE) + return ret; + + switch (transition) { + case GST_STATE_CHANGE_NULL_TO_READY: + if (!gst_libcamera_src_open(self)) + return GST_STATE_CHANGE_FAILURE; + break; + case GST_STATE_CHANGE_READY_TO_PAUSED: + case GST_STATE_CHANGE_PLAYING_TO_PAUSED: + ret = GST_STATE_CHANGE_NO_PREROLL; + break; + case GST_STATE_CHANGE_READY_TO_NULL: + gst_libcamera_src_close(self); + break; + default: + break; + } + + return ret; } static void @@ -83,6 +202,7 @@ gst_libcamera_src_finalize(GObject *object) GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); g_free(self->camera_name); + delete self->state; return klass->finalize(object); } @@ -90,10 +210,12 @@ gst_libcamera_src_finalize(GObject *object) static void gst_libcamera_src_init(GstLibcameraSrc *self) { + GstLibcameraSrcState *state = new GstLibcameraSrcState(); GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); self->srcpad = gst_pad_new_from_template(templ, "src"); gst_element_add_pad(GST_ELEMENT(self), self->srcpad); + self->state = state; } static void @@ -107,6 +229,8 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) object_class->get_property = gst_libcamera_src_get_property; object_class->finalize = gst_libcamera_src_finalize; + element_class->change_state = gst_libcamera_src_change_state; + gst_element_class_set_metadata(element_class, "LibCamera Source", "Source/Video", "Linux Camera source using libcamera", From patchwork Wed Jan 29 03:31:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2744 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 11921608CA for ; Wed, 29 Jan 2020 04:35:30 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 72EC828EA9F; Wed, 29 Jan 2020 03:35:28 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:57 -0500 Message-Id: <20200129033210.278800-11-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 10/23] gst: libcamerasrc: Start/Stop the streaming thread X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:30 -0000 From: Nicolas Dufresne Use a GstTask as our internal streaming thread. Unlike GstBaseSrc, we will be running an streaming thread at the element level rather then per pad. This is needed to combine buffer request for multiple pads. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerasrc.cpp | 45 +++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index abb376b..4d0c7d0 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -26,7 +26,11 @@ struct GstLibcameraSrcState { struct _GstLibcameraSrc { GstElement parent; + + GRecMutex stream_lock; + GstTask *task; GstPad *srcpad; + gchar *camera_name; GstLibcameraSrcState *state; @@ -112,6 +116,30 @@ gst_libcamera_src_open(GstLibcameraSrc *self) return true; } +static void +gst_libcamera_src_task_run(gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread it now capturing"); +} + +static void +gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread has started"); +} + +static void +gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); +} + static void gst_libcamera_src_close(GstLibcameraSrc *self) { @@ -182,9 +210,18 @@ gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) return GST_STATE_CHANGE_FAILURE; break; case GST_STATE_CHANGE_READY_TO_PAUSED: + /* This needs to be called after pads activation */ + if (!gst_task_pause(self->task)) + return GST_STATE_CHANGE_FAILURE; + ret = GST_STATE_CHANGE_NO_PREROLL; + break; case GST_STATE_CHANGE_PLAYING_TO_PAUSED: ret = GST_STATE_CHANGE_NO_PREROLL; break; + case GST_STATE_CHANGE_PAUSED_TO_READY: + /* FIXME maybe this should happen before pad deactivation ? */ + gst_task_join(self->task); + break; case GST_STATE_CHANGE_READY_TO_NULL: gst_libcamera_src_close(self); break; @@ -201,6 +238,8 @@ gst_libcamera_src_finalize(GObject *object) GObjectClass *klass = G_OBJECT_CLASS(gst_libcamera_src_parent_class); GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + g_rec_mutex_clear(&self->stream_lock); + g_clear_object(&self->task); g_free(self->camera_name); delete self->state; @@ -213,6 +252,12 @@ gst_libcamera_src_init(GstLibcameraSrc *self) GstLibcameraSrcState *state = new GstLibcameraSrcState(); GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); + g_rec_mutex_init(&self->stream_lock); + self->task = gst_task_new(gst_libcamera_src_task_run, self, nullptr); + gst_task_set_enter_callback(self->task, gst_libcamera_src_task_enter, self, nullptr); + gst_task_set_leave_callback(self->task, gst_libcamera_src_task_leave, self, nullptr); + gst_task_set_lock(self->task, &self->stream_lock); + self->srcpad = gst_pad_new_from_template(templ, "src"); gst_element_add_pad(GST_ELEMENT(self), self->srcpad); self->state = state; From patchwork Wed Jan 29 03:31:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2745 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F2BA9608BD for ; Wed, 29 Jan 2020 04:35:30 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 2A83628EA9F; Wed, 29 Jan 2020 03:35:30 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:58 -0500 Message-Id: <20200129033210.278800-12-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 11/23] gst: libcamerapad: Add a method to access the role X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:31 -0000 From: Nicolas Dufresne Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 8 ++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ 2 files changed, 10 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index c6c22e4..b18584c 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -99,3 +99,11 @@ gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) | G_PARAM_STATIC_STRINGS)); g_object_class_install_property(object_class, PROP_STREAM_ROLE, spec); } + +StreamRole +gst_libcamera_pad_get_role(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GST_OBJECT_LOCKER(self); + return self->role; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 2e745f1..3dea0e7 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -7,6 +7,7 @@ */ #include +#include #ifndef __GST_LIBCAMERA_PAD_H__ #define __GST_LIBCAMERA_PAD_H__ @@ -15,5 +16,6 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_LIBCAMERA, PAD, GstPad) +libcamera::StreamRole gst_libcamera_pad_get_role(GstPad *pad); #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Wed Jan 29 03:31:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2746 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C907608CA for ; Wed, 29 Jan 2020 04:35:32 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 286C128EA9F; Wed, 29 Jan 2020 03:35:30 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:31:59 -0500 Message-Id: <20200129033210.278800-13-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 12/23] gst: libcamerasrc: Store the srcpad in a vector X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:32 -0000 From: Nicolas Dufresne This will allow implementing generic algorithm even if we cannot request pads yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 4d0c7d0..be32a11 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -22,6 +22,7 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); struct GstLibcameraSrcState { std::shared_ptr cm; std::shared_ptr cam; + std::vector srcpads; }; struct _GstLibcameraSrc { @@ -29,7 +30,6 @@ struct _GstLibcameraSrc { GRecMutex stream_lock; GstTask *task; - GstPad *srcpad; gchar *camera_name; @@ -258,8 +258,8 @@ gst_libcamera_src_init(GstLibcameraSrc *self) gst_task_set_leave_callback(self->task, gst_libcamera_src_task_leave, self, nullptr); gst_task_set_lock(self->task, &self->stream_lock); - self->srcpad = gst_pad_new_from_template(templ, "src"); - gst_element_add_pad(GST_ELEMENT(self), self->srcpad); + state->srcpads.push_back(gst_pad_new_from_template(templ, "src")); + gst_element_add_pad(GST_ELEMENT(self), state->srcpads[0]); self->state = state; } From patchwork Wed Jan 29 03:32:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2747 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AB7D608B9 for ; Wed, 29 Jan 2020 04:35:33 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 52E7F292988; Wed, 29 Jan 2020 03:35:32 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:00 -0500 Message-Id: <20200129033210.278800-14-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 13/23] gst: libcamerasrc: Send stream start event X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:33 -0000 From: Nicolas Dufresne Prior to sending caps, we need to send a stream-start event. This requires generating a stream and a group id. The stream id is random for live sources and the group id is shared across all pads. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index be32a11..b1a21dc 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -18,6 +18,8 @@ using namespace libcamera; GST_DEBUG_CATEGORY_STATIC(source_debug); #define GST_CAT_DEFAULT source_debug +#define STREAM_LOCKER(obj) g_autoptr(GRecMutexLocker) stream_locker = g_rec_mutex_locker_new(&GST_LIBCAMERA_SRC(obj)->stream_lock) + /* Used for C++ object with destructors */ struct GstLibcameraSrcState { std::shared_ptr cm; @@ -127,9 +129,20 @@ gst_libcamera_src_task_run(gpointer user_data) static void gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) { + STREAM_LOCKER(user_data); GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread has started"); + + guint group_id = gst_util_group_id_next(); + for (GstPad *srcpad : state->srcpads) { + /* Create stream-id and push stream-start */ + g_autofree gchar *stream_id = gst_pad_create_stream_id(srcpad, GST_ELEMENT(self), nullptr); + GstEvent *event = gst_event_new_stream_start(stream_id); + gst_event_set_group_id(event, group_id); + gst_pad_push_event(srcpad, event); + } } static void From patchwork Wed Jan 29 03:32:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2748 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 633B7608D4 for ; Wed, 29 Jan 2020 04:35:34 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 53DCF28EA9F; Wed, 29 Jan 2020 03:35:33 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:01 -0500 Message-Id: <20200129033210.278800-15-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 14/23] gst: utils: Add StreamConfiguration helpers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:35 -0000 From: Nicolas Dufresne This add helpers to deal with the conversion from StreamConfiguration to caps and vis-versa. This is needed to implement caps negotiation. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamera-utils.cpp | 61 ++++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 3 ++ 2 files changed, 64 insertions(+) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 2540be0..d9b217a 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -41,6 +41,18 @@ drm_to_gst_format(guint drm_fourcc) return GST_VIDEO_FORMAT_UNKNOWN; } +static inline guint +gst_format_to_drm(GstVideoFormat gst_format) +{ + if (gst_format == GST_VIDEO_FORMAT_ENCODED) + return DRM_FORMAT_INVALID; + + for (guint i = 0; i < FORMAT_MAP_SIZE; i++) + if (format_map[i].gst_format == gst_format) + return format_map[i].drm_fourcc; + return DRM_FORMAT_INVALID; +} + static GstStructure * bare_structure_from_fourcc(guint fourcc) { @@ -100,3 +112,52 @@ gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) return caps; } + +GstCaps * +gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg) +{ + GstCaps *caps = gst_caps_new_empty(); + GstStructure *s = bare_structure_from_fourcc(stream_cfg.pixelFormat); + + gst_structure_set(s, + "width", G_TYPE_INT, stream_cfg.size.width, + "height", G_TYPE_INT, stream_cfg.size.height, + NULL); + gst_caps_append_structure(caps, s); + + return caps; +} + +void +gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, + GstCaps *caps) +{ + GstVideoFormat gst_format = drm_to_gst_format(stream_cfg.pixelFormat); + + /* First fixate the caps using default configuration value */ + g_assert(gst_caps_is_writable(caps)); + caps = gst_caps_truncate(caps); + GstStructure *s = gst_caps_get_structure(caps, 0); + + gst_structure_fixate_field_nearest_int(s, "width", stream_cfg.size.width); + gst_structure_fixate_field_nearest_int(s, "height", stream_cfg.size.height); + + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_video_format_to_string(gst_format); + gst_structure_fixate_field_string(s, "format", format); + } + + /* The configure the stream with the result. */ + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_structure_get_string(s, "format"); + gst_format = gst_video_format_from_string(format); + stream_cfg.pixelFormat = gst_format_to_drm(gst_format); + } else if (gst_structure_has_name(s, "image/jpeg")) { + stream_cfg.pixelFormat = DRM_FORMAT_MJPEG; + } else { + g_critical("Unsupported media type: %s", gst_structure_get_name(s)); + } + + gst_structure_get_int(s, "width", (gint *)&stream_cfg.size.width); + gst_structure_get_int(s, "height", (gint *)&stream_cfg.size.height); +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 528dc2c..91cfdcd 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -16,5 +16,8 @@ #define GST_OBJECT_LOCKER(obj) g_autoptr(GMutexLocker) locker = g_mutex_locker_new(GST_OBJECT_GET_LOCK(obj)) GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); +GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); +void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, + GstCaps *caps); #endif /* __GST_LIBCAMERA_UTILS_H_ */ From patchwork Wed Jan 29 03:32:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2749 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 141F4608CA for ; Wed, 29 Jan 2020 04:35:36 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 8568028EA9F; Wed, 29 Jan 2020 03:35:34 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:02 -0500 Message-Id: <20200129033210.278800-16-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 15/23] gst: libcamerasrc: Implement minimal caps negotiation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:36 -0000 From: Nicolas Dufresne This is not expected to work in every possible cases, but should be sufficient as an initial implementation. What is does it that it turns the StreamFormats into caps and query downstream caps with that as a filter. The result is the subset of caps that can be used. We then keep the first structure in that result and fixate using the default values found in StreamConfiguration as a default in case a range is available. We then validate this configuration and turn the potentially modified configuration into caps that we push downstream. Note that we strust the order in StreamFormats as being sorted best first, but this is not currently in libcamera. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerasrc.cpp | 71 +++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index b1a21dc..7b478fa 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -24,6 +24,7 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); struct GstLibcameraSrcState { std::shared_ptr cm; std::shared_ptr cam; + std::unique_ptr config; std::vector srcpads; }; @@ -132,16 +133,86 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) STREAM_LOCKER(user_data); GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); GstLibcameraSrcState *state = self->state; + GstFlowReturn flow_ret = GST_FLOW_OK; GST_DEBUG_OBJECT(self, "Streaming thread has started"); guint group_id = gst_util_group_id_next(); + StreamRoles roles; for (GstPad *srcpad : state->srcpads) { /* Create stream-id and push stream-start */ g_autofree gchar *stream_id = gst_pad_create_stream_id(srcpad, GST_ELEMENT(self), nullptr); GstEvent *event = gst_event_new_stream_start(stream_id); gst_event_set_group_id(event, group_id); gst_pad_push_event(srcpad, event); + + /* Collect the streams roles for the next iteration */ + roles.push_back(gst_libcamera_pad_get_role(srcpad)); + } + + /* Generate the stream configurations, there should be one per pad */ + state->config = state->cam->generateConfiguration(roles); + g_assert(state->config->size() == state->srcpads.size()); + + for (gsize i = 0; i < state->srcpads.size(); i++) { + GstPad *srcpad = state->srcpads[i]; + StreamConfiguration &stream_cfg = state->config->at(i); + + /* Retreive the supported caps */ + g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); + g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad, filter); + if (gst_caps_is_empty(caps)) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + break; + } + + /* Fixate caps and configure the stream */ + caps = gst_caps_make_writable(caps); + gst_libcamera_configure_stream_from_caps(stream_cfg, caps); + } + + if (flow_ret != GST_FLOW_OK) + goto done; + + /* Validate the configuration */ + if (state->config->validate() == CameraConfiguration::Invalid) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + goto done; + } + + /* Regardless if it has been modified, create clean caps and push the + * caps event, downstream will decide if hte caps are acceptable */ + for (gsize i = 0; i < state->srcpads.size(); i++) { + GstPad *srcpad = state->srcpads[i]; + const StreamConfiguration &stream_cfg = state->config->at(i); + + g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg); + if (!gst_pad_push_event(srcpad, gst_event_new_caps(caps))) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + break; + } + } + + ret = state->cam->configure(state->config.get()); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Failed to configure camera: %s", g_strerror(-ret)), + ("Camera.configure() failed with error code %i", ret)); + gst_task_stop(task); + return; + } + +done: + switch (flow_ret) { + case GST_FLOW_NOT_NEGOTIATED: + for (GstPad *srcpad : state->srcpads) { + gst_pad_push_event(srcpad, gst_event_new_eos()); + } + GST_ELEMENT_FLOW_ERROR(self, flow_ret); + gst_task_stop(task); + return; + default: + break; } } From patchwork Wed Jan 29 03:32:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2750 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DF14608B8 for ; Wed, 29 Jan 2020 04:35:37 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 5E09128EA9F; Wed, 29 Jan 2020 03:35:36 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:03 -0500 Message-Id: <20200129033210.278800-17-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 16/23] gst: libcamerasrc: Push segment event X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:37 -0000 From: Nicolas Dufresne Now that we have stream-start and caps, we can now push a segment event to announce what time will our buffer correlate to. For live sources this is just an open segment in time format. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 7b478fa..0df71c6 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -191,6 +191,11 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) flow_ret = GST_FLOW_NOT_NEGOTIATED; break; } + + /* Send an open segment event with time format */ + GstSegment segment; + gst_segment_init(&segment, GST_FORMAT_TIME); + gst_pad_push_event(srcpad, gst_event_new_segment(&segment)); } ret = state->cam->configure(state->config.get()); From patchwork Wed Jan 29 03:32:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2751 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E026608BE for ; Wed, 29 Jan 2020 04:35:39 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 5791F2942F4; Wed, 29 Jan 2020 03:35:37 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:04 -0500 Message-Id: <20200129033210.278800-18-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 17/23] gst: Add a pool and an allocator implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:39 -0000 From: Nicolas Dufresne This is needed to track the livetime of the FrameBufferAllocator in relation to the GstBuffer/GstMemory objects travelling inside GStreamer. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcameraallocator.cpp | 240 ++++++++++++++++++++++++ src/gstreamer/gstlibcameraallocator.h | 29 +++ src/gstreamer/gstlibcamerapool.cpp | 109 +++++++++++ src/gstreamer/gstlibcamerapool.h | 27 +++ src/gstreamer/meson.build | 11 +- 5 files changed, 413 insertions(+), 3 deletions(-) create mode 100644 src/gstreamer/gstlibcameraallocator.cpp create mode 100644 src/gstreamer/gstlibcameraallocator.h create mode 100644 src/gstreamer/gstlibcamerapool.cpp create mode 100644 src/gstreamer/gstlibcamerapool.h diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp new file mode 100644 index 0000000..0f248b4 --- /dev/null +++ b/src/gstreamer/gstlibcameraallocator.cpp @@ -0,0 +1,240 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraallocator.cpp - GStreamer Custom Allocator + */ + +#include "gstlibcameraallocator.h" +#include "gstlibcamera-utils.h" + +#include +#include +#include + +using namespace libcamera; + +/***********************************************************************/ +/* Internal object for tracking memories associated with a FrameBuffer */ +/***********************************************************************/ + +static gboolean gst_libcamera_allocator_release(GstMiniObject *mini_object); + +/* This internal object is used to track the outstanding GstMemory object that + * are part of a FrameBuffer. The allocator will only re-use a FrameBuffer when + * al all outstranging GstMemory have returned. + */ +struct FrameWrap { + FrameWrap(GstAllocator *allocator, FrameBuffer *buffer, + gpointer stream); + ~FrameWrap(); + + void AcquirePlane() { ++outstanding_planes_; } + bool ReleasePlane() { return --outstanding_planes_ == 0; } + + gpointer stream_; + FrameBuffer *buffer_; + std::vector planes_; + gint outstanding_planes_; +}; + +static GQuark +gst_libcamera_frame_quark(void) +{ + static gsize frame_quark = 0; + + if (g_once_init_enter(&frame_quark)) { + GQuark quark = g_quark_from_string("GstLibCameraFrameWrap"); + g_once_init_leave(&frame_quark, quark); + } + + return frame_quark; +} + +FrameWrap::FrameWrap(GstAllocator *allocator, FrameBuffer *buffer, + gpointer stream) + + : stream_(stream), + buffer_(buffer), + outstanding_planes_(0) +{ + for (const FrameBuffer::Plane &plane : buffer->planes()) { + GstMemory *mem = gst_fd_allocator_alloc(allocator, + plane.fd.fd(), + plane.length, + GST_FD_MEMORY_FLAG_DONT_CLOSE); + gst_mini_object_set_qdata(GST_MINI_OBJECT(mem), + gst_libcamera_frame_quark(), + this, NULL); + GST_MINI_OBJECT(mem)->dispose = gst_libcamera_allocator_release; + g_object_unref(mem->allocator); + planes_.push_back(mem); + } +} + +FrameWrap::~FrameWrap() +{ + for (GstMemory *mem : planes_) { + GST_MINI_OBJECT(mem)->dispose = nullptr; + g_object_ref(mem->allocator); + gst_memory_unref(mem); + } +} + +/***********************************/ +/* The GstAllocator implementation */ +/***********************************/ +struct _GstLibcameraAllocator { + GstDmaBufAllocator parent; + FrameBufferAllocator *fb_allocator; + /* A hash table using Stream pointer as key and returning a GQueue of + * FrameWrap. */ + GHashTable *pools; +}; + +G_DEFINE_TYPE(GstLibcameraAllocator, gst_libcamera_allocator, + GST_TYPE_DMABUF_ALLOCATOR); + +static gboolean +gst_libcamera_allocator_release(GstMiniObject *mini_object) +{ + GstMemory *mem = GST_MEMORY_CAST(mini_object); + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(mem->allocator); + GST_OBJECT_LOCKER(self); + gpointer data = gst_mini_object_get_qdata(mini_object, + gst_libcamera_frame_quark()); + auto *frame = (FrameWrap *)data; + + gst_memory_ref(mem); + + if (frame->ReleasePlane()) { + auto *pool = (GQueue *)g_hash_table_lookup(self->pools, frame->stream_); + g_return_val_if_fail(pool, TRUE); + g_queue_push_tail(pool, frame); + } + + /* Keep last in case we are holding on the last allocator ref */ + g_object_unref(mem->allocator); + + /* Rreturns FALSE so that our mini object isn't freed */ + return FALSE; +} + +static void +gst_libcamera_allocator_free_pool(gpointer data) +{ + GQueue *queue = (GQueue *)data; + FrameWrap *frame; + + while ((frame = (FrameWrap *)g_queue_pop_head(queue))) { + g_warn_if_fail(frame->outstanding_planes_ == 0); + delete frame; + } + + g_queue_free(queue); +} + +static void +gst_libcamera_allocator_init(GstLibcameraAllocator *self) +{ + self->pools = g_hash_table_new_full(NULL, NULL, NULL, + gst_libcamera_allocator_free_pool); + GST_OBJECT_FLAG_SET(self, GST_ALLOCATOR_FLAG_CUSTOM_ALLOC); +} + +static void +gst_libcamera_allocator_dispose(GObject *object) +{ + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object); + + if (self->pools) { + g_hash_table_unref(self->pools); + self->pools = NULL; + } + + G_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->dispose(object); +} + +static void +gst_libcamera_allocator_finalize(GObject *object) +{ + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object); + + delete self->fb_allocator; + + G_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->finalize(object); +} + +static void +gst_libcamera_allocator_class_init(GstLibcameraAllocatorClass *klass) +{ + auto *allocator_class = GST_ALLOCATOR_CLASS(klass); + auto *object_class = G_OBJECT_CLASS(klass); + + object_class->dispose = gst_libcamera_allocator_dispose; + object_class->finalize = gst_libcamera_allocator_finalize; + allocator_class->alloc = NULL; +} + +GstLibcameraAllocator * +gst_libcamera_allocator_new(std::shared_ptr camera) +{ + auto *self = (GstLibcameraAllocator *)g_object_new(GST_TYPE_LIBCAMERA_ALLOCATOR, + nullptr); + + self->fb_allocator = FrameBufferAllocator::create(camera); + for (Stream *stream : camera->streams()) { + gint ret; + + ret = self->fb_allocator->allocate(stream); + if (ret == 0) + return nullptr; + + GQueue *pool = g_queue_new(); + for (const std::unique_ptr &buffer : + self->fb_allocator->buffers(stream)) { + auto *fb = new FrameWrap(GST_ALLOCATOR(self), + buffer.get(), stream); + g_queue_push_tail(pool, fb); + } + + g_hash_table_insert(self->pools, stream, pool); + } + + return self; +} + +bool +gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, + Stream *stream, GstBuffer *buffer) +{ + GST_OBJECT_LOCKER(self); + + auto *pool = (GQueue *)g_hash_table_lookup(self->pools, stream); + g_return_val_if_fail(pool, false); + + auto *frame = (FrameWrap *)g_queue_pop_head(pool); + if (!frame) + return false; + + for (GstMemory *mem : frame->planes_) { + frame->AcquirePlane(); + gst_buffer_append_memory(buffer, mem); + g_object_ref(mem->allocator); + } + + return true; +} + +gsize +gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self, + Stream *stream) +{ + GST_OBJECT_LOCKER(self); + + auto *pool = (GQueue *)g_hash_table_lookup(self->pools, stream); + g_return_val_if_fail(pool, false); + + return pool->length; +} diff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h new file mode 100644 index 0000000..f2a0f58 --- /dev/null +++ b/src/gstreamer/gstlibcameraallocator.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraallocator.h - GStreamer Custom Allocator + */ + +#include +#include +#include + +#ifndef __GST_LIBCAMERA_ALLOCATOR_H__ +#define __GST_LIBCAMERA_ALLOCATOR_H__ + +#define GST_TYPE_LIBCAMERA_ALLOCATOR gst_libcamera_allocator_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraAllocator, gst_libcamera_allocator, + GST_LIBCAMERA, ALLOCATOR, GstDmaBufAllocator) + +GstLibcameraAllocator *gst_libcamera_allocator_new(std::shared_ptr camera); + +bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, + libcamera::Stream *stream, + GstBuffer *buffer); + +gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator, + libcamera::Stream *stream); + +#endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */ diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp new file mode 100644 index 0000000..f84d1d6 --- /dev/null +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapool.cpp - GStreamer Buffer Pool + */ + +#include "gstlibcamerapool.h" +#include "gstlibcamera-utils.h" + +#include + +using namespace libcamera; + +struct _GstLibcameraPool { + GstBufferPool parent; + + GstAtomicQueue *queue; + GstLibcameraAllocator *allocator; + Stream *stream; +}; + +G_DEFINE_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_TYPE_BUFFER_POOL); + +static GstFlowReturn +gst_libcamera_pool_acquire_buffer(GstBufferPool *pool, GstBuffer **buffer, + GstBufferPoolAcquireParams *params) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + GstBuffer *buf = (GstBuffer *)gst_atomic_queue_pop(self->queue); + if (!buf) + return GST_FLOW_ERROR; + + if (!gst_libcamera_allocator_prepare_buffer(self->allocator, self->stream, buf)) + return GST_FLOW_ERROR; + + *buffer = buf; + return GST_FLOW_OK; +} + +static void +gst_libcamera_pool_reset_buffer(GstBufferPool *pool, GstBuffer *buffer) +{ + GstBufferPoolClass *klass = GST_BUFFER_POOL_CLASS(gst_libcamera_pool_parent_class); + + /* Clears all the memories and only pool the GstBuffer objects */ + gst_buffer_remove_all_memory(buffer); + klass->reset_buffer(pool, buffer); + GST_BUFFER_FLAGS(buffer) = 0; +} + +static void +gst_libcamera_pool_release_buffer(GstBufferPool *pool, GstBuffer *buffer) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + gst_atomic_queue_push(self->queue, (gpointer)buffer); +} + +static void +gst_libcamera_pool_init(GstLibcameraPool *self) +{ + self->queue = gst_atomic_queue_new(4); +} + +static void +gst_libcamera_pool_finalize(GObject *object) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(object); + GstBuffer *buf; + + while ((buf = (GstBuffer *)gst_atomic_queue_pop(self->queue))) + gst_buffer_unref(buf); + + gst_atomic_queue_unref(self->queue); + g_object_unref(self->allocator); + + G_OBJECT_CLASS(gst_libcamera_pool_parent_class)->finalize(object); +} + +static void +gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass) +{ + auto *object_class = G_OBJECT_CLASS(klass); + auto *pool_class = GST_BUFFER_POOL_CLASS(klass); + + object_class->finalize = gst_libcamera_pool_finalize; + pool_class->start = nullptr; + pool_class->acquire_buffer = gst_libcamera_pool_acquire_buffer; + pool_class->reset_buffer = gst_libcamera_pool_reset_buffer; + pool_class->release_buffer = gst_libcamera_pool_release_buffer; +} + +GstLibcameraPool * +gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) +{ + auto *pool = (GstLibcameraPool *)g_object_new(GST_TYPE_LIBCAMERA_POOL, nullptr); + + pool->allocator = (GstLibcameraAllocator *)g_object_ref(allocator); + pool->stream = stream; + + gsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream); + for (gsize i = 0; i < pool_size; i++) { + GstBuffer *buffer = gst_buffer_new(); + gst_atomic_queue_push(pool->queue, buffer); + } + + return pool; +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h new file mode 100644 index 0000000..ca6b299 --- /dev/null +++ b/src/gstreamer/gstlibcamerapool.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapool.h - GStreamer Buffer Pool + * + * This is a partial implementation of GstBufferPool intended for internal use + * only. This pool cannot be configured or activated. + */ + +#include +#include + +#include "gstlibcameraallocator.h" + +#ifndef __GST_LIBCAMERA_POOL_H__ +#define __GST_LIBCAMERA_POOL_H__ + +#define GST_TYPE_LIBCAMERA_POOL gst_libcamera_pool_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, + GST_LIBCAMERA, POOL, GstBufferPool) + +GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, + libcamera::Stream *stream); + +#endif /* __GST_LIBCAMERA_POOL_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index e497bf4..346910f 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -4,6 +4,8 @@ libcamera_gst_sources = [ 'gstlibcamerasrc.cpp', 'gstlibcameraprovider.cpp', 'gstlibcamerapad.cpp', + 'gstlibcameraallocator.cpp', + 'gstlibcamerapool.cpp' ] libcamera_gst_c_args = [ @@ -11,15 +13,18 @@ libcamera_gst_c_args = [ '-DPACKAGE="@0@"'.format(meson.project_name()), ] -gst_dep = dependency('gstreamer-video-1.0', version : '>=1.16.1', +gst_req = '>=1.16.1' +gstvideo_dep = dependency('gstreamer-video-1.0', version : gst_req, + required : get_option('gstreamer')) +gstallocator_dep = dependency('gstreamer-allocators-1.0', version : gst_req, required : get_option('gstreamer')) -if gst_dep.found() +if gstvideo_dep.found() and gstallocator_dep.found() libcamera_gst = shared_library('gstlibcamera', libcamera_gst_sources, c_args : libcamera_gst_c_args, include_directories : [], - dependencies : [libcamera_dep, gst_dep], + dependencies : [libcamera_dep, gstvideo_dep, gstallocator_dep], install: true, install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')), ) From patchwork Wed Jan 29 03:32:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2752 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73E4F608C6 for ; Wed, 29 Jan 2020 04:35:40 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 6FC0C28EA9F; Wed, 29 Jan 2020 03:35:39 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:05 -0500 Message-Id: <20200129033210.278800-19-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 18/23] gst: libcamerapad: Allow storing a pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:40 -0000 From: Nicolas Dufresne This add get/set helper to store a pool on the pad. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 18 ++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 6 ++++++ 2 files changed, 24 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index b18584c..b9e6dc5 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -16,6 +16,7 @@ using namespace libcamera; struct _GstLibcameraPad { GstPad parent; StreamRole role; + GstLibcameraPool *pool; }; enum { @@ -107,3 +108,20 @@ gst_libcamera_pad_get_role(GstPad *pad) GST_OBJECT_LOCKER(self); return self->role; } + +GstLibcameraPool * +gst_libcamera_pad_get_pool(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + return self->pool; +} + +void +gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (self->pool) + g_object_unref(self->pool); + self->pool = pool; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 3dea0e7..4570c0c 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -9,6 +9,8 @@ #include #include +#include "gstlibcamerapool.h" + #ifndef __GST_LIBCAMERA_PAD_H__ #define __GST_LIBCAMERA_PAD_H__ @@ -18,4 +20,8 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, libcamera::StreamRole gst_libcamera_pad_get_role(GstPad *pad); +GstLibcameraPool *gst_libcamera_pad_get_pool(GstPad *pad); + +void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); + #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Wed Jan 29 03:32:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2753 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50182608C6 for ; Wed, 29 Jan 2020 04:35:42 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 8ECCF28EA9F; Wed, 29 Jan 2020 03:35:40 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:06 -0500 Message-Id: <20200129033210.278800-20-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 19/23] gst: libcamerasrc: Allocate and release buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:42 -0000 From: Nicolas Dufresne Setup the allocation and the release of buffers in the element. We have one pooling GstAllocator that wraps the FrameBufferAllocator and tracks the live time of FrameBuffer object. Then for each pads we have a GstBufferPool object which is only used to avoid re-allocation the GstBuffer shell everything. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerasrc.cpp | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 0df71c6..5fc4393 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -8,6 +8,8 @@ #include "gstlibcamerasrc.h" #include "gstlibcamerapad.h" +#include "gstlibcameraallocator.h" +#include "gstlibcamerapool.h" #include "gstlibcamera-utils.h" #include @@ -37,6 +39,7 @@ struct _GstLibcameraSrc { gchar *camera_name; GstLibcameraSrcState *state; + GstLibcameraAllocator *allocator; }; enum { @@ -207,6 +210,23 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) return; } + self->allocator = gst_libcamera_allocator_new(state->cam); + if (!self->allocator) { + GST_ELEMENT_ERROR(self, RESOURCE, NO_SPACE_LEFT, + ("Failed to allocate memory"), + ("gst_libcamera_allocator_new() failed.")); + gst_task_stop(task); + return; + } + + for (gsize i = 0; i < state->srcpads.size(); i++) { + GstPad *srcpad = state->srcpads[i]; + const StreamConfiguration &stream_cfg = state->config->at(i); + GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, + stream_cfg.stream()); + gst_libcamera_pad_set_pool(srcpad, pool); + } + done: switch (flow_ret) { case GST_FLOW_NOT_NEGOTIATED: @@ -225,8 +245,15 @@ static void gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); + state->cam->stop(); + + for (GstPad *srcpad : state->srcpads) + gst_libcamera_pad_set_pool(srcpad, NULL); + + g_clear_object(&self->allocator); } static void @@ -235,6 +262,8 @@ gst_libcamera_src_close(GstLibcameraSrc *self) GstLibcameraSrcState *state = self->state; gint ret; + GST_DEBUG_OBJECT(self, "Releasing resources"); + ret = state->cam->release(); if (ret) { GST_ELEMENT_WARNING(self, RESOURCE, BUSY, From patchwork Wed Jan 29 03:32:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2754 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 81E35608C6 for ; Wed, 29 Jan 2020 04:35:44 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 6A00328EA9F; Wed, 29 Jan 2020 03:35:42 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:07 -0500 Message-Id: <20200129033210.278800-21-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 20/23] gst: Add getters for Stream and FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:44 -0000 From: Nicolas Dufresne This add getters on pad/pool/allocator so that we can retrieve the Stream or FrameBuffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcameraallocator.cpp | 9 +++++++++ src/gstreamer/gstlibcameraallocator.h | 2 ++ src/gstreamer/gstlibcamerapad.cpp | 11 +++++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ src/gstreamer/gstlibcamerapool.cpp | 20 ++++++++++++++++++++ src/gstreamer/gstlibcamerapool.h | 7 +++++++ 6 files changed, 51 insertions(+) diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp index 0f248b4..bdc6de6 100644 --- a/src/gstreamer/gstlibcameraallocator.cpp +++ b/src/gstreamer/gstlibcameraallocator.cpp @@ -238,3 +238,12 @@ gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self, return pool->length; } + +FrameBuffer * +gst_libcamera_memory_get_frame_buffer(GstMemory *mem) +{ + gpointer data = gst_mini_object_get_qdata(GST_MINI_OBJECT_CAST(mem), + gst_libcamera_frame_quark()); + auto *frame = (FrameWrap *)data; + return frame->buffer_; +} diff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h index f2a0f58..b4de65a 100644 --- a/src/gstreamer/gstlibcameraallocator.h +++ b/src/gstreamer/gstlibcameraallocator.h @@ -26,4 +26,6 @@ bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +libcamera::FrameBuffer *gst_libcamera_memory_get_frame_buffer(GstMemory *mem); + #endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */ diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index b9e6dc5..6401810 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -125,3 +125,14 @@ gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool) g_object_unref(self->pool); self->pool = pool; } + +Stream * +gst_libcamera_pad_get_stream(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (self->pool) + return gst_libcamera_pool_get_stream(self->pool); + + return nullptr; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 4570c0c..81d0d58 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -24,4 +24,6 @@ GstLibcameraPool *gst_libcamera_pad_get_pool(GstPad *pad); void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); +libcamera::Stream *gst_libcamera_pad_get_stream(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp index f84d1d6..6ab01cb 100644 --- a/src/gstreamer/gstlibcamerapool.cpp +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -107,3 +107,23 @@ gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) return pool; } + +Stream * +gst_libcamera_pool_get_stream(GstLibcameraPool *self) +{ + return self->stream; +} + +Stream * +gst_libcamera_buffer_get_stream(GstBuffer *buffer) +{ + auto *self = (GstLibcameraPool *)buffer->pool; + return self->stream; +} + +FrameBuffer * +gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer) +{ + GstMemory *mem = gst_buffer_peek_memory(buffer, 0); + return gst_libcamera_memory_get_frame_buffer(mem); +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h index ca6b299..8cc6cdf 100644 --- a/src/gstreamer/gstlibcamerapool.h +++ b/src/gstreamer/gstlibcamerapool.h @@ -24,4 +24,11 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +libcamera::Stream *gst_libcamera_pool_get_stream(GstLibcameraPool *self); + +libcamera::Stream *gst_libcamera_buffer_get_stream(GstBuffer *buffer); + +libcamera::FrameBuffer *gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer); + + #endif /* __GST_LIBCAMERA_POOL_H__ */ From patchwork Wed Jan 29 03:32:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2755 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A1D85608C6 for ; Wed, 29 Jan 2020 04:35:45 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id BF2A828EA9F; Wed, 29 Jan 2020 03:35:44 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:08 -0500 Message-Id: <20200129033210.278800-22-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 21/23] gst: pad: Add method to store retrieve pending buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:45 -0000 From: Nicolas Dufresne These will be useful for streaming. The requestComplete callback will store the buffers on each pads so that the _run() can pick them up and push them through the pads from a streaming thread. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 28 ++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 4 ++++ 2 files changed, 32 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 6401810..4a775e4 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -17,6 +17,7 @@ struct _GstLibcameraPad { GstPad parent; StreamRole role; GstLibcameraPool *pool; + GQueue pending_buffers; }; enum { @@ -136,3 +137,30 @@ gst_libcamera_pad_get_stream(GstPad *pad) return nullptr; } + +void +gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer) +{ + GST_OBJECT_LOCKER(pad); + auto *self = GST_LIBCAMERA_PAD(pad); + + g_queue_push_head(&self->pending_buffers, buffer); +} + +GstFlowReturn +gst_libcamera_pad_push_pending(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GstBuffer *buffer; + GstFlowReturn ret = GST_FLOW_CUSTOM_SUCCESS; + + { + GST_OBJECT_LOCKER(self); + buffer = (GstBuffer *)g_queue_pop_tail(&self->pending_buffers); + } + + if (buffer) + ret = gst_pad_push(pad, buffer); + + return ret; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 81d0d58..d928570 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -26,4 +26,8 @@ void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); libcamera::Stream *gst_libcamera_pad_get_stream(GstPad *pad); +void gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer); + +GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Wed Jan 29 03:32:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2756 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 89840608BE for ; Wed, 29 Jan 2020 04:35:47 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id D7B3928EA9F; Wed, 29 Jan 2020 03:35:45 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:09 -0500 Message-Id: <20200129033210.278800-23-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 22/23] gst: libcamerasrc: Implement initial streaming X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:47 -0000 From: Nicolas Dufresne With this patch, the element is not able to push buffers to the next element in the graph. The buffers are currently missing any metadata like timestamp, sequence number. The handling of the GstFlowReturn for multiple pads isn't using a GstFlowCombiner as it should. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerapad.cpp | 18 ++- src/gstreamer/gstlibcamerapad.h | 2 + src/gstreamer/gstlibcamerasrc.cpp | 191 +++++++++++++++++++++++++++++- 3 files changed, 208 insertions(+), 3 deletions(-) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 4a775e4..7bc44c1 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -152,7 +152,7 @@ gst_libcamera_pad_push_pending(GstPad *pad) { auto *self = GST_LIBCAMERA_PAD(pad); GstBuffer *buffer; - GstFlowReturn ret = GST_FLOW_CUSTOM_SUCCESS; + GstFlowReturn ret = GST_FLOW_OK; { GST_OBJECT_LOCKER(self); @@ -164,3 +164,19 @@ gst_libcamera_pad_push_pending(GstPad *pad) return ret; } + +bool +gst_libcamera_pad_has_pending(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GST_OBJECT_LOCKER(self); + return (self->pending_buffers.length > 0); +} + +void +gst_libcamera_pad_set_latency(GstPad *pad, GstClockTime latency) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GST_OBJECT_LOCKER(self); + self->latency = latency; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index d928570..eb24000 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -30,4 +30,6 @@ void gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer); GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); +bool gst_libcamera_pad_has_pending(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 5fc4393..947a8bf 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -12,8 +12,10 @@ #include "gstlibcamerapool.h" #include "gstlibcamera-utils.h" +#include #include #include +#include using namespace libcamera; @@ -22,12 +24,73 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); #define STREAM_LOCKER(obj) g_autoptr(GRecMutexLocker) stream_locker = g_rec_mutex_locker_new(&GST_LIBCAMERA_SRC(obj)->stream_lock) -/* Used for C++ object with destructors */ +struct RequestWrap { + RequestWrap(Request *request); + ~RequestWrap(); + + void AttachBuffer(GstBuffer *buffer); + GstBuffer *DetachBuffer(Stream *stream); + + /* For ptr comparision only */ + Request *request_; + std::map buffers_; +}; + +RequestWrap::RequestWrap(Request *request) + : request_(request) +{ +} + +RequestWrap::~RequestWrap() +{ + for (std::pair &item : buffers_) { + if (item.second) + gst_buffer_unref(item.second); + } +} + +void +RequestWrap::AttachBuffer(GstBuffer *buffer) +{ + FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); + Stream *stream = gst_libcamera_buffer_get_stream(buffer); + + request_->addBuffer(stream, fb); + + auto item = buffers_.find(stream); + if (item != buffers_.end()) { + gst_buffer_unref(item->second); + item->second = buffer; + } else { + buffers_[stream] = buffer; + } +} + +GstBuffer * +RequestWrap::DetachBuffer(Stream *stream) +{ + GstBuffer *buffer = nullptr; + + auto item = buffers_.find(stream); + if (item != buffers_.end()) { + buffer = item->second; + item->second = nullptr; + } + + return buffer; +} + +/* Used for C++ object with destructors and callbacks */ struct GstLibcameraSrcState { + GstLibcameraSrc *src; + std::shared_ptr cm; std::shared_ptr cam; std::unique_ptr config; std::vector srcpads; + std::queue> requests; + + void requestCompleted(Request *request); }; struct _GstLibcameraSrc { @@ -40,6 +103,7 @@ struct _GstLibcameraSrc { GstLibcameraSrcState *state; GstLibcameraAllocator *allocator; + GstFlowCombiner *flow_combiner; }; enum { @@ -63,6 +127,41 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +void +GstLibcameraSrcState::requestCompleted(Request *request) +{ + GST_OBJECT_LOCKER(this->src); + + GST_DEBUG_OBJECT(this->src, "buffers are ready"); + + std::unique_ptr wrap = std::move(this->requests.front()); + this->requests.pop(); + + g_return_if_fail(wrap->request_ == request); + + if ((request->status() == Request::RequestCancelled)) { + GST_DEBUG_OBJECT(this->src, "Request was cancelled"); + return; + } + + GstBuffer *buffer; + for (GstPad *srcpad : this->srcpads) { + Stream *stream = gst_libcamera_pad_get_stream(srcpad); + buffer = wrap->DetachBuffer(stream); + gst_libcamera_pad_queue_buffer(srcpad, buffer); + } + + { + /* We only want to resume the task if it's paused */ + GstTask *task = this->src->task; + GST_OBJECT_LOCKER(task); + if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { + GST_TASK_STATE(task) = GST_TASK_STARTED; + GST_TASK_SIGNAL(task); + } + } +} + static bool gst_libcamera_src_open(GstLibcameraSrc *self) { @@ -115,6 +214,8 @@ gst_libcamera_src_open(GstLibcameraSrc *self) return false; } + cam->requestCompleted.connect(self->state, &GstLibcameraSrcState::requestCompleted); + /* no need to lock here we didn't start our threads */ self->state->cm = cm; self->state->cam = cam; @@ -126,8 +227,74 @@ static void gst_libcamera_src_task_run(gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; + + Request *request = new Request(state->cam.get()); + auto wrap = std::make_unique(request); + for (GstPad *srcpad : state->srcpads) { + GstLibcameraPool *pool = gst_libcamera_pad_get_pool(srcpad); + GstBuffer *buffer; + GstFlowReturn ret; + + ret = gst_buffer_pool_acquire_buffer(GST_BUFFER_POOL(pool), + &buffer, nullptr); + if (ret != GST_FLOW_OK) { + /* RequestWrap does not take ownership, and we won't be + * queueing this one due to lack of buffers */ + delete request; + request = NULL; + break; + } + + wrap->AttachBuffer(buffer); + } + + if (request) { + GST_OBJECT_LOCKER(self); + GST_TRACE_OBJECT(self, "Requesting buffers"); + state->cam->queueRequest(request); + state->requests.push(std::move(wrap)); + } + + GstFlowReturn ret = GST_FLOW_OK; + gst_flow_combiner_reset(self->flow_combiner); + for (GstPad *srcpad : state->srcpads) { + ret = gst_libcamera_pad_push_pending(srcpad); + ret = gst_flow_combiner_update_pad_flow(self->flow_combiner, + srcpad, ret); + } + + { + /* Here we need to decide if we want to pause or stop the task. This + * needs to happend in lock step with the callback thread which may want + * to resume the task. + */ + GST_OBJECT_LOCKER(self); + if (ret != GST_FLOW_OK) { + if (ret == GST_FLOW_EOS) { + g_autoptr(GstEvent) eos = gst_event_new_eos(); + guint32 seqnum = gst_util_seqnum_next(); + gst_event_set_seqnum(eos, seqnum); + for (GstPad *srcpad : state->srcpads) + gst_pad_push_event(srcpad, gst_event_ref(eos)); + } else if (ret != GST_FLOW_FLUSHING) { + GST_ELEMENT_FLOW_ERROR(self, ret); + } + gst_task_stop(self->task); + return; + } - GST_DEBUG_OBJECT(self, "Streaming thread it now capturing"); + gboolean do_pause = true; + for (GstPad *srcpad : state->srcpads) { + if (gst_libcamera_pad_has_pending(srcpad)) { + do_pause = false; + break; + } + } + + if (do_pause) + gst_task_pause(self->task); + } } static void @@ -137,6 +304,7 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); GstLibcameraSrcState *state = self->state; GstFlowReturn flow_ret = GST_FLOW_OK; + gint ret = 0; GST_DEBUG_OBJECT(self, "Streaming thread has started"); @@ -219,12 +387,23 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) return; } + self->flow_combiner = gst_flow_combiner_new(); for (gsize i = 0; i < state->srcpads.size(); i++) { GstPad *srcpad = state->srcpads[i]; const StreamConfiguration &stream_cfg = state->config->at(i); GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, stream_cfg.stream()); gst_libcamera_pad_set_pool(srcpad, pool); + gst_flow_combiner_add_pad(self->flow_combiner, srcpad); + } + + ret = state->cam->start(); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Failed to start the camera: %s", g_strerror(-ret)), + ("Camera.start() failed with error code %i", ret)); + gst_task_stop(task); + return; } done: @@ -254,6 +433,8 @@ gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) gst_libcamera_pad_set_pool(srcpad, NULL); g_clear_object(&self->allocator); + g_clear_pointer(&self->flow_combiner, + (GDestroyNotify)gst_flow_combiner_free); } static void @@ -333,6 +514,9 @@ gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) return GST_STATE_CHANGE_FAILURE; ret = GST_STATE_CHANGE_NO_PREROLL; break; + case GST_STATE_CHANGE_PAUSED_TO_PLAYING: + gst_task_start(self->task); + break; case GST_STATE_CHANGE_PLAYING_TO_PAUSED: ret = GST_STATE_CHANGE_NO_PREROLL; break; @@ -378,6 +562,9 @@ gst_libcamera_src_init(GstLibcameraSrc *self) state->srcpads.push_back(gst_pad_new_from_template(templ, "src")); gst_element_add_pad(GST_ELEMENT(self), state->srcpads[0]); + + /* C-style friend */ + state->src = self; self->state = state; } From patchwork Wed Jan 29 03:32:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2757 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DF988608C5 for ; Wed, 29 Jan 2020 04:35:48 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id E866228EA9F; Wed, 29 Jan 2020 03:35:47 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Tue, 28 Jan 2020 22:32:10 -0500 Message-Id: <20200129033210.278800-24-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200129033210.278800-1-nicolas@ndufresne.ca> References: <20200129033210.278800-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 23/23] gst: libcamerasrc: Add a TODO comment X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2020 03:35:49 -0000 From: Nicolas Dufresne This is to guide upcoming contributors toward what is left to do to get toward a production element. Signed-off-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerasrc.cpp | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 947a8bf..878ae2f 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -6,6 +6,26 @@ * gstlibcamerasrc.cpp - GStreamer Capture Element */ +/* TODO + * - Implement GstElement::send_event + * + Allowing application to send EOS + * + Allowing application to use FLUSH/FLUSH_STOP + * + Prevent the main thread from accessing streaming thread + * - Implement renegotiation (even if slow) + * - Implement GstElement::request-new-pad (multi stream) + * + Evaluate if a single streaming thread is fine + * - Add application driven request (snapshot) + * - Add framerate control + * + * Requires new libcamera API: + * - Add framerate negotiation support + * - Add colorimetry support + * - Add timestamp support + * - Use unique names to select the camera + * - Add GstVideoMeta support (strides and offsets) + * - Add buffer importation support + */ + #include "gstlibcamerasrc.h" #include "gstlibcamerapad.h" #include "gstlibcameraallocator.h"