From patchwork Fri Mar 6 20:26:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3007 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99AE560424 for ; Fri, 6 Mar 2020 21:26:57 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 15F5F2970E7; Fri, 6 Mar 2020 20:26:57 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:11 -0500 Message-Id: <20200306202637.525587-2-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 01/27] Add GStreamer plugin and element skeleton X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:26:57 -0000 From: Nicolas Dufresne This implements the GStreamer plugin interface and adds libcamerasrc element feature to it. This is just enough to allow plugin introspection. gst-inspect-1.0 build/src/gstreamer/libgstlibcamera.so Plugin Details: Name libcamera Description libcamera capture plugin Filename build/src/gstreamer/libgstlibcamera.so Version 0.0.0+1042-6c9f16d3-dirty License LGPL Source module libcamera Binary package libcamera Origin URL https://libcamera.org libcamerasrc: libcamera Source 1 features: GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer gst-inspect-1.0 libcamerasrc Factory Details: Rank primary (256) Long-name libcamera Source Klass Source/Video Description Linux Camera source using libcamera Author Nicolas Dufresne Reviewed-by: Laurent Pinchart --- meson_options.txt | 5 +++++ src/gstreamer/gstlibcamera.c | 21 +++++++++++++++++++++ src/gstreamer/gstlibcamerasrc.cpp | 31 +++++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerasrc.h | 22 ++++++++++++++++++++++ src/gstreamer/meson.build | 22 ++++++++++++++++++++++ src/meson.build | 2 ++ 6 files changed, 103 insertions(+) create mode 100644 src/gstreamer/gstlibcamera.c create mode 100644 src/gstreamer/gstlibcamerasrc.cpp create mode 100644 src/gstreamer/gstlibcamerasrc.h create mode 100644 src/gstreamer/meson.build diff --git a/meson_options.txt b/meson_options.txt index 79ee4de..b17b6de 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -7,6 +7,11 @@ option('documentation', type : 'boolean', description : 'Generate the project documentation') +option('gstreamer', + type : 'feature', + value : 'auto', + description : 'Compile libcamera GStreamer plugin') + option('test', type : 'boolean', description: 'Compile and include the tests') diff --git a/src/gstreamer/gstlibcamera.c b/src/gstreamer/gstlibcamera.c new file mode 100644 index 0000000..7dd94ca --- /dev/null +++ b/src/gstreamer/gstlibcamera.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera.c - GStreamer plugin + */ + +#include "gstlibcamerasrc.h" + +static gboolean +plugin_init(GstPlugin *plugin) +{ + return gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_SRC); + return TRUE; +} + +GST_PLUGIN_DEFINE(GST_VERSION_MAJOR, GST_VERSION_MINOR, + libcamera, "libcamera capture plugin", + plugin_init, VERSION, "LGPL", PACKAGE, "https://libcamera.org"); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp new file mode 100644 index 0000000..3807503 --- /dev/null +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerasrc.cpp - GStreamer Capture Element + */ + +#include "gstlibcamerasrc.h" + +struct _GstLibcameraSrc { + GstElement parent; +}; + +G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); + +static void +gst_libcamera_src_init(GstLibcameraSrc *self) +{ +} + +static void +gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) +{ + GstElementClass *element_class = (GstElementClass *)klass; + + gst_element_class_set_metadata(element_class, + "libcamera Source", "Source/Video", + "Linux Camera source using libcamera", + "Nicolas Dufresne + * + * gstlibcamerasrc.h - GStreamer Capture Element + */ + +#ifndef __GST_LIBCAMERA_SRC_H__ +#define __GST_LIBCAMERA_SRC_H__ + +#include + +G_BEGIN_DECLS + +#define GST_TYPE_LIBCAMERA_SRC gst_libcamera_src_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraSrc, gst_libcamera_src, + GST_LIBCAMERA, SRC, GstElement) + +G_END_DECLS + +#endif /* __GST_LIBCAMERA_SRC_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build new file mode 100644 index 0000000..25350f2 --- /dev/null +++ b/src/gstreamer/meson.build @@ -0,0 +1,22 @@ +libcamera_gst_sources = [ + 'gstlibcamera.c', + 'gstlibcamerasrc.cpp', +] + +libcamera_gst_c_args = [ + '-DVERSION="@0@"'.format(libcamera_git_version), + '-DPACKAGE="@0@"'.format(meson.project_name()), +] + +gst_dep = dependency('gstreamer-video-1.0', version : '>=1.14.0', + required : get_option('gstreamer')) + +if gst_dep.found() + libcamera_gst = shared_library('gstlibcamera', + libcamera_gst_sources, + c_args : libcamera_gst_c_args, + dependencies : [libcamera_dep, gst_dep], + install: true, + install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')), + ) +endif diff --git a/src/meson.build b/src/meson.build index 5adcd61..d818d8b 100644 --- a/src/meson.build +++ b/src/meson.build @@ -10,3 +10,5 @@ subdir('qcam') if get_option('v4l2') subdir('v4l2') endif + +subdir('gstreamer') From patchwork Fri Mar 6 20:26:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3008 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BFD9660424 for ; Fri, 6 Mar 2020 21:26:59 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 3D2E22970E7; Fri, 6 Mar 2020 20:26:59 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 6 Mar 2020 15:26:12 -0500 Message-Id: <20200306202637.525587-3-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 02/27] gst: Add utility to convert StreamFormats to GstCaps X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:00 -0000 From: Nicolas Dufresne This transforms the basic information found in StreamFormats to GstCaps. This can be handy to reply to early caps query or inside a device provider. Note that we ignored generated range as they are harmful to caps negotiation. We also don't simplify the caps for readability reasons, so some of the discrete value may be included in a range. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.cpp | 105 +++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 19 +++++ src/gstreamer/meson.build | 1 + 3 files changed, 125 insertions(+) create mode 100644 src/gstreamer/gstlibcamera-utils.cpp create mode 100644 src/gstreamer/gstlibcamera-utils.h diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp new file mode 100644 index 0000000..386ec36 --- /dev/null +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -0,0 +1,105 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera-utils.c - GStreamer libcamera Utility Function + */ + +#include "gstlibcamera-utils.h" + +#include + +using namespace libcamera; + +static struct { + GstVideoFormat gst_format; + guint drm_fourcc; +} format_map[] = { + { GST_VIDEO_FORMAT_ENCODED, DRM_FORMAT_MJPEG }, + { GST_VIDEO_FORMAT_RGB, DRM_FORMAT_BGR888 }, + { GST_VIDEO_FORMAT_BGR, DRM_FORMAT_RGB888 }, + { GST_VIDEO_FORMAT_ARGB, DRM_FORMAT_BGRA8888 }, + { GST_VIDEO_FORMAT_NV12, DRM_FORMAT_NV12 }, + { GST_VIDEO_FORMAT_NV21, DRM_FORMAT_NV21 }, + { GST_VIDEO_FORMAT_NV16, DRM_FORMAT_NV16 }, + { GST_VIDEO_FORMAT_NV61, DRM_FORMAT_NV61 }, + { GST_VIDEO_FORMAT_NV24, DRM_FORMAT_NV24 }, + { GST_VIDEO_FORMAT_UYVY, DRM_FORMAT_UYVY }, + { GST_VIDEO_FORMAT_VYUY, DRM_FORMAT_VYUY }, + { GST_VIDEO_FORMAT_YUY2, DRM_FORMAT_YUYV }, + { GST_VIDEO_FORMAT_YVYU, DRM_FORMAT_YVYU }, + /* \todo NV42 is used in libcamera but is not mapped in GStreamer yet. */ +}; + +static GstVideoFormat +drm_to_gst_format(guint drm_fourcc) +{ + for (const auto &item : format_map) { + if (item.drm_fourcc == drm_fourcc) + return item.gst_format; + } + return GST_VIDEO_FORMAT_UNKNOWN; +} + +static GstStructure * +bare_structure_from_fourcc(guint fourcc) +{ + GstVideoFormat gst_format = drm_to_gst_format(fourcc); + + if (gst_format == GST_VIDEO_FORMAT_UNKNOWN) + return nullptr; + + if (gst_format != GST_VIDEO_FORMAT_ENCODED) + return gst_structure_new("video/x-raw", "format", G_TYPE_STRING, + gst_video_format_to_string(gst_format), nullptr); + + switch (fourcc) { + case DRM_FORMAT_MJPEG: + return gst_structure_new_empty("image/jpeg"); + default: + return nullptr; + } +} + +GstCaps * +gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) +{ + GstCaps *caps = gst_caps_new_empty(); + + for (unsigned int fourcc : formats.pixelformats()) { + g_autoptr(GstStructure) bare_s = bare_structure_from_fourcc(fourcc); + + if (!bare_s) { + GST_WARNING("Unsupported DRM format %" GST_FOURCC_FORMAT, + GST_FOURCC_ARGS(fourcc)); + continue; + } + + for (const Size &size : formats.sizes(fourcc)) { + GstStructure *s = gst_structure_copy(bare_s); + gst_structure_set(s, + "width", G_TYPE_INT, size.width, + "height", G_TYPE_INT, size.height, + nullptr); + gst_caps_append_structure(caps, s); + } + + const SizeRange &range = formats.range(fourcc); + if (range.hStep && range.vStep) { + GstStructure *s = gst_structure_copy(bare_s); + GValue val = G_VALUE_INIT; + + g_value_init(&val, GST_TYPE_INT_RANGE); + gst_value_set_int_range_step(&val, range.min.width, range.max.width, range.hStep); + gst_structure_set_value(s, "width", &val); + gst_value_set_int_range_step(&val, range.min.height, range.max.height, range.vStep); + gst_structure_set_value(s, "height", &val); + g_value_unset(&val); + + gst_caps_append_structure(caps, s); + } + } + + return caps; +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h new file mode 100644 index 0000000..2e4e304 --- /dev/null +++ b/src/gstreamer/gstlibcamera-utils.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamera-utils.h - GStreamer libcamera Utility Functions + */ + +#ifndef __GST_LIBCAMERA_UTILS_H__ +#define __GST_LIBCAMERA_UTILS_H__ + +#include +#include + +#include + +GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); + +#endif /* __GST_LIBCAMERA_UTILS_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 25350f2..1539d9e 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,4 +1,5 @@ libcamera_gst_sources = [ + 'gstlibcamera-utils.cpp', 'gstlibcamera.c', 'gstlibcamerasrc.cpp', ] From patchwork Fri Mar 6 20:26:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3009 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32D8560424 for ; Fri, 6 Mar 2020 21:27:01 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 950942970E7; Fri, 6 Mar 2020 20:27:00 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:13 -0500 Message-Id: <20200306202637.525587-4-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 03/27] gst: Add initial device provider X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:01 -0000 From: Nicolas Dufresne This feature is used with GstDeviceMonitor in order to enumerate and monitor devices to be used with the source element. The resulting GstDevice implementation is also used by application to abstract the configuration of the source element. Implementation notes: - libcamera does not support polling yet - The device ID isn't unique in libcamera yet - The "name" property does not yet exist in libcamerasrc yet Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera.c | 10 +- src/gstreamer/gstlibcameraprovider.cpp | 238 +++++++++++++++++++++++++ src/gstreamer/gstlibcameraprovider.h | 23 +++ src/gstreamer/meson.build | 1 + 4 files changed, 270 insertions(+), 2 deletions(-) create mode 100644 src/gstreamer/gstlibcameraprovider.cpp create mode 100644 src/gstreamer/gstlibcameraprovider.h diff --git a/src/gstreamer/gstlibcamera.c b/src/gstreamer/gstlibcamera.c index 7dd94ca..81c7bb1 100644 --- a/src/gstreamer/gstlibcamera.c +++ b/src/gstreamer/gstlibcamera.c @@ -6,13 +6,19 @@ * gstlibcamera.c - GStreamer plugin */ +#include "gstlibcameraprovider.h" #include "gstlibcamerasrc.h" static gboolean plugin_init(GstPlugin *plugin) { - return gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, - GST_TYPE_LIBCAMERA_SRC); + if (!gst_element_register(plugin, "libcamerasrc", GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_SRC) || + !gst_device_provider_register(plugin, "libcameraprovider", + GST_RANK_PRIMARY, + GST_TYPE_LIBCAMERA_PROVIDER)) + return FALSE; + return TRUE; } diff --git a/src/gstreamer/gstlibcameraprovider.cpp b/src/gstreamer/gstlibcameraprovider.cpp new file mode 100644 index 0000000..914ed4f --- /dev/null +++ b/src/gstreamer/gstlibcameraprovider.cpp @@ -0,0 +1,238 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraprovider.c - GStreamer Device Provider + */ + +#include "gstlibcameraprovider.h" + +#include +#include + +#include "gstlibcamerasrc.h" +#include "gstlibcamera-utils.h" + +using namespace libcamera; + +GST_DEBUG_CATEGORY_STATIC(provider_debug); +#define GST_CAT_DEFAULT provider_debug + +/** + * \struct _GstLibcameraDevice + * \brief libcamera GstDevice implementation + * + * This object is used by GstLibcameraProvider to abstract a libcamera + * device. It also provides helpers to create and configure the + * libcamerasrc GstElement to be used with this device. The implementation is + * private to the plugin. + */ + +enum { + PROP_DEVICE_NAME = 1, +}; + +#define GST_TYPE_LIBCAMERA_DEVICE gst_libcamera_device_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraDevice, gst_libcamera_device, + GST_LIBCAMERA, DEVICE, GstDevice); + +struct _GstLibcameraDevice { + GstDevice parent; + gchar *name; +}; + +G_DEFINE_TYPE(GstLibcameraDevice, gst_libcamera_device, GST_TYPE_DEVICE); + +static GstElement * +gst_libcamera_device_create_element(GstDevice *device, const gchar *name) +{ + GstElement *source = gst_element_factory_make("libcamerasrc", name); + + /* + * Provider and source lives in the same plugin, so making the source + * should never fail. + */ + g_assert(source); + + g_object_set(source, "camera-name", GST_LIBCAMERA_DEVICE(device)->name, nullptr); + + return source; +} + +static gboolean +gst_libcamera_device_reconfigure_element(GstDevice *device, + GstElement *element) +{ + if (!GST_LIBCAMERA_IS_SRC(element)) + return FALSE; + + g_object_set(element, "camera-name", GST_LIBCAMERA_DEVICE(device)->name, nullptr); + + return TRUE; +} + +static void +gst_libcamera_device_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + GstLibcameraDevice *device = GST_LIBCAMERA_DEVICE(object); + + switch (prop_id) { + case PROP_DEVICE_NAME: + device->name = g_value_dup_string(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_device_init(GstLibcameraDevice *self) +{ +} + +static void +gst_libcamera_device_finalize(GObject *object) +{ + GstLibcameraDevice *self = GST_LIBCAMERA_DEVICE(object); + gpointer klass = gst_libcamera_device_parent_class; + + g_free(self->name); + + G_OBJECT_GET_CLASS(klass)->finalize(object); +} + +static void +gst_libcamera_device_class_init(GstLibcameraDeviceClass *klass) +{ + GstDeviceClass *device_class = GST_DEVICE_CLASS(klass); + GObjectClass *object_class = G_OBJECT_CLASS(klass); + + device_class->create_element = gst_libcamera_device_create_element; + device_class->reconfigure_element = gst_libcamera_device_reconfigure_element; + + object_class->set_property = gst_libcamera_device_set_property; + object_class->finalize = gst_libcamera_device_finalize; + + GParamSpec *pspec = g_param_spec_string("name", "Name", + "The name of the camera device", "", + (GParamFlags)(G_PARAM_STATIC_STRINGS | G_PARAM_WRITABLE | + G_PARAM_CONSTRUCT_ONLY)); + g_object_class_install_property(object_class, PROP_DEVICE_NAME, pspec); +} + +static GstDevice * +gst_libcamera_device_new(const std::shared_ptr &camera) +{ + g_autoptr(GstCaps) caps = gst_caps_new_empty(); + const gchar *name = camera->name().c_str(); + StreamRoles roles; + + roles.push_back(StreamRole::VideoRecording); + std::unique_ptr config = camera->generateConfiguration(roles); + + for (const StreamConfiguration &stream_cfg : *config) { + GstCaps *sub_caps = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); + if (sub_caps) + gst_caps_append(caps, sub_caps); + } + + return GST_DEVICE(g_object_new(GST_TYPE_LIBCAMERA_DEVICE, + /* \todo Use a unique identifier instead of camera name. */ + "name", name, + "display-name", name, + "caps", caps, + "device-class", "Source/Video", + nullptr)); +} + +/** + * \struct _GstLibcameraProvider + * \brief libcamera GstDeviceProvider implementation + * + * This GstFeature is used by GstDeviceMonitor to probe the available + * libcamera devices. The implementation is private to the plugin. + */ + +struct _GstLibcameraProvider { + GstDeviceProvider parent; + CameraManager *cm; +}; + +G_DEFINE_TYPE_WITH_CODE(GstLibcameraProvider, gst_libcamera_provider, + GST_TYPE_DEVICE_PROVIDER, + GST_DEBUG_CATEGORY_INIT(provider_debug, "libcamera-provider", 0, + "libcamera Device Provider")); + +static GList * +gst_libcamera_provider_probe(GstDeviceProvider *provider) +{ + GstLibcameraProvider *self = GST_LIBCAMERA_PROVIDER(provider); + CameraManager *cm = self->cm; + GList *devices = nullptr; + gint ret; + + GST_INFO_OBJECT(self, "Probing cameras using libcamera"); + + /* \todo Move the CameraMananger start()/stop() calls into + * GstDeviceProvider start()/stop() virtual function when CameraMananger + * gains monitoring support. Meanwhile we need to cycle start()/stop() + * to ensure every probe() calls return the latest list. + */ + ret = cm->start(); + if (ret) { + GST_ERROR_OBJECT(self, "Failed to retrieve device list: %s", + g_strerror(-ret)); + return nullptr; + } + + for (const std::shared_ptr &camera : cm->cameras()) { + GST_INFO_OBJECT(self, "Found camera '%s'", camera->name().c_str()); + devices = g_list_append(devices, + g_object_ref_sink(gst_libcamera_device_new(camera))); + } + + cm->stop(); + + return devices; +} + +static void +gst_libcamera_provider_init(GstLibcameraProvider *self) +{ + GstDeviceProvider *provider = GST_DEVICE_PROVIDER(self); + + self->cm = new CameraManager(); + + /* Avoid devices being duplicated. */ + gst_device_provider_hide_provider(provider, "v4l2deviceprovider"); +} + +static void +gst_libcamera_provider_finalize(GObject *object) +{ + GstLibcameraProvider *self = GST_LIBCAMERA_PROVIDER(object); + gpointer klass = gst_libcamera_provider_parent_class; + + delete self->cm; + + return G_OBJECT_GET_CLASS(klass)->finalize(object); +} + +static void +gst_libcamera_provider_class_init(GstLibcameraProviderClass *klass) +{ + GstDeviceProviderClass *provider_class = GST_DEVICE_PROVIDER_CLASS(klass); + GObjectClass *object_class = G_OBJECT_CLASS(klass); + + provider_class->probe = gst_libcamera_provider_probe; + object_class->finalize = gst_libcamera_provider_finalize; + + gst_device_provider_class_set_metadata(provider_class, + "libcamera Device Provider", + "Source/Video", + "List camera device using libcamera", + "Nicolas Dufresne "); +} diff --git a/src/gstreamer/gstlibcameraprovider.h b/src/gstreamer/gstlibcameraprovider.h new file mode 100644 index 0000000..bdd19db --- /dev/null +++ b/src/gstreamer/gstlibcameraprovider.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraprovider.h - GStreamer Device Provider + */ + +#ifndef __GST_LIBCAMERA_PROVIDER_H__ +#define __GST_LIBCAMERA_PROVIDER_H__ + +#include + +G_BEGIN_DECLS + +#define GST_TYPE_LIBCAMERA_PROVIDER gst_libcamera_provider_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraProvider, gst_libcamera_provider, + GST_LIBCAMERA, PROVIDER, GstDeviceProvider) + +G_END_DECLS + +#endif /* __GST_LIBCAMERA_PROVIDER_H__ */ + diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index 1539d9e..12c8ae1 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,6 +1,7 @@ libcamera_gst_sources = [ 'gstlibcamera-utils.cpp', 'gstlibcamera.c', + 'gstlibcameraprovider.cpp', 'gstlibcamerasrc.cpp', ] From patchwork Fri Mar 6 20:26:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3010 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE31C60424 for ; Fri, 6 Mar 2020 21:27:01 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 26CF4297189; Fri, 6 Mar 2020 20:27:01 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:14 -0500 Message-Id: <20200306202637.525587-5-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 04/27] gst: utils: Add simple scoped lockers for GMutex and GRectMutex X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:02 -0000 From: Nicolas Dufresne While GLib has locker implementation already using g_autoptr(), recursive mutex locker was only introduced in recent GLib. Implement a simple locker for GMutex and GRectMutex in order to allow making locking simpler and safer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.h | 50 ++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 2e4e304..562ffe9 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -16,4 +16,54 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); +/** + * \class GLibLocker + * \brief A simple scoped mutex locker for GMutex + */ +class GLibLocker +{ +public: + GLibLocker(GMutex *mutex) + : mutex_(mutex) + { + g_mutex_lock(mutex_); + } + + GLibLocker(GstObject *object) + : mutex_(GST_OBJECT_GET_LOCK(object)) + { + g_mutex_lock(mutex_); + } + + ~GLibLocker() + { + g_mutex_unlock(mutex_); + } + +private: + GMutex *mutex_; +}; + +/** + * \class GLibRecLocker + * \brief A simple scoped mutex locker for GRecMutex + */ +class GLibRecLocker +{ +public: + GLibRecLocker(GRecMutex *mutex) + : mutex_(mutex) + { + g_rec_mutex_lock(mutex_); + } + + ~GLibRecLocker() + { + g_rec_mutex_unlock(mutex_); + } + +private: + GRecMutex *mutex_; +}; + #endif /* __GST_LIBCAMERA_UTILS_H__ */ From patchwork Fri Mar 6 20:26:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3011 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 47DE560424 for ; Fri, 6 Mar 2020 21:27:02 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id ABE0C2970E7; Fri, 6 Mar 2020 20:27:01 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:15 -0500 Message-Id: <20200306202637.525587-6-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 05/27] gst: Add pads to the source X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:02 -0000 From: Nicolas Dufresne This simply adds the boiler plate for pads on the source element. The design is that we have one pad, called "src", that will always be present, and then more pads can be requested prior in READY or less state. Initially pads have one property "stream-role" that let you decide which role this pad will have. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 102 ++++++++++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 19 ++++++ src/gstreamer/gstlibcamerasrc.cpp | 20 ++++++ src/gstreamer/meson.build | 1 + 4 files changed, 142 insertions(+) create mode 100644 src/gstreamer/gstlibcamerapad.cpp create mode 100644 src/gstreamer/gstlibcamerapad.h diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp new file mode 100644 index 0000000..ca5c820 --- /dev/null +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapad.cpp - GStreamer Capture Pad + */ + +#include "gstlibcamerapad.h" + +#include + +#include "gstlibcamera-utils.h" + +using namespace libcamera; + +struct _GstLibcameraPad { + GstPad parent; + StreamRole role; +}; + +enum { + PROP_0, + PROP_STREAM_ROLE +}; + +G_DEFINE_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_TYPE_PAD); + +static void +gst_libcamera_pad_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + auto *self = GST_LIBCAMERA_PAD(object); + GLibLocker lock(GST_OBJECT(self)); + + switch (prop_id) { + case PROP_STREAM_ROLE: + self->role = (StreamRole)g_value_get_enum(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_pad_get_property(GObject *object, guint prop_id, GValue *value, + GParamSpec *pspec) +{ + auto *self = GST_LIBCAMERA_PAD(object); + GLibLocker lock(GST_OBJECT(self)); + + switch (prop_id) { + case PROP_STREAM_ROLE: + g_value_set_enum(value, self->role); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_pad_init(GstLibcameraPad *self) +{ +} + +static GType +gst_libcamera_stream_role_get_type(void) +{ + static GType type = 0; + static const GEnumValue values[] = { + { StillCapture, "libcamera::StillCapture", "still-capture" }, + { VideoRecording, "libcamera::VideoRecording", "video-recording" }, + { Viewfinder, "libcamera::Viewfinder", "view-finder" }, + { 0, NULL, NULL } + }; + + if (!type) + type = g_enum_register_static("GstLibcameraStreamRole", values); + + return type; +} + +static void +gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) +{ + auto *object_class = G_OBJECT_CLASS(klass); + + object_class->set_property = gst_libcamera_pad_set_property; + object_class->get_property = gst_libcamera_pad_get_property; + + auto *spec = g_param_spec_enum("stream-role", "Stream Role", + "The selected stream role", + gst_libcamera_stream_role_get_type(), + VideoRecording, + (GParamFlags)(GST_PARAM_MUTABLE_READY + | G_PARAM_CONSTRUCT + | G_PARAM_READWRITE + | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_STREAM_ROLE, spec); +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h new file mode 100644 index 0000000..2e745f1 --- /dev/null +++ b/src/gstreamer/gstlibcamerapad.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2019, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapad.h - GStreamer Capture Element + */ + +#include + +#ifndef __GST_LIBCAMERA_PAD_H__ +#define __GST_LIBCAMERA_PAD_H__ + +#define GST_TYPE_LIBCAMERA_PAD gst_libcamera_pad_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, + GST_LIBCAMERA, PAD, GstPad) + + +#endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 3807503..79d4196 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -8,12 +8,26 @@ #include "gstlibcamerasrc.h" +#include "gstlibcamerapad.h" + struct _GstLibcameraSrc { GstElement parent; }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); +#define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw; image/jpeg") + +/* For the simple case, we have a src pad that is always present. */ +GstStaticPadTemplate src_template = { + "src", GST_PAD_SRC, GST_PAD_ALWAYS, TEMPLATE_CAPS +}; + +/* More pads can be requested in state < PAUSED */ +GstStaticPadTemplate request_src_template = { + "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS +}; + static void gst_libcamera_src_init(GstLibcameraSrc *self) { @@ -28,4 +42,10 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) "libcamera Source", "Source/Video", "Linux Camera source using libcamera", "Nicolas Dufresne X-Patchwork-Id: 3012 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CB70262899 for ; Fri, 6 Mar 2020 21:27:02 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 3C4DD297189; Fri, 6 Mar 2020 20:27:02 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:16 -0500 Message-Id: <20200306202637.525587-7-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 06/27] gst: libcamerasrc: Allocate and add static pad X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:03 -0000 From: Nicolas Dufresne This pad will always be present and will allow simple pipeline to be used to stream from the camera. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 79d4196..cba936c 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -12,6 +12,7 @@ struct _GstLibcameraSrc { GstElement parent; + GstPad *srcpad; }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); @@ -31,6 +32,10 @@ GstStaticPadTemplate request_src_template = { static void gst_libcamera_src_init(GstLibcameraSrc *self) { + GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); + + self->srcpad = gst_pad_new_from_template(templ, "src"); + gst_element_add_pad(GST_ELEMENT(self), self->srcpad); } static void From patchwork Fri Mar 6 20:26:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3013 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 84DF560424 for ; Fri, 6 Mar 2020 21:27:03 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id C15A42970E7; Fri, 6 Mar 2020 20:27:02 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:17 -0500 Message-Id: <20200306202637.525587-8-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 07/27] gst: libcamerasrc: Add camera-name property X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:03 -0000 From: Nicolas Dufresne This property will be used to select by name the camera to use. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 68 ++++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index cba936c..e0c1c4e 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -9,10 +9,17 @@ #include "gstlibcamerasrc.h" #include "gstlibcamerapad.h" +#include "gstlibcamera-utils.h" struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; + gchar *camera_name; +}; + +enum { + PROP_0, + PROP_CAMERA_NAME }; G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); @@ -29,6 +36,52 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +static void +gst_libcamera_src_set_property(GObject *object, guint prop_id, + const GValue *value, GParamSpec *pspec) +{ + GLibLocker lock(GST_OBJECT(object)); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + switch (prop_id) { + case PROP_CAMERA_NAME: + g_free(self->camera_name); + self->camera_name = g_value_dup_string(value); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, + GParamSpec *pspec) +{ + GLibLocker lock(GST_OBJECT(object)); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + switch (prop_id) { + case PROP_CAMERA_NAME: + g_value_set_string(value, self->camera_name); + break; + default: + G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); + break; + } +} + +static void +gst_libcamera_src_finalize(GObject *object) +{ + GObjectClass *klass = G_OBJECT_CLASS(gst_libcamera_src_parent_class); + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + + g_free(self->camera_name); + + return klass->finalize(object); +} + static void gst_libcamera_src_init(GstLibcameraSrc *self) { @@ -41,7 +94,12 @@ gst_libcamera_src_init(GstLibcameraSrc *self) static void gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) { - GstElementClass *element_class = (GstElementClass *)klass; + GstElementClass *element_class = GST_ELEMENT_CLASS(klass); + GObjectClass *object_class = G_OBJECT_CLASS(klass); + + object_class->set_property = gst_libcamera_src_set_property; + object_class->get_property = gst_libcamera_src_get_property; + object_class->finalize = gst_libcamera_src_finalize; gst_element_class_set_metadata(element_class, "libcamera Source", "Source/Video", @@ -53,4 +111,12 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) gst_element_class_add_static_pad_template_with_gtype(element_class, &request_src_template, GST_TYPE_LIBCAMERA_PAD); + + GParamSpec *spec = g_param_spec_string("camera-name", "Camera Name", + "Select by name which camera to use.", nullptr, + (GParamFlags)(GST_PARAM_MUTABLE_READY + | G_PARAM_CONSTRUCT + | G_PARAM_READWRITE + | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_CAMERA_NAME, spec); } From patchwork Fri Mar 6 20:26:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3014 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D7631628CC for ; Fri, 6 Mar 2020 21:27:03 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 51F63297189; Fri, 6 Mar 2020 20:27:03 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:18 -0500 Message-Id: <20200306202637.525587-9-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 08/27] gst: libcamerasrc: Add a debug category X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:04 -0000 From: Nicolas Dufresne This will allow selecting libcamerasrc traces with the following environment: GST_DEBUG=libcamerasrc:7 Or all libcamera GStreamer element traces using GST_DEBUG="libcamera*:7" Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index e0c1c4e..3e13a00 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -11,6 +11,9 @@ #include "gstlibcamerapad.h" #include "gstlibcamera-utils.h" +GST_DEBUG_CATEGORY_STATIC(source_debug); +#define GST_CAT_DEFAULT source_debug + struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; @@ -22,7 +25,9 @@ enum { PROP_CAMERA_NAME }; -G_DEFINE_TYPE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT); +G_DEFINE_TYPE_WITH_CODE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT, + GST_DEBUG_CATEGORY_INIT(source_debug, "libcamerasrc", 0, + "libcamera Source")); #define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw; image/jpeg") From patchwork Fri Mar 6 20:26:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3015 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6914C628CC for ; Fri, 6 Mar 2020 21:27:04 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id D6D432972F1; Fri, 6 Mar 2020 20:27:03 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:19 -0500 Message-Id: <20200306202637.525587-10-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 09/27] gst: libcamerasrc: Implement selection and acquisition X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:04 -0000 From: Nicolas Dufresne This adds code to select and acquire a camera. With this, it is now possible to run a pipeline like: gst-launch-1.0 libcamerasrc ! fakesink Though no buffer will be streamed yet. In this function, we implement the change_state() virtual method to trigger actions on specific state transitions. Note that we also return GST_STATE_CHANGE_NO_PREROLL in GST_STATE_CHANGE_READY_TO_PAUSED and GST_STATE_CHANGE_PLAYING_TO_PAUSED transitions as this is required for all live sources. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 125 ++++++++++++++++++++++++++++++ 1 file changed, 125 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 3e13a00..fd0eaf5 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -8,16 +8,29 @@ #include "gstlibcamerasrc.h" +#include +#include + #include "gstlibcamerapad.h" #include "gstlibcamera-utils.h" +using namespace libcamera; + GST_DEBUG_CATEGORY_STATIC(source_debug); #define GST_CAT_DEFAULT source_debug +/* Used for C++ object with destructors. */ +struct GstLibcameraSrcState { + std::unique_ptr cm_; + std::shared_ptr cam_; +}; + struct _GstLibcameraSrc { GstElement parent; GstPad *srcpad; gchar *camera_name; + + GstLibcameraSrcState *state; }; enum { @@ -41,6 +54,83 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +static bool +gst_libcamera_src_open(GstLibcameraSrc *self) +{ + std::unique_ptr cm = std::make_unique(); + std::shared_ptr cam; + gint ret = 0; + + GST_DEBUG_OBJECT(self, "Opening camera device ..."); + + ret = cm->start(); + if (ret) { + GST_ELEMENT_ERROR(self, LIBRARY, INIT, + ("Failed listing cameras."), + ("libcamera::CameraMananger::start() failed: %s", g_strerror(-ret))); + return false; + } + + g_autofree gchar *camera_name = nullptr; + { + GLibLocker lock(GST_OBJECT(self)); + if (self->camera_name) + camera_name = g_strdup(self->camera_name); + } + + if (camera_name) { + cam = cm->get(self->camera_name); + if (!cam) { + GST_ELEMENT_ERROR(self, RESOURCE, NOT_FOUND, + ("Could not find a camera named '%s'.", self->camera_name), + ("libcamera::CameraMananger::get() returned nullptr")); + return false; + } + } else { + if (cm->cameras().empty()) { + GST_ELEMENT_ERROR(self, RESOURCE, NOT_FOUND, + ("Could not find any supported camera on this system."), + ("libcamera::CameraMananger::cameras() is empty")); + return false; + } + cam = cm->cameras()[0]; + } + + GST_INFO_OBJECT(self, "Using camera named '%s'", cam->name().c_str()); + + ret = cam->acquire(); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, BUSY, + ("Camera name '%s' is already in use.", cam->name().c_str()), + ("libcamera::Camera::acquire() failed: %s", g_strerror(ret))); + return false; + } + + /* No need to lock here, we didn't start our threads yet. */ + self->state->cm_ = std::move(cm); + self->state->cam_ = cam; + + return true; +} + +static void +gst_libcamera_src_close(GstLibcameraSrc *self) +{ + GstLibcameraSrcState *state = self->state; + gint ret; + + ret = state->cam_->release(); + if (ret) { + GST_ELEMENT_WARNING(self, RESOURCE, BUSY, + ("Camera name '%s' is still in use.", state->cam_->name().c_str()), + ("libcamera::Camera.release() failed: %s", g_strerror(-ret))); + } + + state->cam_.reset(); + state->cm_->stop(); + state->cm_.reset(); +} + static void gst_libcamera_src_set_property(GObject *object, guint prop_id, const GValue *value, GParamSpec *pspec) @@ -76,6 +166,36 @@ gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, } } +static GstStateChangeReturn +gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(element); + GstStateChangeReturn ret = GST_STATE_CHANGE_SUCCESS; + GstElementClass *klass = GST_ELEMENT_CLASS(gst_libcamera_src_parent_class); + + ret = klass->change_state(element, transition); + if (ret == GST_STATE_CHANGE_FAILURE) + return ret; + + switch (transition) { + case GST_STATE_CHANGE_NULL_TO_READY: + if (!gst_libcamera_src_open(self)) + return GST_STATE_CHANGE_FAILURE; + break; + case GST_STATE_CHANGE_READY_TO_PAUSED: + case GST_STATE_CHANGE_PLAYING_TO_PAUSED: + ret = GST_STATE_CHANGE_NO_PREROLL; + break; + case GST_STATE_CHANGE_READY_TO_NULL: + gst_libcamera_src_close(self); + break; + default: + break; + } + + return ret; +} + static void gst_libcamera_src_finalize(GObject *object) { @@ -83,6 +203,7 @@ gst_libcamera_src_finalize(GObject *object) GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); g_free(self->camera_name); + delete self->state; return klass->finalize(object); } @@ -90,10 +211,12 @@ gst_libcamera_src_finalize(GObject *object) static void gst_libcamera_src_init(GstLibcameraSrc *self) { + GstLibcameraSrcState *state = new GstLibcameraSrcState(); GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); self->srcpad = gst_pad_new_from_template(templ, "src"); gst_element_add_pad(GST_ELEMENT(self), self->srcpad); + self->state = state; } static void @@ -106,6 +229,8 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) object_class->get_property = gst_libcamera_src_get_property; object_class->finalize = gst_libcamera_src_finalize; + element_class->change_state = gst_libcamera_src_change_state; + gst_element_class_set_metadata(element_class, "libcamera Source", "Source/Video", "Linux Camera source using libcamera", From patchwork Fri Mar 6 20:26:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3016 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 05928628CC for ; Fri, 6 Mar 2020 21:27:05 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 676BD2970E7; Fri, 6 Mar 2020 20:27:04 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:20 -0500 Message-Id: <20200306202637.525587-11-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 10/27] gst: libcamerasrc: Add a task for the streaming thread X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:05 -0000 From: Nicolas Dufresne Use a GstTask as our internal streaming thread. Unlike GstBaseSrc, we will be running a streaming thread at the element level rather than per pad. This is needed to combine buffer request for multiple pads. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 51 +++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index fd0eaf5..35f7512 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -27,7 +27,11 @@ struct GstLibcameraSrcState { struct _GstLibcameraSrc { GstElement parent; + + GRecMutex stream_lock; + GstTask *task; GstPad *srcpad; + gchar *camera_name; GstLibcameraSrcState *state; @@ -113,6 +117,30 @@ gst_libcamera_src_open(GstLibcameraSrc *self) return true; } +static void +gst_libcamera_src_task_run(gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread is now capturing"); +} + +static void +gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread has started"); +} + +static void +gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) +{ + GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + + GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); +} + static void gst_libcamera_src_close(GstLibcameraSrc *self) { @@ -183,9 +211,24 @@ gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) return GST_STATE_CHANGE_FAILURE; break; case GST_STATE_CHANGE_READY_TO_PAUSED: + /* This needs to be called after pads activation.*/ + if (!gst_task_pause(self->task)) + return GST_STATE_CHANGE_FAILURE; + ret = GST_STATE_CHANGE_NO_PREROLL; + break; case GST_STATE_CHANGE_PLAYING_TO_PAUSED: ret = GST_STATE_CHANGE_NO_PREROLL; break; + case GST_STATE_CHANGE_PAUSED_TO_READY: + /* + * \todo this might require some thread unblocking in the future + * if the streaming thread starts doing any kind of blocking + * operations. If this was the case, we would need to do so + * before pad deactivation, so before chaining to the parent + * change_state function. + */ + gst_task_join(self->task); + break; case GST_STATE_CHANGE_READY_TO_NULL: gst_libcamera_src_close(self); break; @@ -202,6 +245,8 @@ gst_libcamera_src_finalize(GObject *object) GObjectClass *klass = G_OBJECT_CLASS(gst_libcamera_src_parent_class); GstLibcameraSrc *self = GST_LIBCAMERA_SRC(object); + g_rec_mutex_clear(&self->stream_lock); + g_clear_object(&self->task); g_free(self->camera_name); delete self->state; @@ -214,6 +259,12 @@ gst_libcamera_src_init(GstLibcameraSrc *self) GstLibcameraSrcState *state = new GstLibcameraSrcState(); GstPadTemplate *templ = gst_element_get_pad_template(GST_ELEMENT(self), "src"); + g_rec_mutex_init(&self->stream_lock); + self->task = gst_task_new(gst_libcamera_src_task_run, self, nullptr); + gst_task_set_enter_callback(self->task, gst_libcamera_src_task_enter, self, nullptr); + gst_task_set_leave_callback(self->task, gst_libcamera_src_task_leave, self, nullptr); + gst_task_set_lock(self->task, &self->stream_lock); + self->srcpad = gst_pad_new_from_template(templ, "src"); gst_element_add_pad(GST_ELEMENT(self), self->srcpad); self->state = state; From patchwork Fri Mar 6 20:26:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3017 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E8C3628CC for ; Fri, 6 Mar 2020 21:27:05 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id EC87C297190; Fri, 6 Mar 2020 20:27:04 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:21 -0500 Message-Id: <20200306202637.525587-12-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 11/27] gst: libcamerapad: Add a method to access the role X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:05 -0000 From: Nicolas Dufresne Each pad can have a different roles. Users will have to request and configure their pads role before moving to a higher state. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 8 ++++++++ src/gstreamer/gstlibcamerapad.h | 10 ++++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index ca5c820..ed24557 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -100,3 +100,11 @@ gst_libcamera_pad_class_init(GstLibcameraPadClass *klass) | G_PARAM_STATIC_STRINGS)); g_object_class_install_property(object_class, PROP_STREAM_ROLE, spec); } + +StreamRole +gst_libcamera_pad_get_role(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + return self->role; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 2e745f1..f0a5e32 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -6,14 +6,16 @@ * gstlibcamerapad.h - GStreamer Capture Element */ -#include - #ifndef __GST_LIBCAMERA_PAD_H__ #define __GST_LIBCAMERA_PAD_H__ +#include + +#include + #define GST_TYPE_LIBCAMERA_PAD gst_libcamera_pad_get_type() -G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, - GST_LIBCAMERA, PAD, GstPad) +G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_LIBCAMERA, PAD, GstPad) +libcamera::StreamRole gst_libcamera_pad_get_role(GstPad *pad); #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Fri Mar 6 20:26:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3018 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 28C1C62899 for ; Fri, 6 Mar 2020 21:27:06 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 7CD812970E7; Fri, 6 Mar 2020 20:27:05 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:22 -0500 Message-Id: <20200306202637.525587-13-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 12/27] gst: libcamerasrc: Store the srcpad in a vector X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:06 -0000 From: Nicolas Dufresne This will allow implementing generic algorithm even if we cannot request pads yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 35f7512..aa91619 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -8,6 +8,8 @@ #include "gstlibcamerasrc.h" +#include + #include #include @@ -23,6 +25,7 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); struct GstLibcameraSrcState { std::unique_ptr cm_; std::shared_ptr cam_; + std::vector srcpads_; }; struct _GstLibcameraSrc { @@ -30,7 +33,6 @@ struct _GstLibcameraSrc { GRecMutex stream_lock; GstTask *task; - GstPad *srcpad; gchar *camera_name; @@ -265,8 +267,8 @@ gst_libcamera_src_init(GstLibcameraSrc *self) gst_task_set_leave_callback(self->task, gst_libcamera_src_task_leave, self, nullptr); gst_task_set_lock(self->task, &self->stream_lock); - self->srcpad = gst_pad_new_from_template(templ, "src"); - gst_element_add_pad(GST_ELEMENT(self), self->srcpad); + state->srcpads_.push_back(gst_pad_new_from_template(templ, "src")); + gst_element_add_pad(GST_ELEMENT(self), state->srcpads_[0]); self->state = state; } From patchwork Fri Mar 6 20:26:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3019 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B67D162899 for ; Fri, 6 Mar 2020 21:27:06 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 26223297189; Fri, 6 Mar 2020 20:27:06 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:23 -0500 Message-Id: <20200306202637.525587-14-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 13/27] gst: libcamerasrc: Send stream start event X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:06 -0000 From: Nicolas Dufresne Prior to sending caps, we need to send a stream-start event. This requires generating a stream and a group id. The stream id is random for live sources and the group id is shared across all pads. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index aa91619..1680314 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -131,8 +131,19 @@ static void gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GLibRecLocker lock(&self->stream_lock); + GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread has started"); + + guint group_id = gst_util_group_id_next(); + for (GstPad *srcpad : state->srcpads_) { + /* Create stream-id and push stream-start. */ + g_autofree gchar *stream_id = gst_pad_create_stream_id(srcpad, GST_ELEMENT(self), nullptr); + GstEvent *event = gst_event_new_stream_start(stream_id); + gst_event_set_group_id(event, group_id); + gst_pad_push_event(srcpad, event); + } } static void From patchwork Fri Mar 6 20:26:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3020 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 462EA62899 for ; Fri, 6 Mar 2020 21:27:07 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id AA8D52970E7; Fri, 6 Mar 2020 20:27:06 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:24 -0500 Message-Id: <20200306202637.525587-15-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 14/27] gst: utils: Add StreamConfiguration helpers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:07 -0000 From: Nicolas Dufresne This adds helpers to deal with the conversion from StreamConfiguration to caps and vice-versa. This is needed to implement caps negotiation. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.cpp | 64 ++++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 3 ++ 2 files changed, 67 insertions(+) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 386ec36..482439a 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -42,6 +42,18 @@ drm_to_gst_format(guint drm_fourcc) return GST_VIDEO_FORMAT_UNKNOWN; } +static guint +gst_format_to_drm(GstVideoFormat gst_format) +{ + if (gst_format == GST_VIDEO_FORMAT_ENCODED) + return DRM_FORMAT_INVALID; + + for (const auto &item : format_map) + if (item.gst_format == gst_format) + return item.drm_fourcc; + return DRM_FORMAT_INVALID; +} + static GstStructure * bare_structure_from_fourcc(guint fourcc) { @@ -103,3 +115,55 @@ gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) return caps; } + +GstCaps * +gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg) +{ + GstCaps *caps = gst_caps_new_empty(); + GstStructure *s = bare_structure_from_fourcc(stream_cfg.pixelFormat); + + gst_structure_set(s, + "width", G_TYPE_INT, stream_cfg.size.width, + "height", G_TYPE_INT, stream_cfg.size.height, + nullptr); + gst_caps_append_structure(caps, s); + + return caps; +} + +void +gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, + GstCaps *caps) +{ + GstVideoFormat gst_format = drm_to_gst_format(stream_cfg.pixelFormat); + + /* First fixate the caps using default configuration value. */ + g_assert(gst_caps_is_writable(caps)); + caps = gst_caps_truncate(caps); + GstStructure *s = gst_caps_get_structure(caps, 0); + + gst_structure_fixate_field_nearest_int(s, "width", stream_cfg.size.width); + gst_structure_fixate_field_nearest_int(s, "height", stream_cfg.size.height); + + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_video_format_to_string(gst_format); + gst_structure_fixate_field_string(s, "format", format); + } + + /* Then configure the stream with the result. */ + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_structure_get_string(s, "format"); + gst_format = gst_video_format_from_string(format); + stream_cfg.pixelFormat = gst_format_to_drm(gst_format); + } else if (gst_structure_has_name(s, "image/jpeg")) { + stream_cfg.pixelFormat = DRM_FORMAT_MJPEG; + } else { + g_critical("Unsupported media type: %s", gst_structure_get_name(s)); + } + + gint width, height; + gst_structure_get_int(s, "width", &width); + gst_structure_get_int(s, "height", &height); + stream_cfg.size.width = width; + stream_cfg.size.height = height; +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 562ffe9..cbfb1cc 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -15,6 +15,9 @@ #include GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); +GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); +void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, + GstCaps *caps); /** * \class GLibLocker From patchwork Fri Mar 6 20:26:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3021 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA33D628D4 for ; Fri, 6 Mar 2020 21:27:07 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 3B12D297189; Fri, 6 Mar 2020 20:27:07 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 6 Mar 2020 15:26:25 -0500 Message-Id: <20200306202637.525587-16-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 15/27] gst: libcamerasrc: Implement minimal caps negotiation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:08 -0000 From: Nicolas Dufresne This is not expected to work in every possible cases, but should be sufficient as an initial implementation. What it does is that it turns the StreamFormats into caps and queries downstream caps with that as a filter. The result is the subset of caps that can be used. We then keep the first structure in that result and fixate using the default values found in StreamConfiguration as a default in case a range is available. We then validate this configuration and turn the potentially modified configuration into caps that we push downstream. Note that we trust the order in StreamFormats as being sorted best first, but this is not currently in libcamera. A todo has been added in the head of this file as a reminder to fix that in the core. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 81 +++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 1680314..95af141 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -6,6 +6,12 @@ * gstlibcamerasrc.cpp - GStreamer Capture Element */ +/** + * \todo libcamera UVC drivers picks the lowest possible resolution first, this + * should be fixed so that we get a decent resolution and framerate for the + * role by default. + */ + #include "gstlibcamerasrc.h" #include @@ -25,6 +31,7 @@ GST_DEBUG_CATEGORY_STATIC(source_debug); struct GstLibcameraSrcState { std::unique_ptr cm_; std::shared_ptr cam_; + std::unique_ptr config_; std::vector srcpads_; }; @@ -133,16 +140,90 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); GLibRecLocker lock(&self->stream_lock); GstLibcameraSrcState *state = self->state; + GstFlowReturn flow_ret = GST_FLOW_OK; + gint ret; GST_DEBUG_OBJECT(self, "Streaming thread has started"); guint group_id = gst_util_group_id_next(); + StreamRoles roles; for (GstPad *srcpad : state->srcpads_) { /* Create stream-id and push stream-start. */ g_autofree gchar *stream_id = gst_pad_create_stream_id(srcpad, GST_ELEMENT(self), nullptr); GstEvent *event = gst_event_new_stream_start(stream_id); gst_event_set_group_id(event, group_id); gst_pad_push_event(srcpad, event); + + /* Collect the streams roles for the next iteration. */ + roles.push_back(gst_libcamera_pad_get_role(srcpad)); + } + + /* Generate the stream configurations, there should be one per pad. */ + state->config_ = state->cam_->generateConfiguration(roles); + /* + * \todo Check if camera may increase or decrease the number of streams + * regardless of the number of roles. + */ + g_assert(state->config_->size() == state->srcpads_.size()); + + for (gsize i = 0; i < state->srcpads_.size(); i++) { + GstPad *srcpad = state->srcpads_[i]; + StreamConfiguration &stream_cfg = state->config_->at(i); + + /* Retrieve the supported caps. */ + g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); + g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad, filter); + if (gst_caps_is_empty(caps)) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + break; + } + + /* Fixate caps and configure the stream. */ + caps = gst_caps_make_writable(caps); + gst_libcamera_configure_stream_from_caps(stream_cfg, caps); + } + + if (flow_ret != GST_FLOW_OK) + goto done; + + /* Validate the configuration. */ + if (state->config_->validate() == CameraConfiguration::Invalid) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + goto done; + } + + /* + * Regardless if it has been modified, create clean caps and push the + * caps event. Downstream will decide if the caps are acceptable. + */ + for (gsize i = 0; i < state->srcpads_.size(); i++) { + GstPad *srcpad = state->srcpads_[i]; + const StreamConfiguration &stream_cfg = state->config_->at(i); + + g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg); + if (!gst_pad_push_event(srcpad, gst_event_new_caps(caps))) { + flow_ret = GST_FLOW_NOT_NEGOTIATED; + break; + } + } + + ret = state->cam_->configure(state->config_.get()); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Failed to configure camera: %s", g_strerror(-ret)), + ("Camera::configure() failed with error code %i", ret)); + gst_task_stop(task); + return; + } + +done: + switch (flow_ret) { + case GST_FLOW_NOT_NEGOTIATED: + GST_ELEMENT_FLOW_ERROR(self, flow_ret); + gst_task_stop(task); + break; + default: + break; } } From patchwork Fri Mar 6 20:26:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3022 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4ABF862913 for ; Fri, 6 Mar 2020 21:27:08 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id AD20F2970E7; Fri, 6 Mar 2020 20:27:07 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:26 -0500 Message-Id: <20200306202637.525587-17-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 16/27] gst: libcamerasrc: Push segment event X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:09 -0000 From: Nicolas Dufresne Now that we have stream-start and caps, we can now push a segment event to announce what time will our buffer correlate to. For live sources this is just an open segment in time format. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 95af141..540a094 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -205,6 +205,11 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) flow_ret = GST_FLOW_NOT_NEGOTIATED; break; } + + /* Send an open segment event with time format. */ + GstSegment segment; + gst_segment_init(&segment, GST_FORMAT_TIME); + gst_pad_push_event(srcpad, gst_event_new_segment(&segment)); } ret = state->cam_->configure(state->config_.get()); From patchwork Fri Mar 6 20:26:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3023 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6562628D7 for ; Fri, 6 Mar 2020 21:27:08 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 3E641297189; Fri, 6 Mar 2020 20:27:08 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 6 Mar 2020 15:26:27 -0500 Message-Id: <20200306202637.525587-18-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 17/27] gst: Add a pool and an allocator implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:09 -0000 From: Nicolas Dufresne This is needed to track the lifetime of the FrameBufferAllocator in relation to the GstBuffer/GstMemory objects travelling inside GStreamer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcameraallocator.cpp | 245 ++++++++++++++++++++++++ src/gstreamer/gstlibcameraallocator.h | 30 +++ src/gstreamer/gstlibcamerapool.cpp | 110 +++++++++++ src/gstreamer/gstlibcamerapool.h | 27 +++ src/gstreamer/meson.build | 13 +- 5 files changed, 421 insertions(+), 4 deletions(-) create mode 100644 src/gstreamer/gstlibcameraallocator.cpp create mode 100644 src/gstreamer/gstlibcameraallocator.h create mode 100644 src/gstreamer/gstlibcamerapool.cpp create mode 100644 src/gstreamer/gstlibcamerapool.h diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp new file mode 100644 index 0000000..861af59 --- /dev/null +++ b/src/gstreamer/gstlibcameraallocator.cpp @@ -0,0 +1,245 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraallocator.cpp - GStreamer Custom Allocator + */ + +#include "gstlibcameraallocator.h" + +#include +#include +#include + +#include "gstlibcamera-utils.h" + +using namespace libcamera; + +static gboolean gst_libcamera_allocator_release(GstMiniObject *mini_object); + +/** + * \struct FrameWrap + * \brief An internal wrapper to track the relation between FrameBuffer and + * GstMemory(s) + * + * This wrapper maintains a count of the outstanding GstMemory (there may be + * multiple GstMemory per FrameBuffer), and give back the FrameBuffer to the + * allocator pool when all memory objects have returned. + */ + +struct FrameWrap { + FrameWrap(GstAllocator *allocator, FrameBuffer *buffer, + gpointer stream); + ~FrameWrap(); + + void acquirePlane() { ++outstandingPlanes_; } + bool releasePlane() { return --outstandingPlanes_ == 0; } + + static GQuark getQuark(); + + gpointer stream_; + FrameBuffer *buffer_; + std::vector planes_; + gint outstandingPlanes_; +}; + +FrameWrap::FrameWrap(GstAllocator *allocator, FrameBuffer *buffer, + gpointer stream) + + : stream_(stream), + buffer_(buffer), + outstandingPlanes_(0) +{ + for (const FrameBuffer::Plane &plane : buffer->planes()) { + GstMemory *mem = gst_fd_allocator_alloc(allocator, plane.fd.fd(), plane.length, + GST_FD_MEMORY_FLAG_DONT_CLOSE); + gst_mini_object_set_qdata(GST_MINI_OBJECT(mem), getQuark(), this, nullptr); + GST_MINI_OBJECT(mem)->dispose = gst_libcamera_allocator_release; + g_object_unref(mem->allocator); + planes_.push_back(mem); + } +} + +FrameWrap::~FrameWrap() +{ + for (GstMemory *mem : planes_) { + GST_MINI_OBJECT(mem)->dispose = nullptr; + g_object_ref(mem->allocator); + gst_memory_unref(mem); + } +} + +GQuark FrameWrap::getQuark(void) +{ + static gsize frame_quark = 0; + + if (g_once_init_enter(&frame_quark)) { + GQuark quark = g_quark_from_string("GstLibcameraFrameWrap"); + g_once_init_leave(&frame_quark, quark); + } + + return frame_quark; +} + +/** + * \struct _GstLibcameraAllocator + * \brief A pooling GstDmaBufAllocator for libcamera + * + * This is a pooling GstDmaBufAllocator implementation. This implementation override + * the dispose function of memory object in order to keep them alive when they + * are disposed by downstream elements. + */ +struct _GstLibcameraAllocator { + GstDmaBufAllocator parent; + FrameBufferAllocator *fb_allocator; + /* + * A hash table using Stream pointer as key and returning a GQueue of + * FrameWrap. + */ + GHashTable *pools; +}; + +G_DEFINE_TYPE(GstLibcameraAllocator, gst_libcamera_allocator, + GST_TYPE_DMABUF_ALLOCATOR); + +static gboolean +gst_libcamera_allocator_release(GstMiniObject *mini_object) +{ + GstMemory *mem = GST_MEMORY_CAST(mini_object); + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(mem->allocator); + GLibLocker lock(GST_OBJECT(self)); + auto *frame = reinterpret_cast(gst_mini_object_get_qdata(mini_object, FrameWrap::getQuark())); + + gst_memory_ref(mem); + + if (frame->releasePlane()) { + auto *pool = reinterpret_cast(g_hash_table_lookup(self->pools, frame->stream_)); + g_return_val_if_fail(pool, TRUE); + g_queue_push_tail(pool, frame); + } + + /* Keep last in case we are holding on the last allocator ref. */ + g_object_unref(mem->allocator); + + /* Return FALSE so that our mini object isn't freed. */ + return FALSE; +} + +static void +gst_libcamera_allocator_free_pool(gpointer data) +{ + GQueue *queue = reinterpret_cast(data); + FrameWrap *frame; + + while ((frame = reinterpret_cast(g_queue_pop_head(queue)))) { + g_warn_if_fail(frame->outstandingPlanes_ == 0); + delete frame; + } + + g_queue_free(queue); +} + +static void +gst_libcamera_allocator_init(GstLibcameraAllocator *self) +{ + self->pools = g_hash_table_new_full(nullptr, nullptr, nullptr, + gst_libcamera_allocator_free_pool); + GST_OBJECT_FLAG_SET(self, GST_ALLOCATOR_FLAG_CUSTOM_ALLOC); +} + +static void +gst_libcamera_allocator_dispose(GObject *object) +{ + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object); + + if (self->pools) { + g_hash_table_unref(self->pools); + self->pools = nullptr; + } + + G_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->dispose(object); +} + +static void +gst_libcamera_allocator_finalize(GObject *object) +{ + GstLibcameraAllocator *self = GST_LIBCAMERA_ALLOCATOR(object); + + delete self->fb_allocator; + + G_OBJECT_CLASS(gst_libcamera_allocator_parent_class)->finalize(object); +} + +static void +gst_libcamera_allocator_class_init(GstLibcameraAllocatorClass *klass) +{ + auto *allocator_class = GST_ALLOCATOR_CLASS(klass); + auto *object_class = G_OBJECT_CLASS(klass); + + object_class->dispose = gst_libcamera_allocator_dispose; + object_class->finalize = gst_libcamera_allocator_finalize; + allocator_class->alloc = nullptr; +} + +GstLibcameraAllocator * +gst_libcamera_allocator_new(std::shared_ptr camera) +{ + auto *self = GST_LIBCAMERA_ALLOCATOR(g_object_new(GST_TYPE_LIBCAMERA_ALLOCATOR, + nullptr)); + + self->fb_allocator = FrameBufferAllocator::create(camera); + for (Stream *stream : camera->streams()) { + gint ret; + + ret = self->fb_allocator->allocate(stream); + if (ret == 0) + return nullptr; + + GQueue *pool = g_queue_new(); + for (const std::unique_ptr &buffer : + self->fb_allocator->buffers(stream)) { + auto *fb = new FrameWrap(GST_ALLOCATOR(self), + buffer.get(), stream); + g_queue_push_tail(pool, fb); + } + + g_hash_table_insert(self->pools, stream, pool); + } + + return self; +} + +bool +gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, + Stream *stream, GstBuffer *buffer) +{ + GLibLocker lock(GST_OBJECT(self)); + + auto *pool = reinterpret_cast(g_hash_table_lookup(self->pools, stream)); + g_return_val_if_fail(pool, false); + + auto *frame = reinterpret_cast(g_queue_pop_head(pool)); + if (!frame) + return false; + + for (GstMemory *mem : frame->planes_) { + frame->acquirePlane(); + gst_buffer_append_memory(buffer, mem); + g_object_ref(mem->allocator); + } + + return true; +} + +gsize +gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self, + Stream *stream) +{ + GLibLocker lock(GST_OBJECT(self)); + + auto *pool = reinterpret_cast(g_hash_table_lookup(self->pools, stream)); + g_return_val_if_fail(pool, false); + + return pool->length; +} diff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h new file mode 100644 index 0000000..b3f7f36 --- /dev/null +++ b/src/gstreamer/gstlibcameraallocator.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcameraallocator.h - GStreamer Custom Allocator + */ + +#ifndef __GST_LIBCAMERA_ALLOCATOR_H__ +#define __GST_LIBCAMERA_ALLOCATOR_H__ + +#include +#include + +#include + +#define GST_TYPE_LIBCAMERA_ALLOCATOR gst_libcamera_allocator_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraAllocator, gst_libcamera_allocator, + GST_LIBCAMERA, ALLOCATOR, GstDmaBufAllocator) + +GstLibcameraAllocator *gst_libcamera_allocator_new(std::shared_ptr camera); + +bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, + libcamera::Stream *stream, + GstBuffer *buffer); + +gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator, + libcamera::Stream *stream); + +#endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */ diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp new file mode 100644 index 0000000..75e189a --- /dev/null +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapool.cpp - GStreamer Buffer Pool + */ + +#include "gstlibcamerapool.h" + +#include + +#include "gstlibcamera-utils.h" + +using namespace libcamera; + +struct _GstLibcameraPool { + GstBufferPool parent; + + GstAtomicQueue *queue; + GstLibcameraAllocator *allocator; + Stream *stream; +}; + +G_DEFINE_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_TYPE_BUFFER_POOL); + +static GstFlowReturn +gst_libcamera_pool_acquire_buffer(GstBufferPool *pool, GstBuffer **buffer, + GstBufferPoolAcquireParams *params) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + GstBuffer *buf = GST_BUFFER(gst_atomic_queue_pop(self->queue)); + if (!buf) + return GST_FLOW_ERROR; + + if (!gst_libcamera_allocator_prepare_buffer(self->allocator, self->stream, buf)) + return GST_FLOW_ERROR; + + *buffer = buf; + return GST_FLOW_OK; +} + +static void +gst_libcamera_pool_reset_buffer(GstBufferPool *pool, GstBuffer *buffer) +{ + GstBufferPoolClass *klass = GST_BUFFER_POOL_CLASS(gst_libcamera_pool_parent_class); + + /* Clears all the memories and only pool the GstBuffer objects */ + gst_buffer_remove_all_memory(buffer); + klass->reset_buffer(pool, buffer); + GST_BUFFER_FLAGS(buffer) = 0; +} + +static void +gst_libcamera_pool_release_buffer(GstBufferPool *pool, GstBuffer *buffer) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + gst_atomic_queue_push(self->queue, buffer); +} + +static void +gst_libcamera_pool_init(GstLibcameraPool *self) +{ + self->queue = gst_atomic_queue_new(4); +} + +static void +gst_libcamera_pool_finalize(GObject *object) +{ + GstLibcameraPool *self = GST_LIBCAMERA_POOL(object); + GstBuffer *buf; + + while ((buf = GST_BUFFER(gst_atomic_queue_pop(self->queue)))) + gst_buffer_unref(buf); + + gst_atomic_queue_unref(self->queue); + g_object_unref(self->allocator); + + G_OBJECT_CLASS(gst_libcamera_pool_parent_class)->finalize(object); +} + +static void +gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass) +{ + auto *object_class = G_OBJECT_CLASS(klass); + auto *pool_class = GST_BUFFER_POOL_CLASS(klass); + + object_class->finalize = gst_libcamera_pool_finalize; + pool_class->start = nullptr; + pool_class->acquire_buffer = gst_libcamera_pool_acquire_buffer; + pool_class->reset_buffer = gst_libcamera_pool_reset_buffer; + pool_class->release_buffer = gst_libcamera_pool_release_buffer; +} + +GstLibcameraPool * +gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) +{ + auto *pool = GST_LIBCAMERA_POOL(g_object_new(GST_TYPE_LIBCAMERA_POOL, nullptr)); + + pool->allocator = GST_LIBCAMERA_ALLOCATOR(g_object_ref(allocator)); + pool->stream = stream; + + gsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream); + for (gsize i = 0; i < pool_size; i++) { + GstBuffer *buffer = gst_buffer_new(); + gst_atomic_queue_push(pool->queue, buffer); + } + + return pool; +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h new file mode 100644 index 0000000..c0a04cf --- /dev/null +++ b/src/gstreamer/gstlibcamerapool.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Collabora Ltd. + * Author: Nicolas Dufresne + * + * gstlibcamerapool.h - GStreamer Buffer Pool + * + * This is a partial implementation of GstBufferPool intended for internal use + * only. This pool cannot be configured or activated. + */ + +#ifndef __GST_LIBCAMERA_POOL_H__ +#define __GST_LIBCAMERA_POOL_H__ + +#include "gstlibcameraallocator.h" + +#include + +#include + +#define GST_TYPE_LIBCAMERA_POOL gst_libcamera_pool_get_type() +G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstBufferPool) + +GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, + libcamera::Stream *stream); + +#endif /* __GST_LIBCAMERA_POOL_H__ */ diff --git a/src/gstreamer/meson.build b/src/gstreamer/meson.build index c396710..ca64bf1 100644 --- a/src/gstreamer/meson.build +++ b/src/gstreamer/meson.build @@ -1,7 +1,9 @@ libcamera_gst_sources = [ 'gstlibcamera-utils.cpp', 'gstlibcamera.c', + 'gstlibcameraallocator.cpp', 'gstlibcamerapad.cpp', + 'gstlibcamerapool.cpp', 'gstlibcameraprovider.cpp', 'gstlibcamerasrc.cpp', ] @@ -11,14 +13,17 @@ libcamera_gst_c_args = [ '-DPACKAGE="@0@"'.format(meson.project_name()), ] -gst_dep = dependency('gstreamer-video-1.0', version : '>=1.14.0', - required : get_option('gstreamer')) +gst_dep_version = '>=1.14.0' +gstvideo_dep = dependency('gstreamer-video-1.0', version : gst_dep_version, + required : get_option('gstreamer')) +gstallocator_dep = dependency('gstreamer-allocators-1.0', version : gst_dep_version, + required : get_option('gstreamer')) -if gst_dep.found() +if gstvideo_dep.found() and gstallocator_dep.found() libcamera_gst = shared_library('gstlibcamera', libcamera_gst_sources, c_args : libcamera_gst_c_args, - dependencies : [libcamera_dep, gst_dep], + dependencies : [libcamera_dep, gstvideo_dep, gstallocator_dep], install: true, install_dir : '@0@/gstreamer-1.0'.format(get_option('libdir')), ) From patchwork Fri Mar 6 20:26:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3024 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56D1A628EC for ; Fri, 6 Mar 2020 21:27:09 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id C2BCE2972F3; Fri, 6 Mar 2020 20:27:08 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:28 -0500 Message-Id: <20200306202637.525587-19-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 18/27] gst: libcamerapad: Allow storing a pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:10 -0000 From: Nicolas Dufresne This adds get/set helper to store a pool on the pad. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 18 ++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 6 ++++++ 2 files changed, 24 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index ed24557..dbe3361 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -17,6 +17,7 @@ using namespace libcamera; struct _GstLibcameraPad { GstPad parent; StreamRole role; + GstLibcameraPool *pool; }; enum { @@ -108,3 +109,20 @@ gst_libcamera_pad_get_role(GstPad *pad) GLibLocker lock(GST_OBJECT(self)); return self->role; } + +GstLibcameraPool * +gst_libcamera_pad_get_pool(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + return self->pool; +} + +void +gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (self->pool) + g_object_unref(self->pool); + self->pool = pool; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index f0a5e32..b03f47a 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -9,6 +9,8 @@ #ifndef __GST_LIBCAMERA_PAD_H__ #define __GST_LIBCAMERA_PAD_H__ +#include "gstlibcamerapool.h" + #include #include @@ -18,4 +20,8 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPad, gst_libcamera_pad, GST_LIBCAMERA, PAD, Gst libcamera::StreamRole gst_libcamera_pad_get_role(GstPad *pad); +GstLibcameraPool *gst_libcamera_pad_get_pool(GstPad *pad); + +void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); + #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Fri Mar 6 20:26:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3025 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D0C83628F2 for ; Fri, 6 Mar 2020 21:27:09 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 53773297189; Fri, 6 Mar 2020 20:27:09 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 6 Mar 2020 15:26:29 -0500 Message-Id: <20200306202637.525587-20-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 19/27] gst: libcamerasrc: Allocate and release buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:10 -0000 From: Nicolas Dufresne Setup the allocation and the release of buffers in the element. We have one pooling GstAllocator that wraps the FrameBufferAllocator and tracks the lifetime of FrameBuffer objects. Then, for each pad we have a GstBufferPool object which is only used to avoid re-allocating the GstBuffer structure every time we push a buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 540a094..5ffc004 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -19,7 +19,9 @@ #include #include +#include "gstlibcameraallocator.h" #include "gstlibcamerapad.h" +#include "gstlibcamerapool.h" #include "gstlibcamera-utils.h" using namespace libcamera; @@ -44,6 +46,7 @@ struct _GstLibcameraSrc { gchar *camera_name; GstLibcameraSrcState *state; + GstLibcameraAllocator *allocator; }; enum { @@ -221,6 +224,23 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) return; } + self->allocator = gst_libcamera_allocator_new(state->cam_); + if (!self->allocator) { + GST_ELEMENT_ERROR(self, RESOURCE, NO_SPACE_LEFT, + ("Failed to allocate memory"), + ("gst_libcamera_allocator_new() failed.")); + gst_task_stop(task); + return; + } + + for (gsize i = 0; i < state->srcpads_.size(); i++) { + GstPad *srcpad = state->srcpads_[i]; + const StreamConfiguration &stream_cfg = state->config_->at(i); + GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, + stream_cfg.stream()); + gst_libcamera_pad_set_pool(srcpad, pool); + } + done: switch (flow_ret) { case GST_FLOW_NOT_NEGOTIATED: @@ -236,8 +256,14 @@ static void gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); + + for (GstPad *srcpad : state->srcpads_) + gst_libcamera_pad_set_pool(srcpad, NULL); + + g_clear_object(&self->allocator); } static void @@ -246,6 +272,8 @@ gst_libcamera_src_close(GstLibcameraSrc *self) GstLibcameraSrcState *state = self->state; gint ret; + GST_DEBUG_OBJECT(self, "Releasing resources"); + ret = state->cam_->release(); if (ret) { GST_ELEMENT_WARNING(self, RESOURCE, BUSY, From patchwork Fri Mar 6 20:26:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3026 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6394D628C5 for ; Fri, 6 Mar 2020 21:27:10 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id C52CA2970E7; Fri, 6 Mar 2020 20:27:09 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:30 -0500 Message-Id: <20200306202637.525587-21-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 20/27] gst: Add getters for Stream and FrameBuffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:10 -0000 From: Nicolas Dufresne This adds getters on pad/pool/allocator so that we can retrieve the Stream or FrameBuffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcameraallocator.cpp | 8 ++++++++ src/gstreamer/gstlibcameraallocator.h | 2 ++ src/gstreamer/gstlibcamerapad.cpp | 11 +++++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ src/gstreamer/gstlibcamerapool.cpp | 20 ++++++++++++++++++++ src/gstreamer/gstlibcamerapool.h | 7 +++++++ 6 files changed, 50 insertions(+) diff --git a/src/gstreamer/gstlibcameraallocator.cpp b/src/gstreamer/gstlibcameraallocator.cpp index 861af59..d0b90ec 100644 --- a/src/gstreamer/gstlibcameraallocator.cpp +++ b/src/gstreamer/gstlibcameraallocator.cpp @@ -243,3 +243,11 @@ gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *self, return pool->length; } + +FrameBuffer * +gst_libcamera_memory_get_frame_buffer(GstMemory *mem) +{ + auto *frame = reinterpret_cast(gst_mini_object_get_qdata(GST_MINI_OBJECT_CAST(mem), + FrameWrap::getQuark())); + return frame->buffer_; +} diff --git a/src/gstreamer/gstlibcameraallocator.h b/src/gstreamer/gstlibcameraallocator.h index b3f7f36..befdcad 100644 --- a/src/gstreamer/gstlibcameraallocator.h +++ b/src/gstreamer/gstlibcameraallocator.h @@ -27,4 +27,6 @@ bool gst_libcamera_allocator_prepare_buffer(GstLibcameraAllocator *self, gsize gst_libcamera_allocator_get_pool_size(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +libcamera::FrameBuffer *gst_libcamera_memory_get_frame_buffer(GstMemory *mem); + #endif /* __GST_LIBCAMERA_ALLOCATOR_H__ */ diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index dbe3361..8616e06 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -126,3 +126,14 @@ gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool) g_object_unref(self->pool); self->pool = pool; } + +Stream * +gst_libcamera_pad_get_stream(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (self->pool) + return gst_libcamera_pool_get_stream(self->pool); + + return nullptr; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index b03f47a..f7dfc28 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -24,4 +24,6 @@ GstLibcameraPool *gst_libcamera_pad_get_pool(GstPad *pad); void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); +libcamera::Stream *gst_libcamera_pad_get_stream(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp index 75e189a..3868905 100644 --- a/src/gstreamer/gstlibcamerapool.cpp +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -108,3 +108,23 @@ gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) return pool; } + +Stream * +gst_libcamera_pool_get_stream(GstLibcameraPool *self) +{ + return self->stream; +} + +Stream * +gst_libcamera_buffer_get_stream(GstBuffer *buffer) +{ + auto *self = (GstLibcameraPool *)buffer->pool; + return self->stream; +} + +FrameBuffer * +gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer) +{ + GstMemory *mem = gst_buffer_peek_memory(buffer, 0); + return gst_libcamera_memory_get_frame_buffer(mem); +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h index c0a04cf..a3f1b68 100644 --- a/src/gstreamer/gstlibcamerapool.h +++ b/src/gstreamer/gstlibcamerapool.h @@ -24,4 +24,11 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, libcamera::Stream *stream); +libcamera::Stream *gst_libcamera_pool_get_stream(GstLibcameraPool *self); + +libcamera::Stream *gst_libcamera_buffer_get_stream(GstBuffer *buffer); + +libcamera::FrameBuffer *gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer); + + #endif /* __GST_LIBCAMERA_POOL_H__ */ From patchwork Fri Mar 6 20:26:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3027 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E39B562922 for ; Fri, 6 Mar 2020 21:27:10 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 56F4E297189; Fri, 6 Mar 2020 20:27:10 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:31 -0500 Message-Id: <20200306202637.525587-22-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 21/27] gst: pad: Add method to store retrieve pending buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:12 -0000 From: Nicolas Dufresne These will be useful for streaming. The requestComplete callback will store the buffers on each pads so that the _run() can pick them up and push them through the pads from a streaming thread. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 27 +++++++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 4 ++++ 2 files changed, 31 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 8616e06..49dd35b 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -18,6 +18,7 @@ struct _GstLibcameraPad { GstPad parent; StreamRole role; GstLibcameraPool *pool; + GQueue pending_buffers; }; enum { @@ -137,3 +138,29 @@ gst_libcamera_pad_get_stream(GstPad *pad) return nullptr; } + +void +gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + + g_queue_push_head(&self->pending_buffers, buffer); +} + +GstFlowReturn +gst_libcamera_pad_push_pending(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GstBuffer *buffer; + + { + GLibLocker lock(GST_OBJECT(self)); + buffer = GST_BUFFER(g_queue_pop_tail(&self->pending_buffers)); + } + + if (!buffer) + return GST_FLOW_OK; + + return gst_pad_push(pad, buffer); +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index f7dfc28..2e9ec20 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -26,4 +26,8 @@ void gst_libcamera_pad_set_pool(GstPad *pad, GstLibcameraPool *pool); libcamera::Stream *gst_libcamera_pad_get_stream(GstPad *pad); +void gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer); + +GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ From patchwork Fri Mar 6 20:26:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3028 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A555628A3 for ; Fri, 6 Mar 2020 21:27:11 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id DC1762970E7; Fri, 6 Mar 2020 20:27:10 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 6 Mar 2020 15:26:32 -0500 Message-Id: <20200306202637.525587-23-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 22/27] gst: libcamerasrc: Implement initial streaming X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:12 -0000 From: Nicolas Dufresne With this patch, the element is now able to push buffers to the next element in the graph. The buffers are currently missing any metadata like timestamp, sequence number. This will be added in the next commit. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 9 ++ src/gstreamer/gstlibcamerapad.h | 2 + src/gstreamer/gstlibcamerasrc.cpp | 192 +++++++++++++++++++++++++++++- 3 files changed, 202 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 49dd35b..840f391 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -19,6 +19,7 @@ struct _GstLibcameraPad { StreamRole role; GstLibcameraPool *pool; GQueue pending_buffers; + GstClockTime latency; }; enum { @@ -164,3 +165,11 @@ gst_libcamera_pad_push_pending(GstPad *pad) return gst_pad_push(pad, buffer); } + +bool +gst_libcamera_pad_has_pending(GstPad *pad) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + return self->pending_buffers.length > 0; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 2e9ec20..9d43129 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -30,4 +30,6 @@ void gst_libcamera_pad_queue_buffer(GstPad *pad, GstBuffer *buffer); GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); +bool gst_libcamera_pad_has_pending(GstPad *pad); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 5ffc004..e3718db 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -14,8 +14,11 @@ #include "gstlibcamerasrc.h" +#include #include +#include + #include #include @@ -29,12 +32,71 @@ using namespace libcamera; GST_DEBUG_CATEGORY_STATIC(source_debug); #define GST_CAT_DEFAULT source_debug +struct RequestWrap { + RequestWrap(Request *request); + ~RequestWrap(); + + void attachBuffer(GstBuffer *buffer); + GstBuffer *detachBuffer(Stream *stream); + + /* For ptr comparison only. */ + Request *request_; + std::map buffers_; +}; + +RequestWrap::RequestWrap(Request *request) + : request_(request) +{ +} + +RequestWrap::~RequestWrap() +{ + for (std::pair &item : buffers_) { + if (item.second) + gst_buffer_unref(item.second); + } +} + +void RequestWrap::attachBuffer(GstBuffer *buffer) +{ + FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); + Stream *stream = gst_libcamera_buffer_get_stream(buffer); + + request_->addBuffer(stream, fb); + + auto item = buffers_.find(stream); + if (item != buffers_.end()) { + gst_buffer_unref(item->second); + item->second = buffer; + } else { + buffers_[stream] = buffer; + } +} + +GstBuffer *RequestWrap::detachBuffer(Stream *stream) +{ + GstBuffer *buffer = nullptr; + + auto item = buffers_.find(stream); + if (item != buffers_.end()) { + buffer = item->second; + item->second = nullptr; + } + + return buffer; +} + /* Used for C++ object with destructors. */ struct GstLibcameraSrcState { + GstLibcameraSrc *src_; + std::unique_ptr cm_; std::shared_ptr cam_; std::unique_ptr config_; std::vector srcpads_; + std::queue> requests_; + + void requestCompleted(Request *request); }; struct _GstLibcameraSrc { @@ -47,6 +109,7 @@ struct _GstLibcameraSrc { GstLibcameraSrcState *state; GstLibcameraAllocator *allocator; + GstFlowCombiner *flow_combiner; }; enum { @@ -70,6 +133,41 @@ GstStaticPadTemplate request_src_template = { "src_%s", GST_PAD_SRC, GST_PAD_REQUEST, TEMPLATE_CAPS }; +void +GstLibcameraSrcState::requestCompleted(Request *request) +{ + GLibLocker lock(GST_OBJECT(src_)); + + GST_DEBUG_OBJECT(src_, "buffers are ready"); + + std::unique_ptr wrap = std::move(requests_.front()); + requests_.pop(); + + g_return_if_fail(wrap->request_ == request); + + if ((request->status() == Request::RequestCancelled)) { + GST_DEBUG_OBJECT(src_, "Request was cancelled"); + return; + } + + GstBuffer *buffer; + for (GstPad *srcpad : srcpads_) { + Stream *stream = gst_libcamera_pad_get_stream(srcpad); + buffer = wrap->detachBuffer(stream); + gst_libcamera_pad_queue_buffer(srcpad, buffer); + } + + { + /* We only want to resume the task if it's paused. */ + GstTask *task = src_->task; + GLibLocker lock(GST_OBJECT(task)); + if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { + GST_TASK_STATE(task) = GST_TASK_STARTED; + GST_TASK_SIGNAL(task); + } + } +} + static bool gst_libcamera_src_open(GstLibcameraSrc *self) { @@ -122,6 +220,8 @@ gst_libcamera_src_open(GstLibcameraSrc *self) return false; } + cam->requestCompleted.connect(self->state, &GstLibcameraSrcState::requestCompleted); + /* No need to lock here, we didn't start our threads yet. */ self->state->cm_ = std::move(cm); self->state->cam_ = cam; @@ -133,8 +233,77 @@ static void gst_libcamera_src_task_run(gpointer user_data) { GstLibcameraSrc *self = GST_LIBCAMERA_SRC(user_data); + GstLibcameraSrcState *state = self->state; + + Request *request = state->cam_->createRequest(); + auto wrap = std::make_unique(request); + for (GstPad *srcpad : state->srcpads_) { + GstLibcameraPool *pool = gst_libcamera_pad_get_pool(srcpad); + GstBuffer *buffer; + GstFlowReturn ret; + + ret = gst_buffer_pool_acquire_buffer(GST_BUFFER_POOL(pool), + &buffer, nullptr); + if (ret != GST_FLOW_OK) { + /* + * RequestWrap does not take ownership, and we won't be + * queueing this one due to lack of buffers. + */ + delete request; + request = nullptr; + break; + } + + wrap->attachBuffer(buffer); + } + + if (request) { + GLibLocker lock(GST_OBJECT(self)); + GST_TRACE_OBJECT(self, "Requesting buffers"); + state->cam_->queueRequest(request); + state->requests_.push(std::move(wrap)); + } + + GstFlowReturn ret = GST_FLOW_OK; + gst_flow_combiner_reset(self->flow_combiner); + for (GstPad *srcpad : state->srcpads_) { + ret = gst_libcamera_pad_push_pending(srcpad); + ret = gst_flow_combiner_update_pad_flow(self->flow_combiner, + srcpad, ret); + } - GST_DEBUG_OBJECT(self, "Streaming thread is now capturing"); + { + /* + * Here we need to decide if we want to pause or stop the task. This + * needs to happen in lock step with the callback thread which may want + * to resume the task. + */ + GLibLocker lock(GST_OBJECT(self)); + if (ret != GST_FLOW_OK) { + if (ret == GST_FLOW_EOS) { + g_autoptr(GstEvent) eos = gst_event_new_eos(); + guint32 seqnum = gst_util_seqnum_next(); + gst_event_set_seqnum(eos, seqnum); + for (GstPad *srcpad : state->srcpads_) + gst_pad_push_event(srcpad, gst_event_ref(eos)); + } else if (ret != GST_FLOW_FLUSHING) { + GST_ELEMENT_FLOW_ERROR(self, ret); + } + gst_task_stop(self->task); + return; + } + + bool do_pause = true; + for (GstPad *srcpad : state->srcpads_) { + if (gst_libcamera_pad_has_pending(srcpad)) { + do_pause = false; + break; + } + } + + if (do_pause) + gst_task_pause(self->task); + } } static void @@ -233,12 +402,23 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) return; } + self->flow_combiner = gst_flow_combiner_new(); for (gsize i = 0; i < state->srcpads_.size(); i++) { GstPad *srcpad = state->srcpads_[i]; const StreamConfiguration &stream_cfg = state->config_->at(i); GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, stream_cfg.stream()); gst_libcamera_pad_set_pool(srcpad, pool); + gst_flow_combiner_add_pad(self->flow_combiner, srcpad); + } + + ret = state->cam_->start(); + if (ret) { + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Failed to start the camera: %s", g_strerror(-ret)), + ("Camera.start() failed with error code %i", ret)); + gst_task_stop(task); + return; } done: @@ -260,10 +440,14 @@ gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) GST_DEBUG_OBJECT(self, "Streaming thread is about to stop"); + state->cam_->stop(); + for (GstPad *srcpad : state->srcpads_) gst_libcamera_pad_set_pool(srcpad, NULL); g_clear_object(&self->allocator); + g_clear_pointer(&self->flow_combiner, + (GDestroyNotify)gst_flow_combiner_free); } static void @@ -343,6 +527,9 @@ gst_libcamera_src_change_state(GstElement *element, GstStateChange transition) return GST_STATE_CHANGE_FAILURE; ret = GST_STATE_CHANGE_NO_PREROLL; break; + case GST_STATE_CHANGE_PAUSED_TO_PLAYING: + gst_task_start(self->task); + break; case GST_STATE_CHANGE_PLAYING_TO_PAUSED: ret = GST_STATE_CHANGE_NO_PREROLL; break; @@ -394,6 +581,9 @@ gst_libcamera_src_init(GstLibcameraSrc *self) state->srcpads_.push_back(gst_pad_new_from_template(templ, "src")); gst_element_add_pad(GST_ELEMENT(self), state->srcpads_[0]); + + /* C-style friend. */ + state->src_ = self; self->state = state; } From patchwork Fri Mar 6 20:26:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3029 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EC4E562795 for ; Fri, 6 Mar 2020 21:27:11 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 59DFF297190; Fri, 6 Mar 2020 20:27:11 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:33 -0500 Message-Id: <20200306202637.525587-24-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 23/27] gst: libcamerasrc: Implement timestamp support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:12 -0000 From: Nicolas Dufresne This is an experimental patch adding timestamp support to the libcamerasrc element. This patch currently assume that the driver timestamp are relative to the system monotonic clock. Without a reference clock source, the timestamp are otherwise unusable, and without timestamp only minor use case can be achieved. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapad.cpp | 23 +++++++++++++++++++++++ src/gstreamer/gstlibcamerapad.h | 2 ++ src/gstreamer/gstlibcamerasrc.cpp | 20 ++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 840f391..e184495 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -63,9 +63,24 @@ gst_libcamera_pad_get_property(GObject *object, guint prop_id, GValue *value, } } +static gboolean +gst_libcamera_pad_query(GstPad *pad, GstObject *parent, GstQuery *query) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + + if (query->type != GST_QUERY_LATENCY) + return gst_pad_query_default(pad, parent, query); + + /* TRUE here means live, we assumes that max latency is the same as min + * as we have no idea that duration of frames. */ + gst_query_set_latency(query, TRUE, self->latency, self->latency); + return TRUE; +} + static void gst_libcamera_pad_init(GstLibcameraPad *self) { + GST_PAD_QUERYFUNC(self) = gst_libcamera_pad_query; } static GType @@ -173,3 +188,11 @@ gst_libcamera_pad_has_pending(GstPad *pad) GLibLocker lock(GST_OBJECT(self)); return self->pending_buffers.length > 0; } + +void +gst_libcamera_pad_set_latency(GstPad *pad, GstClockTime latency) +{ + auto *self = GST_LIBCAMERA_PAD(pad); + GLibLocker lock(GST_OBJECT(self)); + self->latency = latency; +} diff --git a/src/gstreamer/gstlibcamerapad.h b/src/gstreamer/gstlibcamerapad.h index 9d43129..779f2d1 100644 --- a/src/gstreamer/gstlibcamerapad.h +++ b/src/gstreamer/gstlibcamerapad.h @@ -32,4 +32,6 @@ GstFlowReturn gst_libcamera_pad_push_pending(GstPad *pad); bool gst_libcamera_pad_has_pending(GstPad *pad); +void gst_libcamera_pad_set_latency(GstPad *pad, GstClockTime latency); + #endif /* __GST_LIBCAMERA_PAD_H__ */ diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index e3718db..b04e9f1 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -154,6 +154,26 @@ GstLibcameraSrcState::requestCompleted(Request *request) for (GstPad *srcpad : srcpads_) { Stream *stream = gst_libcamera_pad_get_stream(srcpad); buffer = wrap->detachBuffer(stream); + + FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); + + if (GST_ELEMENT_CLOCK(src_)) { + GstClockTime gst_base_time = GST_ELEMENT(src_)->base_time; + GstClockTime gst_now = gst_clock_get_time(GST_ELEMENT_CLOCK(src_)); + /* \todo Need to expose which reference clock the timestamp relates to. */ + GstClockTime sys_now = g_get_monotonic_time() * 1000; + + /* Deduced from: sys_now - sys_base_time == gst_now - gst_base_time */ + GstClockTime sys_base_time = sys_now - (gst_now - gst_base_time); + GST_BUFFER_PTS(buffer) = fb->metadata().timestamp - sys_base_time; + gst_libcamera_pad_set_latency(srcpad, sys_now - fb->metadata().timestamp); + } else { + GST_BUFFER_PTS(buffer) = 0; + } + + GST_BUFFER_OFFSET(buffer) = fb->metadata().sequence; + GST_BUFFER_OFFSET_END(buffer) = fb->metadata().sequence; + gst_libcamera_pad_queue_buffer(srcpad, buffer); } From patchwork Fri Mar 6 20:26:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3030 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7336A628B3 for ; Fri, 6 Mar 2020 21:27:12 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id DECE02970E7; Fri, 6 Mar 2020 20:27:11 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:34 -0500 Message-Id: <20200306202637.525587-25-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 24/27] gst: libcamerasrc: Add a TODO comment X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:12 -0000 From: Nicolas Dufresne This is to guide upcoming contributors toward what is left to do to get toward a production ready element. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index b04e9f1..8d0d0e1 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -7,6 +7,25 @@ */ /** + * \todo The following is a list of items that needs implementation in the GStreamer plugin + * - Implement GstElement::send_event + * + Allowing application to send EOS + * + Allowing application to use FLUSH/FLUSH_STOP + * + Prevent the main thread from accessing streaming thread + * - Implement renegotiation (even if slow) + * - Implement GstElement::request-new-pad (multi stream) + * + Evaluate if a single streaming thread is fine + * - Add application driven request (snapshot) + * - Add framerate control + * - Add buffer importation support + * + * Requires new libcamera API: + * - Add framerate negotiation support + * - Add colorimetry support + * - Add timestamp support + * - Use unique names to select the camera devices + * - Add GstVideoMeta support (strides and offsets) + * * \todo libcamera UVC drivers picks the lowest possible resolution first, this * should be fixed so that we get a decent resolution and framerate for the * role by default. From patchwork Fri Mar 6 20:26:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3031 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 01B70628B3 for ; Fri, 6 Mar 2020 21:27:13 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 70039297190; Fri, 6 Mar 2020 20:27:12 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Jakub Adam , Laurent Pinchart Date: Fri, 6 Mar 2020 15:26:35 -0500 Message-Id: <20200306202637.525587-26-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 25/27] gst: utils: Factor-out the task resume helper X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:13 -0000 From: Jakub Adam Task resume will be added in the core GStreamer API in the future and we will need to call this in another location in the following patches. Signed-off-by: Jakub Adam Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamera-utils.cpp | 11 +++++++++++ src/gstreamer/gstlibcamera-utils.h | 1 + src/gstreamer/gstlibcamerasrc.cpp | 10 +--------- 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 482439a..44a993f 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -167,3 +167,14 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, stream_cfg.size.width = width; stream_cfg.size.height = height; } + +void +gst_libcamera_resume_task(GstTask *task) +{ + /* We only want to resume the task if it's paused. */ + GLibLocker lock(GST_OBJECT(task)); + if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { + GST_TASK_STATE(task) = GST_TASK_STARTED; + GST_TASK_SIGNAL(task); + } +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index cbfb1cc..2b3f26b 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -18,6 +18,7 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &fo GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, GstCaps *caps); +void gst_libcamera_resume_task(GstTask *task); /** * \class GLibLocker diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 8d0d0e1..22d9175 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -196,15 +196,7 @@ GstLibcameraSrcState::requestCompleted(Request *request) gst_libcamera_pad_queue_buffer(srcpad, buffer); } - { - /* We only want to resume the task if it's paused. */ - GstTask *task = src_->task; - GLibLocker lock(GST_OBJECT(task)); - if (GST_TASK_STATE(task) == GST_TASK_PAUSED) { - GST_TASK_STATE(task) = GST_TASK_STARTED; - GST_TASK_SIGNAL(task); - } - } + gst_libcamera_resume_task(this->src_->task); } static bool From patchwork Fri Mar 6 20:26:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3032 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7F7E0628B3 for ; Fri, 6 Mar 2020 21:27:13 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 008E52970E7; Fri, 6 Mar 2020 20:27:12 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Jakub Adam Date: Fri, 6 Mar 2020 15:26:36 -0500 Message-Id: <20200306202637.525587-27-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 26/27] gst: libcamerasrc: Prevent src task deadlock on exhausted buffer pool X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:13 -0000 From: Jakub Adam Allow GstLibcameraPool to notify the source when a new buffer has become available in a previously exhausted buffer pool. This can be used to resume a src task that got paused because it couldn't acquire a buffer. Without this change the src task will never resume from pause once the pool gets exhausted. To trigger the deadlock (it doesn't happen every time), run: gst-launch-1.0 libcamerasrc ! queue ! glimagesink Signed-off-by: Jakub Adam Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerapool.cpp | 17 +++++++++++++++++ src/gstreamer/gstlibcamerasrc.cpp | 5 ++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp index 3868905..8f53616 100644 --- a/src/gstreamer/gstlibcamerapool.cpp +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -14,6 +14,13 @@ using namespace libcamera; +enum { + SIGNAL_BUFFER_NOTIFY, + N_SIGNALS +}; + +static guint signals[N_SIGNALS]; + struct _GstLibcameraPool { GstBufferPool parent; @@ -55,7 +62,12 @@ static void gst_libcamera_pool_release_buffer(GstBufferPool *pool, GstBuffer *buffer) { GstLibcameraPool *self = GST_LIBCAMERA_POOL(pool); + bool do_notify = gst_atomic_queue_length(self->queue) == 0; + gst_atomic_queue_push(self->queue, buffer); + + if (do_notify) + g_signal_emit(self, signals[SIGNAL_BUFFER_NOTIFY], 0); } static void @@ -90,6 +102,11 @@ gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass) pool_class->acquire_buffer = gst_libcamera_pool_acquire_buffer; pool_class->reset_buffer = gst_libcamera_pool_reset_buffer; pool_class->release_buffer = gst_libcamera_pool_release_buffer; + + signals[SIGNAL_BUFFER_NOTIFY] = g_signal_new("buffer-notify", + G_OBJECT_CLASS_TYPE(klass), G_SIGNAL_RUN_LAST, + 0, nullptr, nullptr, nullptr, + G_TYPE_NONE, 0); } GstLibcameraPool * diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 22d9175..9755922 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -439,6 +439,9 @@ gst_libcamera_src_task_enter(GstTask *task, GThread *thread, gpointer user_data) const StreamConfiguration &stream_cfg = state->config_->at(i); GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, stream_cfg.stream()); + g_signal_connect_swapped(pool, "buffer-notify", + G_CALLBACK(gst_libcamera_resume_task), task); + gst_libcamera_pad_set_pool(srcpad, pool); gst_flow_combiner_add_pad(self->flow_combiner, srcpad); } @@ -474,7 +477,7 @@ gst_libcamera_src_task_leave(GstTask *task, GThread *thread, gpointer user_data) state->cam_->stop(); for (GstPad *srcpad : state->srcpads_) - gst_libcamera_pad_set_pool(srcpad, NULL); + gst_libcamera_pad_set_pool(srcpad, nullptr); g_clear_object(&self->allocator); g_clear_pointer(&self->flow_combiner, From patchwork Fri Mar 6 20:26:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 3033 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 23C9E628B3 for ; Fri, 6 Mar 2020 21:27:14 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 72B52297189; Fri, 6 Mar 2020 20:27:13 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 6 Mar 2020 15:26:37 -0500 Message-Id: <20200306202637.525587-28-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200306202637.525587-1-nicolas@ndufresne.ca> References: <20200306202637.525587-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 27/27] gst: Document dependencies and quick usage X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Mar 2020 20:27:14 -0000 From: Nicolas Dufresne This is a quick startup guide allowing to build and use the GStreamer element from the libcamera source tree. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- README.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/README.rst b/README.rst index fa14113..a06e444 100644 --- a/README.rst +++ b/README.rst @@ -64,4 +64,23 @@ for qcam: [optional] for documentation: [optional] python3-sphinx doxygen +for gstreamer: [optional] + libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev + +Using GStreamer plugin +~~~~~~~~~~~~~~~~~~~~~~ + +To use GStreamer plugin from source tree, set the following environment so that +GStreamer can find it. + + export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer + +The debugging tool `gst-launch-1.0` can be used to construct and pipeline and test +it. The following pipeline will stream from the camera named "Camera 1" onto the +default video display element on your system. + +.. code:: + + gst-launch-1.0 libcamerasrc name="Camera 1" ! videoconvert ! autovideosink + .. section-end-getting-started