[v6] gstreamer: Plumb camera orientation configuration in libcamerasrc
diff mbox series

Message ID 20251106130133.130946-1-uajain@igalia.com
State New
Headers show
Series
  • [v6] gstreamer: Plumb camera orientation configuration in libcamerasrc
Related show

Commit Message

Umang Jain Nov. 6, 2025, 1:01 p.m. UTC 
Plumb the support for CameraConfiguration::orientation in libcamerasrc.
A new "orientation" property is introduced and mappings for
libcamera::Orientation <> GstVideoOrientationMethod are provided
with helpers.

If the orientation is changed after the CameraConfiguration::validate(),
a GstMessage is sent on the GstBus to report the new orientation
property value.

In case the orientation that was requested cannot be satisfied,
a 'image-orientation' tag is pushed downstream for sinks to rotate
the stream accordingly (those implementing
GSTGST_VIDEO_ORIENTATION_AUTO).

Signed-off-by: Umang Jain <uajain@igalia.com>
---
Changes in v6:
- Use GstMessage to send new property changes if any (like orientation
  after validate())
- send 'image-orientation' tag orientation downstream if orientation
  cannot be satisfied
- Drop 'video-orientation' and restore 'orientation' as property name
- change authorship to myself as patch is reworked significantly

Changes in v5:
- patch scrubbing and cleanup
- If different orientation is returned than requested, update the
  property
- Minor string fixes, append appropriate tags

Link to v4: https://patchwork.libcamera.org/patch/23965/

Note: Checkstyle will complain on this patch on a hunk, but the existing format
is quite read-able IMO, hence ignored.
---
 src/gstreamer/gstlibcamera-utils.cpp | 60 ++++++++++++++++++++++++++++
 src/gstreamer/gstlibcamera-utils.h   |  5 +++
 src/gstreamer/gstlibcamerasrc.cpp    | 58 +++++++++++++++++++++++++++
 3 files changed, 123 insertions(+)

Comments

Jacopo Mondi Aug. 6, 2026, 8:40 a.m. UTC | #1 
Hi Umang,
  I noticed this patch -really- fell into the cracks..

Is it still valid, still something that hasn't been addressed
otherwise, right ?

On Thu, Nov 06, 2025 at 01:01:33PM +0000, Umang Jain wrote:
> Plumb the support for CameraConfiguration::orientation in libcamerasrc.
> A new "orientation" property is introduced and mappings for
> libcamera::Orientation <> GstVideoOrientationMethod are provided
> with helpers.
>
> If the orientation is changed after the CameraConfiguration::validate(),
> a GstMessage is sent on the GstBus to report the new orientation
> property value.
>
> In case the orientation that was requested cannot be satisfied,
> a 'image-orientation' tag is pushed downstream for sinks to rotate
> the stream accordingly (those implementing
> GSTGST_VIDEO_ORIENTATION_AUTO).
>
> Signed-off-by: Umang Jain <uajain@igalia.com>
> ---
> Changes in v6:
> - Use GstMessage to send new property changes if any (like orientation
>   after validate())
> - send 'image-orientation' tag orientation downstream if orientation
>   cannot be satisfied
> - Drop 'video-orientation' and restore 'orientation' as property name
> - change authorship to myself as patch is reworked significantly
>
> Changes in v5:
> - patch scrubbing and cleanup
> - If different orientation is returned than requested, update the
>   property
> - Minor string fixes, append appropriate tags
>
> Link to v4: https://patchwork.libcamera.org/patch/23965/
>
> Note: Checkstyle will complain on this patch on a hunk, but the existing format
> is quite read-able IMO, hence ignored.
> ---
>  src/gstreamer/gstlibcamera-utils.cpp | 60 ++++++++++++++++++++++++++++
>  src/gstreamer/gstlibcamera-utils.h   |  5 +++
>  src/gstreamer/gstlibcamerasrc.cpp    | 58 +++++++++++++++++++++++++++
>  3 files changed, 123 insertions(+)
>
> diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp
> index bfb094c9..79fb606b 100644
> --- a/src/gstreamer/gstlibcamera-utils.cpp
> +++ b/src/gstreamer/gstlibcamera-utils.cpp
> @@ -357,6 +357,42 @@ control_type_to_gtype(const ControlType &type)
>  	return G_TYPE_INVALID;
>  }
>
> +static const struct {
> +	Orientation orientation;
> +	GstVideoOrientationMethod method;
> +} orientation_map[]{
> +	{ Orientation::Rotate0, GST_VIDEO_ORIENTATION_IDENTITY },
> +	{ Orientation::Rotate90, GST_VIDEO_ORIENTATION_90R },
> +	{ Orientation::Rotate180, GST_VIDEO_ORIENTATION_180 },
> +	{ Orientation::Rotate270, GST_VIDEO_ORIENTATION_90L },
> +	{ Orientation::Rotate0Mirror, GST_VIDEO_ORIENTATION_HORIZ },
> +	{ Orientation::Rotate180Mirror, GST_VIDEO_ORIENTATION_VERT },
> +	{ Orientation::Rotate90Mirror, GST_VIDEO_ORIENTATION_UL_LR },
> +	{ Orientation::Rotate270Mirror, GST_VIDEO_ORIENTATION_UR_LL },
> +};
> +
> +Orientation
> +gst_video_orientation_to_libcamera_orientation(GstVideoOrientationMethod method)
> +{
> +	for (auto &b : orientation_map) {
> +		if (b.method == method)
> +			return b.orientation;
> +	}
> +
> +	return Orientation::Rotate0;
> +}
> +
> +GstVideoOrientationMethod
> +libcamera_orientation_to_gst_video_orientation(Orientation orientation)
> +{
> +	for (auto &a : orientation_map) {
> +		if (a.orientation == orientation)
> +			return a.method;
> +	}
> +
> +	return GST_VIDEO_ORIENTATION_IDENTITY;
> +}
> +
>  GstCaps *
>  gst_libcamera_stream_formats_to_caps(const StreamFormats &formats)
>  {
> @@ -898,3 +934,27 @@ int gst_libcamera_set_structure_field(GstStructure *structure, const ControlId *
>
>  	return 0;
>  }
> +
> +const gchar *
> +gst_libcamera_transform_to_tag_string(libcamera::Transform transform)
> +{
> +	switch (transform) {
> +	case Transform::Rot90:
> +		return "rotate-90";
> +	case Transform::Rot180:
> +		return "rotate-180";
> +	case Transform::Rot270:
> +		return "rotate-270";
> +	case Transform::HFlip:
> +		return "flip-rotate-0";
> +	case Transform::VFlip:
> +		return "flip-rotate-180";
> +	case Transform::Transpose:
> +		return "flip-rotate-270";
> +	case Transform::Rot180Transpose:
> +		return "flip-rotate-90";
> +	case Transform::Identity:
> +	default:
> +		return "rotate-0";
> +	}
> +}
> diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h
> index 35df56fb..b7983264 100644
> --- a/src/gstreamer/gstlibcamera-utils.h
> +++ b/src/gstreamer/gstlibcamera-utils.h
> @@ -10,7 +10,9 @@
>
>  #include <libcamera/camera_manager.h>
>  #include <libcamera/controls.h>
> +#include <libcamera/orientation.h>
>  #include <libcamera/stream.h>
> +#include <libcamera/transform.h>
>
>  #include <gst/gst.h>
>  #include <gst/video/video.h>
> @@ -32,6 +34,9 @@ libcamera::Rectangle gst_libcamera_gvalue_get_rectangle(const GValue *value);
>  int gst_libcamera_set_structure_field(GstStructure *structure,
>  				      const libcamera::ControlId *id,
>  				      const libcamera::ControlValue &value);
> +libcamera::Orientation gst_video_orientation_to_libcamera_orientation(GstVideoOrientationMethod method);
> +GstVideoOrientationMethod libcamera_orientation_to_gst_video_orientation(libcamera::Orientation orientation);
> +const gchar *gst_libcamera_transform_to_tag_string(libcamera::Transform transform);
>
>  #if !GST_CHECK_VERSION(1, 16, 0)
>  static inline void gst_clear_event(GstEvent **event_ptr)
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp
> index 391b228d..77648e6a 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -141,6 +141,7 @@ struct _GstLibcameraSrc {
>  	GstTask *task;
>
>  	gchar *camera_name;
> +	GstVideoOrientationMethod orientation;
>
>  	std::atomic<GstEvent *> pending_eos;
>
> @@ -152,9 +153,12 @@ struct _GstLibcameraSrc {
>  enum {
>  	PROP_0,
>  	PROP_CAMERA_NAME,
> +	PROP_ORIENTATION,
>  	PROP_LAST
>  };
>
> +static GParamSpec *properties[PROP_LAST];
> +
>  static void gst_libcamera_src_child_proxy_init(gpointer g_iface,
>  					       gpointer iface_data);
>
> @@ -606,6 +610,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>  		gst_libcamera_get_framerate_from_caps(caps, element_caps);
>  	}
>
> +	/* Set orientation in libcamera camera configuration. */
> +	Orientation requestedLibcameraOrientation = gst_video_orientation_to_libcamera_orientation(self->orientation);
> +	{
> +		GLibLocker lock(GST_OBJECT(self));
> +		state->config_->orientation = requestedLibcameraOrientation;
> +	}
> +
>  	/* Validate the configuration. */
>  	CameraConfiguration::Status status = state->config_->validate();
>  	if (status == CameraConfiguration::Invalid)
> @@ -627,6 +638,33 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>  	gst_libcamera_clamp_and_set_frameduration(state->initControls_,
>  						  state->cam_->controls(), element_caps);
>
> +	/*
> +	 * If the requested orientation isn't possible, report the libcamera-provided orientation
> +	 * on GstBus and send the "image-orientation" tag downstream for elements to apply
> +	 * the final requested orientation.
> +	 */
> +	if (state->config_->orientation != requestedLibcameraOrientation) {
> +		{
> +			GLibLocker lock(GST_OBJECT(self));
> +			self->orientation = libcamera_orientation_to_gst_video_orientation(state->config_->orientation);
> +		}
> +
> +		GValue v = G_VALUE_INIT;
> +		g_value_init(&v, properties[PROP_ORIENTATION]->value_type);
> +		g_object_get_property(G_OBJECT(self), properties[PROP_ORIENTATION]->name, &v);
> +		GstMessage *msg = gst_message_new_property_notify(GST_OBJECT(self), properties[PROP_ORIENTATION]->name, &v);
> +		gst_element_post_message(GST_ELEMENT(self), msg);
> +
> +		Transform toApply = requestedLibcameraOrientation / state->config_->orientation;
> +		const gchar *transform_tag = gst_libcamera_transform_to_tag_string(toApply);
> +
> +		g_autoptr(GstEvent) tag_event =
> +			gst_event_new_tag(gst_tag_list_new(GST_TAG_IMAGE_ORIENTATION, transform_tag, NULL));
> +
> +		for (gsize i = 0; i < state->srcpads_.size(); i++)
> +			gst_pad_push_event(state->srcpads_[i], gst_event_ref(tag_event));
> +	}
> +
>  	/*
>  	 * Regardless if it has been modified, create clean caps and push the
>  	 * caps event. Downstream will decide if the caps are acceptable.
> @@ -934,6 +972,9 @@ gst_libcamera_src_set_property(GObject *object, guint prop_id,
>  		g_free(self->camera_name);
>  		self->camera_name = g_value_dup_string(value);
>  		break;
> +	case PROP_ORIENTATION:
> +		self->orientation = static_cast<GstVideoOrientationMethod>(g_value_get_enum(value));
> +		break;
>  	default:
>  		if (!state->controls_.setProperty(prop_id - PROP_LAST, value, pspec))
>  			G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
> @@ -953,6 +994,9 @@ gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value,
>  	case PROP_CAMERA_NAME:
>  		g_value_set_string(value, self->camera_name);
>  		break;
> +	case PROP_ORIENTATION:
> +		g_value_set_enum(value, static_cast<gint>(self->orientation));
> +		break;
>  	default:
>  		if (!state->controls_.getProperty(prop_id - PROP_LAST, value, pspec))
>  			G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
> @@ -1162,6 +1206,20 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass)
>  							     | G_PARAM_STATIC_STRINGS));
>  	g_object_class_install_property(object_class, PROP_CAMERA_NAME, spec);
>
> +
> +	properties[PROP_ORIENTATION] = g_param_spec_enum("orientation", "orientation",
> +							 "Property to control flipping and rotation operations of the camera. "
> +							 "If the orientation cannot be satisfied, libcamerasrc will send 'image-orientation' "
> +							 "tag downstream to assist with the orientation that was requested. Sinks that "
> +							 "implement the GST_VIDEO_ORIENTATION_AUTO should rotate the stream accordingly. ",
> +							 GST_TYPE_VIDEO_ORIENTATION_METHOD,
> +							 GST_VIDEO_ORIENTATION_IDENTITY,
> +							 (GParamFlags)(GST_PARAM_MUTABLE_READY
> +							  | G_PARAM_CONSTRUCT
> +							  | G_PARAM_READWRITE
> +							  | G_PARAM_STATIC_STRINGS));
> +	g_object_class_install_property(object_class, PROP_ORIENTATION, properties[PROP_ORIENTATION]);
> +
>  	GstCameraControls::installProperties(object_class, PROP_LAST);
>  }
>
> --
> 2.51.0
>

Patch
diff mbox series 

diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp
index bfb094c9..79fb606b 100644
--- a/src/gstreamer/gstlibcamera-utils.cpp
+++ b/src/gstreamer/gstlibcamera-utils.cpp
@@ -357,6 +357,42 @@  control_type_to_gtype(const ControlType &type)
 	return G_TYPE_INVALID;
 }
 
+static const struct {
+	Orientation orientation;
+	GstVideoOrientationMethod method;
+} orientation_map[]{
+	{ Orientation::Rotate0, GST_VIDEO_ORIENTATION_IDENTITY },
+	{ Orientation::Rotate90, GST_VIDEO_ORIENTATION_90R },
+	{ Orientation::Rotate180, GST_VIDEO_ORIENTATION_180 },
+	{ Orientation::Rotate270, GST_VIDEO_ORIENTATION_90L },
+	{ Orientation::Rotate0Mirror, GST_VIDEO_ORIENTATION_HORIZ },
+	{ Orientation::Rotate180Mirror, GST_VIDEO_ORIENTATION_VERT },
+	{ Orientation::Rotate90Mirror, GST_VIDEO_ORIENTATION_UL_LR },
+	{ Orientation::Rotate270Mirror, GST_VIDEO_ORIENTATION_UR_LL },
+};
+
+Orientation
+gst_video_orientation_to_libcamera_orientation(GstVideoOrientationMethod method)
+{
+	for (auto &b : orientation_map) {
+		if (b.method == method)
+			return b.orientation;
+	}
+
+	return Orientation::Rotate0;
+}
+
+GstVideoOrientationMethod
+libcamera_orientation_to_gst_video_orientation(Orientation orientation)
+{
+	for (auto &a : orientation_map) {
+		if (a.orientation == orientation)
+			return a.method;
+	}
+
+	return GST_VIDEO_ORIENTATION_IDENTITY;
+}
+
 GstCaps *
 gst_libcamera_stream_formats_to_caps(const StreamFormats &formats)
 {
@@ -898,3 +934,27 @@  int gst_libcamera_set_structure_field(GstStructure *structure, const ControlId *
 
 	return 0;
 }
+
+const gchar *
+gst_libcamera_transform_to_tag_string(libcamera::Transform transform)
+{
+	switch (transform) {
+	case Transform::Rot90:
+		return "rotate-90";
+	case Transform::Rot180:
+		return "rotate-180";
+	case Transform::Rot270:
+		return "rotate-270";
+	case Transform::HFlip:
+		return "flip-rotate-0";
+	case Transform::VFlip:
+		return "flip-rotate-180";
+	case Transform::Transpose:
+		return "flip-rotate-270";
+	case Transform::Rot180Transpose:
+		return "flip-rotate-90";
+	case Transform::Identity:
+	default:
+		return "rotate-0";
+	}
+}
diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h
index 35df56fb..b7983264 100644
--- a/src/gstreamer/gstlibcamera-utils.h
+++ b/src/gstreamer/gstlibcamera-utils.h
@@ -10,7 +10,9 @@ 
 
 #include <libcamera/camera_manager.h>
 #include <libcamera/controls.h>
+#include <libcamera/orientation.h>
 #include <libcamera/stream.h>
+#include <libcamera/transform.h>
 
 #include <gst/gst.h>
 #include <gst/video/video.h>
@@ -32,6 +34,9 @@  libcamera::Rectangle gst_libcamera_gvalue_get_rectangle(const GValue *value);
 int gst_libcamera_set_structure_field(GstStructure *structure,
 				      const libcamera::ControlId *id,
 				      const libcamera::ControlValue &value);
+libcamera::Orientation gst_video_orientation_to_libcamera_orientation(GstVideoOrientationMethod method);
+GstVideoOrientationMethod libcamera_orientation_to_gst_video_orientation(libcamera::Orientation orientation);
+const gchar *gst_libcamera_transform_to_tag_string(libcamera::Transform transform);
 
 #if !GST_CHECK_VERSION(1, 16, 0)
 static inline void gst_clear_event(GstEvent **event_ptr)
diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp
index 391b228d..77648e6a 100644
--- a/src/gstreamer/gstlibcamerasrc.cpp
+++ b/src/gstreamer/gstlibcamerasrc.cpp
@@ -141,6 +141,7 @@  struct _GstLibcameraSrc {
 	GstTask *task;
 
 	gchar *camera_name;
+	GstVideoOrientationMethod orientation;
 
 	std::atomic<GstEvent *> pending_eos;
 
@@ -152,9 +153,12 @@  struct _GstLibcameraSrc {
 enum {
 	PROP_0,
 	PROP_CAMERA_NAME,
+	PROP_ORIENTATION,
 	PROP_LAST
 };
 
+static GParamSpec *properties[PROP_LAST];
+
 static void gst_libcamera_src_child_proxy_init(gpointer g_iface,
 					       gpointer iface_data);
 
@@ -606,6 +610,13 @@  gst_libcamera_src_negotiate(GstLibcameraSrc *self)
 		gst_libcamera_get_framerate_from_caps(caps, element_caps);
 	}
 
+	/* Set orientation in libcamera camera configuration. */
+	Orientation requestedLibcameraOrientation = gst_video_orientation_to_libcamera_orientation(self->orientation);
+	{
+		GLibLocker lock(GST_OBJECT(self));
+		state->config_->orientation = requestedLibcameraOrientation;
+	}
+
 	/* Validate the configuration. */
 	CameraConfiguration::Status status = state->config_->validate();
 	if (status == CameraConfiguration::Invalid)
@@ -627,6 +638,33 @@  gst_libcamera_src_negotiate(GstLibcameraSrc *self)
 	gst_libcamera_clamp_and_set_frameduration(state->initControls_,
 						  state->cam_->controls(), element_caps);
 
+	/*
+	 * If the requested orientation isn't possible, report the libcamera-provided orientation
+	 * on GstBus and send the "image-orientation" tag downstream for elements to apply
+	 * the final requested orientation.
+	 */
+	if (state->config_->orientation != requestedLibcameraOrientation) {
+		{
+			GLibLocker lock(GST_OBJECT(self));
+			self->orientation = libcamera_orientation_to_gst_video_orientation(state->config_->orientation);
+		}
+
+		GValue v = G_VALUE_INIT;
+		g_value_init(&v, properties[PROP_ORIENTATION]->value_type);
+		g_object_get_property(G_OBJECT(self), properties[PROP_ORIENTATION]->name, &v);
+		GstMessage *msg = gst_message_new_property_notify(GST_OBJECT(self), properties[PROP_ORIENTATION]->name, &v);
+		gst_element_post_message(GST_ELEMENT(self), msg);
+
+		Transform toApply = requestedLibcameraOrientation / state->config_->orientation;
+		const gchar *transform_tag = gst_libcamera_transform_to_tag_string(toApply);
+
+		g_autoptr(GstEvent) tag_event =
+			gst_event_new_tag(gst_tag_list_new(GST_TAG_IMAGE_ORIENTATION, transform_tag, NULL));
+
+		for (gsize i = 0; i < state->srcpads_.size(); i++)
+			gst_pad_push_event(state->srcpads_[i], gst_event_ref(tag_event));
+	}
+
 	/*
 	 * Regardless if it has been modified, create clean caps and push the
 	 * caps event. Downstream will decide if the caps are acceptable.
@@ -934,6 +972,9 @@  gst_libcamera_src_set_property(GObject *object, guint prop_id,
 		g_free(self->camera_name);
 		self->camera_name = g_value_dup_string(value);
 		break;
+	case PROP_ORIENTATION:
+		self->orientation = static_cast<GstVideoOrientationMethod>(g_value_get_enum(value));
+		break;
 	default:
 		if (!state->controls_.setProperty(prop_id - PROP_LAST, value, pspec))
 			G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
@@ -953,6 +994,9 @@  gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value,
 	case PROP_CAMERA_NAME:
 		g_value_set_string(value, self->camera_name);
 		break;
+	case PROP_ORIENTATION:
+		g_value_set_enum(value, static_cast<gint>(self->orientation));
+		break;
 	default:
 		if (!state->controls_.getProperty(prop_id - PROP_LAST, value, pspec))
 			G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec);
@@ -1162,6 +1206,20 @@  gst_libcamera_src_class_init(GstLibcameraSrcClass *klass)
 							     | G_PARAM_STATIC_STRINGS));
 	g_object_class_install_property(object_class, PROP_CAMERA_NAME, spec);
 
+
+	properties[PROP_ORIENTATION] = g_param_spec_enum("orientation", "orientation",
+							 "Property to control flipping and rotation operations of the camera. "
+							 "If the orientation cannot be satisfied, libcamerasrc will send 'image-orientation' "
+							 "tag downstream to assist with the orientation that was requested. Sinks that "
+							 "implement the GST_VIDEO_ORIENTATION_AUTO should rotate the stream accordingly. ",
+							 GST_TYPE_VIDEO_ORIENTATION_METHOD,
+							 GST_VIDEO_ORIENTATION_IDENTITY,
+							 (GParamFlags)(GST_PARAM_MUTABLE_READY
+							  | G_PARAM_CONSTRUCT
+							  | G_PARAM_READWRITE
+							  | G_PARAM_STATIC_STRINGS));
+	g_object_class_install_property(object_class, PROP_ORIENTATION, properties[PROP_ORIENTATION]);
+
 	GstCameraControls::installProperties(object_class, PROP_LAST);
 }