[v2] gstreamer: Honor downstream format preferences in caps negotiation
diff mbox series

Message ID 20260722034011.1016641-1-elliot.chen@oss.nxp.com
State New
Headers show
Series
  • [v2] gstreamer: Honor downstream format preferences in caps negotiation
Related show

Commit Message

Elliot Chen (OSS) July 22, 2026, 3:40 a.m. UTC 
From: Elliot Chen <elliot.chen@nxp.com>

When using libcamerasrc as a camera source in camerabin with a
video-source-filter that specifies a particular output format,
the negotiated format was not respecting the downstream preference.
This occurred because gst_pad_peer_query_caps() was called with the
libcamera stream format filter directly as its filter argument, which
caused the returned caps to be ordered by the libcamera source format
list rather than by the downstream element's preference.

Fix this by splitting the caps query into two steps: first query the
full set of caps supported by the downstream peer without any filter,
then intersect with the libcamera stream formats using
GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order
of the first operand (the downstream peer caps), so when the caps are
subsequently fixated, the format preferred by the downstream element
(e.g. the video-source-filter in camerabin) is selected.

Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
---
 src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

Comments

Kieran Bingham July 22, 2026, 10:36 a.m. UTC | #1 
Quoting elliot.chen@oss.nxp.com (2026-07-22 04:40:11)
> From: Elliot Chen <elliot.chen@nxp.com>
> 
> When using libcamerasrc as a camera source in camerabin with a
> video-source-filter that specifies a particular output format,
> the negotiated format was not respecting the downstream preference.
> This occurred because gst_pad_peer_query_caps() was called with the
> libcamera stream format filter directly as its filter argument, which
> caused the returned caps to be ordered by the libcamera source format
> list rather than by the downstream element's preference.
> 
> Fix this by splitting the caps query into two steps: first query the
> full set of caps supported by the downstream peer without any filter,
> then intersect with the libcamera stream formats using
> GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order
> of the first operand (the downstream peer caps), so when the caps are
> subsequently fixated, the format preferred by the downstream element
> (e.g. the video-source-filter in camerabin) is selected.
> 
> Signed-off-by: Elliot Chen <elliot.chen@nxp.com>

Is there any difference here between v1 and v2 or is it just the commit
message?

I can't see any review comments on v1 that prompted a v2, and there's no
changelog.

--
Kieran

> ---
>  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp
> index 9061f9163..579bd466d 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>                 StreamConfiguration &stream_cfg = state->config_->at(i);
>  
>                 /* Retrieve the supported caps. */
> +               g_autoptr(GstCaps) peer_caps = gst_pad_peer_query_caps(srcpad, NULL);
> +               if (!peer_caps || gst_caps_is_empty(peer_caps))
> +                       return false;
> +
>                 g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> -               g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad, filter);
> +               /* Intersect with downstream's preferred format order first. */
> +               g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps, filter, GST_CAPS_INTERSECT_FIRST);
>                 if (gst_caps_is_empty(caps))
>                         return false;
>  
> -- 
> 2.34.1
>
Elliot Chen (OSS) July 22, 2026, 10:48 a.m. UTC | #2 
Hi Kieran Bingham,
Thank you for your comments. I updated the comments in the v1 that should be end with a period. And I also updated the commit message to add the issue description that needs to be fixed by this patch.

-----Original Message-----
From: Kieran Bingham <kieran.bingham@ideasonboard.com> 
Sent: Wednesday, July 22, 2026 6:36 PM
To: Elliot Chen <elliot.chen@nxp.com>; Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org
Subject: Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation

Quoting elliot.chen@oss.nxp.com (2026-07-22 04:40:11)
> From: Elliot Chen <elliot.chen@nxp.com>
> 
> When using libcamerasrc as a camera source in camerabin with a 
> video-source-filter that specifies a particular output format, the 
> negotiated format was not respecting the downstream preference.
> This occurred because gst_pad_peer_query_caps() was called with the 
> libcamera stream format filter directly as its filter argument, which 
> caused the returned caps to be ordered by the libcamera source format 
> list rather than by the downstream element's preference.
> 
> Fix this by splitting the caps query into two steps: first query the 
> full set of caps supported by the downstream peer without any filter, 
> then intersect with the libcamera stream formats using 
> GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> of the first operand (the downstream peer caps), so when the caps are 
> subsequently fixated, the format preferred by the downstream element 
> (e.g. the video-source-filter in camerabin) is selected.
> 
> Signed-off-by: Elliot Chen <elliot.chen@nxp.com>

Is there any difference here between v1 and v2 or is it just the commit message?

I can't see any review comments on v1 that prompted a v2, and there's no changelog.

--
Kieran

> ---
>  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp 
> b/src/gstreamer/gstlibcamerasrc.cpp
> index 9061f9163..579bd466d 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>                 StreamConfiguration &stream_cfg = 
> state->config_->at(i);
>  
>                 /* Retrieve the supported caps. */
> +               g_autoptr(GstCaps) peer_caps = gst_pad_peer_query_caps(srcpad, NULL);
> +               if (!peer_caps || gst_caps_is_empty(peer_caps))
> +                       return false;
> +
>                 g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> -               g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad, filter);
> +               /* Intersect with downstream's preferred format order first. */
> +               g_autoptr(GstCaps) caps = 
> + gst_caps_intersect_full(peer_caps, filter, 
> + GST_CAPS_INTERSECT_FIRST);
>                 if (gst_caps_is_empty(caps))
>                         return false;
>  
> --
> 2.34.1
>
Nicolas Dufresne July 22, 2026, 6:46 p.m. UTC | #3 
Hi Elliot,

Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> From: Elliot Chen <elliot.chen@nxp.com>
> 
> When using libcamerasrc as a camera source in camerabin with a
> video-source-filter that specifies a particular output format,
> the negotiated format was not respecting the downstream preference.
> This occurred because gst_pad_peer_query_caps() was called with the
> libcamera stream format filter directly as its filter argument, which
> caused the returned caps to be ordered by the libcamera source format
> list rather than by the downstream element's preference.

Can you extend on how you have set your filter, and what you described as a
preference ?

Nicolas

> 
> Fix this by splitting the caps query into two steps: first query the
> full set of caps supported by the downstream peer without any filter,
> then intersect with the libcamera stream formats using
> GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order
> of the first operand (the downstream peer caps), so when the caps are
> subsequently fixated, the format preferred by the downstream element
> (e.g. the video-source-filter in camerabin) is selected.
> 
> Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> ---
>  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> b/src/gstreamer/gstlibcamerasrc.cpp
> index 9061f9163..579bd466d 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>  		StreamConfiguration &stream_cfg = state->config_->at(i);
>  
>  		/* Retrieve the supported caps. */
> +		g_autoptr(GstCaps) peer_caps =
> gst_pad_peer_query_caps(srcpad, NULL);
> +		if (!peer_caps || gst_caps_is_empty(peer_caps))
> +			return false;
> +
>  		g_autoptr(GstCaps) filter =
> gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> -		g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> filter);
> +		/* Intersect with downstream's preferred format order first.
> */
> +		g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps,
> filter, GST_CAPS_INTERSECT_FIRST);
>  		if (gst_caps_is_empty(caps))
>  			return false;
>
Elliot Chen July 23, 2026, 6:50 a.m. UTC | #4 
Hi Nicolas Dufresne,
1. Thank you for your comments. The issue occurs when using ap1302 camera on NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
2. First, I create a capsfilter to configure the expected output format and resolution, then configure video-source-filter property of wrappercamerabinsrc. 
3. I wrote a demo to showcase my configuration flow as below. The final pipeline diagram can refer to the attachment.
     libcamerasrc will select NV12 instead of YUY2 format in this flow. But in fact, camera can support YUY2 format. 

//step1: configure the expected video format and resolution
gchar *output_video_format_str = "YUY2";
gint image_width = 1280;
gint image_height = 800;
camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-raw",
                "width", G_TYPE_INT, image_width,
                "height", G_TYPE_INT, image_height,
                "format", G_TYPE_STRING, output_video_format_str,
                NULL), NULL);
gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter caps=", gst_caps_to_string (camera_output_caps));
videosrc_filter = gst_parse_bin_from_description (video_filter_str, TRUE, NULL);

//step2: set video source filter
GstElement *wrapper = gst_element_factory_make ("wrappercamerabinsrc", NULL);
g_object_set (wrapper, "video-source-filter", videosrc_filter, NULL);


-----Original Message-----
From: Nicolas Dufresne <nicolas@ndufresne.ca> 
Sent: Thursday, July 23, 2026 2:46 AM
To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org; Elliot Chen <elliot.chen@nxp.com>
Subject: [EXT] Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation

Hi Elliot,

Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> From: Elliot Chen <elliot.chen@nxp.com>
> 
> When using libcamerasrc as a camera source in camerabin with a 
> video-source-filter that specifies a particular output format, the 
> negotiated format was not respecting the downstream preference.
> This occurred because gst_pad_peer_query_caps() was called with the 
> libcamera stream format filter directly as its filter argument, which 
> caused the returned caps to be ordered by the libcamera source format 
> list rather than by the downstream element's preference.

Can you extend on how you have set your filter, and what you described as a preference ?

Nicolas

> 
> Fix this by splitting the caps query into two steps: first query the 
> full set of caps supported by the downstream peer without any filter, 
> then intersect with the libcamera stream formats using 
> GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> of the first operand (the downstream peer caps), so when the caps are 
> subsequently fixated, the format preferred by the downstream element 
> (e.g. the video-source-filter in camerabin) is selected.
> 
> Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> ---
>  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> b/src/gstreamer/gstlibcamerasrc.cpp
> index 9061f9163..579bd466d 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>  		StreamConfiguration &stream_cfg = state->config_->at(i);
>  
>  		/* Retrieve the supported caps. */
> +		g_autoptr(GstCaps) peer_caps =
> gst_pad_peer_query_caps(srcpad, NULL);
> +		if (!peer_caps || gst_caps_is_empty(peer_caps))
> +			return false;
> +
>  		g_autoptr(GstCaps) filter =
> gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> -		g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> filter);
> +		/* Intersect with downstream's preferred format order first.
> */
> +		g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps,
> filter, GST_CAPS_INTERSECT_FIRST);
>  		if (gst_caps_is_empty(caps))
>  			return false;
>
Elliot Chen (OSS) July 23, 2026, 6:57 a.m. UTC | #5 
Hi Nicolas Dufresne,
1. Thank you for your comments. The issue occurs when using ap1302 camera on NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
2. First, I create a capsfilter to configure the expected output format and resolution, then configure video-source-filter property of wrappercamerabinsrc. 
3. I wrote a demo to showcase my configuration flow as below. The final pipeline diagram can refer to the attachment.
     libcamerasrc will select NV12 instead of YUY2 format in this flow. But in fact, camera can support YUY2 format. 

//step1: configure the expected video format and resolution gchar *output_video_format_str = "YUY2"; gint image_width = 1280; gint image_height = 800; camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-raw",
                "width", G_TYPE_INT, image_width,
                "height", G_TYPE_INT, image_height,
                "format", G_TYPE_STRING, output_video_format_str,
                NULL), NULL);
gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter caps=", gst_caps_to_string (camera_output_caps)); videosrc_filter = gst_parse_bin_from_description (video_filter_str, TRUE, NULL);

//step2: set video source filter
GstElement *wrapper = gst_element_factory_make ("wrappercamerabinsrc", NULL); g_object_set (wrapper, "video-source-filter", videosrc_filter, NULL);

-----Original Message-----
From: Nicolas Dufresne <nicolas@ndufresne.ca> 
Sent: Thursday, July 23, 2026 2:46 AM
To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org; Elliot Chen <elliot.chen@nxp.com>
Subject: Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation

Hi Elliot,

Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> From: Elliot Chen <elliot.chen@nxp.com>
> 
> When using libcamerasrc as a camera source in camerabin with a 
> video-source-filter that specifies a particular output format, the 
> negotiated format was not respecting the downstream preference.
> This occurred because gst_pad_peer_query_caps() was called with the 
> libcamera stream format filter directly as its filter argument, which 
> caused the returned caps to be ordered by the libcamera source format 
> list rather than by the downstream element's preference.

Can you extend on how you have set your filter, and what you described as a preference ?

Nicolas

> 
> Fix this by splitting the caps query into two steps: first query the 
> full set of caps supported by the downstream peer without any filter, 
> then intersect with the libcamera stream formats using 
> GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> of the first operand (the downstream peer caps), so when the caps are 
> subsequently fixated, the format preferred by the downstream element 
> (e.g. the video-source-filter in camerabin) is selected.
> 
> Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> ---
>  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> b/src/gstreamer/gstlibcamerasrc.cpp
> index 9061f9163..579bd466d 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>  		StreamConfiguration &stream_cfg = state->config_->at(i);
>  
>  		/* Retrieve the supported caps. */
> +		g_autoptr(GstCaps) peer_caps =
> gst_pad_peer_query_caps(srcpad, NULL);
> +		if (!peer_caps || gst_caps_is_empty(peer_caps))
> +			return false;
> +
>  		g_autoptr(GstCaps) filter =
> gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> -		g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> filter);
> +		/* Intersect with downstream's preferred format order first.
> */
> +		g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps,
> filter, GST_CAPS_INTERSECT_FIRST);
>  		if (gst_caps_is_empty(caps))
>  			return false;
>
Elliot Chen (OSS) July 24, 2026, 2:15 a.m. UTC | #6 
Hi Nicolas Dufresne,
Another point is that v4l2src plugin also follows this rule.

-----Original Message-----
From: Elliot Chen <elliot.chen@nxp.com> 
Sent: Thursday, July 23, 2026 2:51 PM
To: Nicolas Dufresne <nicolas@ndufresne.ca>; Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org
Subject: RE: [EXT] Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation

Hi Nicolas Dufresne,
1. Thank you for your comments. The issue occurs when using ap1302 camera on NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
2. First, I create a capsfilter to configure the expected output format and resolution, then configure video-source-filter property of wrappercamerabinsrc. 
3. I wrote a demo to showcase my configuration flow as below. The final pipeline diagram can refer to the attachment.
     libcamerasrc will select NV12 instead of YUY2 format in this flow. But in fact, camera can support YUY2 format. 

//step1: configure the expected video format and resolution gchar *output_video_format_str = "YUY2"; gint image_width = 1280; gint image_height = 800; camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-raw",
                "width", G_TYPE_INT, image_width,
                "height", G_TYPE_INT, image_height,
                "format", G_TYPE_STRING, output_video_format_str,
                NULL), NULL);
gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter caps=", gst_caps_to_string (camera_output_caps)); videosrc_filter = gst_parse_bin_from_description (video_filter_str, TRUE, NULL);

//step2: set video source filter
GstElement *wrapper = gst_element_factory_make ("wrappercamerabinsrc", NULL); g_object_set (wrapper, "video-source-filter", videosrc_filter, NULL);


-----Original Message-----
From: Nicolas Dufresne <nicolas@ndufresne.ca>
Sent: Thursday, July 23, 2026 2:46 AM
To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org; Elliot Chen <elliot.chen@nxp.com>
Subject: [EXT] Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation

Hi Elliot,

Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> From: Elliot Chen <elliot.chen@nxp.com>
> 
> When using libcamerasrc as a camera source in camerabin with a 
> video-source-filter that specifies a particular output format, the 
> negotiated format was not respecting the downstream preference.
> This occurred because gst_pad_peer_query_caps() was called with the 
> libcamera stream format filter directly as its filter argument, which 
> caused the returned caps to be ordered by the libcamera source format 
> list rather than by the downstream element's preference.

Can you extend on how you have set your filter, and what you described as a preference ?

Nicolas

> 
> Fix this by splitting the caps query into two steps: first query the 
> full set of caps supported by the downstream peer without any filter, 
> then intersect with the libcamera stream formats using 
> GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> of the first operand (the downstream peer caps), so when the caps are 
> subsequently fixated, the format preferred by the downstream element 
> (e.g. the video-source-filter in camerabin) is selected.
> 
> Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> ---
>  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> b/src/gstreamer/gstlibcamerasrc.cpp
> index 9061f9163..579bd466d 100644
> --- a/src/gstreamer/gstlibcamerasrc.cpp
> +++ b/src/gstreamer/gstlibcamerasrc.cpp
> @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
>  		StreamConfiguration &stream_cfg = state->config_->at(i);
>  
>  		/* Retrieve the supported caps. */
> +		g_autoptr(GstCaps) peer_caps =
> gst_pad_peer_query_caps(srcpad, NULL);
> +		if (!peer_caps || gst_caps_is_empty(peer_caps))
> +			return false;
> +
>  		g_autoptr(GstCaps) filter =
> gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> -		g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> filter);
> +		/* Intersect with downstream's preferred format order first.
> */
> +		g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps,
> filter, GST_CAPS_INTERSECT_FIRST);
>  		if (gst_caps_is_empty(caps))
>  			return false;
>
Nicolas Dufresne July 24, 2026, 2:59 p.m. UTC | #7 
Hi,

Le vendredi 24 juillet 2026 à 02:15 +0000, Elliot Chen (OSS) a écrit :
> Hi Nicolas Dufresne,
> Another point is that v4l2src plugin also follows this rule.

Can you share couple of simplified negotiation cases ? Things have shifted quite
a bit lately in v4l2src, enforcing v4l2src choices over downstream a bit. I'd
like to compare and understand better. camerabin case is something we could
solve by introducing a GstBaseCameraSrc that wraps the libcamerasrc properly,
allowing multi-streams etc.

Nicolas

> 
> -----Original Message-----
> From: Elliot Chen <elliot.chen@nxp.com> 
> Sent: Thursday, July 23, 2026 2:51 PM
> To: Nicolas Dufresne <nicolas@ndufresne.ca>; Elliot Chen (OSS)
> <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org
> Subject: RE: [EXT] Re: [PATCH v2] gstreamer: Honor downstream format
> preferences in caps negotiation
> 
> Hi Nicolas Dufresne,
> 1. Thank you for your comments. The issue occurs when using ap1302 camera on
> NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
> 2. First, I create a capsfilter to configure the expected output format and
> resolution, then configure video-source-filter property of
> wrappercamerabinsrc. 
> 3. I wrote a demo to showcase my configuration flow as below. The final
> pipeline diagram can refer to the attachment.
>      libcamerasrc will select NV12 instead of YUY2 format in this flow. But in
> fact, camera can support YUY2 format. 
> 
> //step1: configure the expected video format and resolution gchar
> *output_video_format_str = "YUY2"; gint image_width = 1280; gint image_height
> = 800; camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-
> raw",
>                 "width", G_TYPE_INT, image_width,
>                 "height", G_TYPE_INT, image_height,
>                 "format", G_TYPE_STRING, output_video_format_str,
>                 NULL), NULL);
> gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter caps=",
> gst_caps_to_string (camera_output_caps)); videosrc_filter =
> gst_parse_bin_from_description (video_filter_str, TRUE, NULL);
> 
> //step2: set video source filter
> GstElement *wrapper = gst_element_factory_make ("wrappercamerabinsrc", NULL);
> g_object_set (wrapper, "video-source-filter", videosrc_filter, NULL);
> 
> 
> -----Original Message-----
> From: Nicolas Dufresne <nicolas@ndufresne.ca>
> Sent: Thursday, July 23, 2026 2:46 AM
> To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>;
> libcamera-devel@lists.libcamera.org; Elliot Chen <elliot.chen@nxp.com>
> Subject: [EXT] Re: [PATCH v2] gstreamer: Honor downstream format preferences
> in caps negotiation
> 
> Hi Elliot,
> 
> Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> > From: Elliot Chen <elliot.chen@nxp.com>
> > 
> > When using libcamerasrc as a camera source in camerabin with a 
> > video-source-filter that specifies a particular output format, the 
> > negotiated format was not respecting the downstream preference.
> > This occurred because gst_pad_peer_query_caps() was called with the 
> > libcamera stream format filter directly as its filter argument, which 
> > caused the returned caps to be ordered by the libcamera source format 
> > list rather than by the downstream element's preference.
> 
> Can you extend on how you have set your filter, and what you described as a
> preference ?
> 
> Nicolas
> 
> > 
> > Fix this by splitting the caps query into two steps: first query the 
> > full set of caps supported by the downstream peer without any filter, 
> > then intersect with the libcamera stream formats using 
> > GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> > of the first operand (the downstream peer caps), so when the caps are 
> > subsequently fixated, the format preferred by the downstream element 
> > (e.g. the video-source-filter in camerabin) is selected.
> > 
> > Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> > ---
> >  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> > 
> > diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> > b/src/gstreamer/gstlibcamerasrc.cpp
> > index 9061f9163..579bd466d 100644
> > --- a/src/gstreamer/gstlibcamerasrc.cpp
> > +++ b/src/gstreamer/gstlibcamerasrc.cpp
> > @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
> >  		StreamConfiguration &stream_cfg = state->config_->at(i);
> >  
> >  		/* Retrieve the supported caps. */
> > +		g_autoptr(GstCaps) peer_caps =
> > gst_pad_peer_query_caps(srcpad, NULL);
> > +		if (!peer_caps || gst_caps_is_empty(peer_caps))
> > +			return false;
> > +
> >  		g_autoptr(GstCaps) filter =
> > gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> > -		g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> > filter);
> > +		/* Intersect with downstream's preferred format order
> > first.
> > */
> > +		g_autoptr(GstCaps) caps =
> > gst_caps_intersect_full(peer_caps,
> > filter, GST_CAPS_INTERSECT_FIRST);
> >  		if (gst_caps_is_empty(caps))
> >  			return false;
> >
Kieran Bingham July 27, 2026, 8:43 a.m. UTC | #8 
Hi Elliot,

I noticed that you are trying to send e-mails with large e-mail attachments
to this thread.

As the latest one is a 5MB attatchment, I'm afraid I'm considering
rejecting that from the mailing-list moderation request. I think sending
5MB of log files to every recipient of the libcamera maililng list is
probably excessive.

Perhaps if the discussions require large media attachments we could move
the discussion to
https://gitlab.freedesktop.org/camera/libcamera/-/work_items ?

--
Regards

Kieran


Quoting Elliot Chen (OSS) (2026-07-23 07:57:36)
> Hi Nicolas Dufresne,
> 1. Thank you for your comments. The issue occurs when using ap1302 camera on NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
> 2. First, I create a capsfilter to configure the expected output format and resolution, then configure video-source-filter property of wrappercamerabinsrc. 
> 3. I wrote a demo to showcase my configuration flow as below. The final pipeline diagram can refer to the attachment.
>      libcamerasrc will select NV12 instead of YUY2 format in this flow. But in fact, camera can support YUY2 format. 
> 
> //step1: configure the expected video format and resolution gchar *output_video_format_str = "YUY2"; gint image_width = 1280; gint image_height = 800; camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-raw",
>                 "width", G_TYPE_INT, image_width,
>                 "height", G_TYPE_INT, image_height,
>                 "format", G_TYPE_STRING, output_video_format_str,
>                 NULL), NULL);
> gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter caps=", gst_caps_to_string (camera_output_caps)); videosrc_filter = gst_parse_bin_from_description (video_filter_str, TRUE, NULL);
> 
> //step2: set video source filter
> GstElement *wrapper = gst_element_factory_make ("wrappercamerabinsrc", NULL); g_object_set (wrapper, "video-source-filter", videosrc_filter, NULL);
> 
> -----Original Message-----
> From: Nicolas Dufresne <nicolas@ndufresne.ca> 
> Sent: Thursday, July 23, 2026 2:46 AM
> To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org; Elliot Chen <elliot.chen@nxp.com>
> Subject: Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation
> 
> Hi Elliot,
> 
> Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> > From: Elliot Chen <elliot.chen@nxp.com>
> > 
> > When using libcamerasrc as a camera source in camerabin with a 
> > video-source-filter that specifies a particular output format, the 
> > negotiated format was not respecting the downstream preference.
> > This occurred because gst_pad_peer_query_caps() was called with the 
> > libcamera stream format filter directly as its filter argument, which 
> > caused the returned caps to be ordered by the libcamera source format 
> > list rather than by the downstream element's preference.
> 
> Can you extend on how you have set your filter, and what you described as a preference ?
> 
> Nicolas
> 
> > 
> > Fix this by splitting the caps query into two steps: first query the 
> > full set of caps supported by the downstream peer without any filter, 
> > then intersect with the libcamera stream formats using 
> > GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> > of the first operand (the downstream peer caps), so when the caps are 
> > subsequently fixated, the format preferred by the downstream element 
> > (e.g. the video-source-filter in camerabin) is selected.
> > 
> > Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> > ---
> >  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> > 
> > diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> > b/src/gstreamer/gstlibcamerasrc.cpp
> > index 9061f9163..579bd466d 100644
> > --- a/src/gstreamer/gstlibcamerasrc.cpp
> > +++ b/src/gstreamer/gstlibcamerasrc.cpp
> > @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
> >               StreamConfiguration &stream_cfg = state->config_->at(i);
> >  
> >               /* Retrieve the supported caps. */
> > +             g_autoptr(GstCaps) peer_caps =
> > gst_pad_peer_query_caps(srcpad, NULL);
> > +             if (!peer_caps || gst_caps_is_empty(peer_caps))
> > +                     return false;
> > +
> >               g_autoptr(GstCaps) filter =
> > gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> > -             g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> > filter);
> > +             /* Intersect with downstream's preferred format order first.
> > */
> > +             g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps,
> > filter, GST_CAPS_INTERSECT_FIRST);
> >               if (gst_caps_is_empty(caps))
> >                       return false;
> >
Laurent Pinchart July 27, 2026, 9:16 a.m. UTC | #9 
On Mon, Jul 27, 2026 at 09:43:07AM +0100, Kieran Bingham wrote:
> Hi Elliot,
> 
> I noticed that you are trying to send e-mails with large e-mail attachments
> to this thread.
> 
> As the latest one is a 5MB attatchment, I'm afraid I'm considering
> rejecting that from the mailing-list moderation request. I think sending
> 5MB of log files to every recipient of the libcamera maililng list is
> probably excessive.
> 
> Perhaps if the discussions require large media attachments we could move
> the discussion to
> https://gitlab.freedesktop.org/camera/libcamera/-/work_items ?

Please don't.

Log files compress well.

> Quoting Elliot Chen (OSS) (2026-07-23 07:57:36)
> > Hi Nicolas Dufresne,
> > 1. Thank you for your comments. The issue occurs when using ap1302 camera on NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
> > 2. First, I create a capsfilter to configure the expected output format and resolution, then configure video-source-filter property of wrappercamerabinsrc. 
> > 3. I wrote a demo to showcase my configuration flow as below. The final pipeline diagram can refer to the attachment.
> >      libcamerasrc will select NV12 instead of YUY2 format in this flow. But in fact, camera can support YUY2 format. 
> > 
> > //step1: configure the expected video format and resolution gchar *output_video_format_str = "YUY2"; gint image_width = 1280; gint image_height = 800; camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-raw",
> >                 "width", G_TYPE_INT, image_width,
> >                 "height", G_TYPE_INT, image_height,
> >                 "format", G_TYPE_STRING, output_video_format_str,
> >                 NULL), NULL);
> > gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter caps=", gst_caps_to_string (camera_output_caps)); videosrc_filter = gst_parse_bin_from_description (video_filter_str, TRUE, NULL);
> > 
> > //step2: set video source filter
> > GstElement *wrapper = gst_element_factory_make ("wrappercamerabinsrc", NULL); g_object_set (wrapper, "video-source-filter", videosrc_filter, NULL);
> > 
> > -----Original Message-----
> > From: Nicolas Dufresne <nicolas@ndufresne.ca> 
> > Sent: Thursday, July 23, 2026 2:46 AM
> > To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org; Elliot Chen <elliot.chen@nxp.com>
> > Subject: Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation
> > 
> > Hi Elliot,
> > 
> > Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> > > From: Elliot Chen <elliot.chen@nxp.com>
> > > 
> > > When using libcamerasrc as a camera source in camerabin with a 
> > > video-source-filter that specifies a particular output format, the 
> > > negotiated format was not respecting the downstream preference.
> > > This occurred because gst_pad_peer_query_caps() was called with the 
> > > libcamera stream format filter directly as its filter argument, which 
> > > caused the returned caps to be ordered by the libcamera source format 
> > > list rather than by the downstream element's preference.
> > 
> > Can you extend on how you have set your filter, and what you described as a preference ?
> > 
> > Nicolas
> > 
> > > 
> > > Fix this by splitting the caps query into two steps: first query the 
> > > full set of caps supported by the downstream peer without any filter, 
> > > then intersect with the libcamera stream formats using 
> > > GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the order 
> > > of the first operand (the downstream peer caps), so when the caps are 
> > > subsequently fixated, the format preferred by the downstream element 
> > > (e.g. the video-source-filter in camerabin) is selected.
> > > 
> > > Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> > > ---
> > >  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
> > >  1 file changed, 6 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> > > b/src/gstreamer/gstlibcamerasrc.cpp
> > > index 9061f9163..579bd466d 100644
> > > --- a/src/gstreamer/gstlibcamerasrc.cpp
> > > +++ b/src/gstreamer/gstlibcamerasrc.cpp
> > > @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
> > >               StreamConfiguration &stream_cfg = state->config_->at(i);
> > >  
> > >               /* Retrieve the supported caps. */
> > > +             g_autoptr(GstCaps) peer_caps =
> > > gst_pad_peer_query_caps(srcpad, NULL);
> > > +             if (!peer_caps || gst_caps_is_empty(peer_caps))
> > > +                     return false;
> > > +
> > >               g_autoptr(GstCaps) filter =
> > > gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> > > -             g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> > > filter);
> > > +             /* Intersect with downstream's preferred format order first.
> > > */
> > > +             g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps,
> > > filter, GST_CAPS_INTERSECT_FIRST);
> > >               if (gst_caps_is_empty(caps))
> > >                       return false;
> > >
Elliot Chen (OSS) July 31, 2026, 3:13 a.m. UTC | #10 
Hi all,
1. I found I can't attach the log file in the mail, otherwise the mails are all filtered out regardless of whether the file is compressed or not. And I deleted it now.
2. Some more details about this patch is as below.
1). Using libcamerasrc as a camera source in camerabin, the pipeline can be simplified as below.
    bin(libcamerasrc) + videoconvert + bin(capsfilter caps="video/x-raw,format=YUY2,width=1280,height=800) +...
2). Firstly, the queried caps is not fixed and is in the order of the default camera format list(The first format is NV12) not the order from downstream(The first format is YUY2) in `gst_libcamera_src_negotiate` function. 
3). Secondly, libcamerasrc prioritize the caps listed earlier from the queried caps in gst_libcamera_configure_stream_from_caps ` function.
4). Finally, libcamerasrc output NV12 format and need to use videoconvert to convert NV12 to YUY2.
5). In v4l2src, the output caps is also in the order which is queried from downstream.

-----Original Message-----
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com> 
Sent: Monday, July 27, 2026 5:16 PM
To: Kieran Bingham <kieran.bingham@ideasonboard.com>
Cc: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; libcamera-devel@lists.libcamera.org; Nicolas Dufresne <nicolas@ndufresne.ca>
Subject: Re: [PATCH v2] gstreamer: Honor downstream format preferences in caps negotiation

On Mon, Jul 27, 2026 at 09:43:07AM +0100, Kieran Bingham wrote:
> Hi Elliot,
> 
> I noticed that you are trying to send e-mails with large e-mail 
> attachments to this thread.
> 
> As the latest one is a 5MB attatchment, I'm afraid I'm considering 
> rejecting that from the mailing-list moderation request. I think 
> sending 5MB of log files to every recipient of the libcamera maililng 
> list is probably excessive.
> 
> Perhaps if the discussions require large media attachments we could 
> move the discussion to 
> https://gitlab.freedesktop.org/camera/libcamera/-/work_items ?

Please don't.

Log files compress well.

> Quoting Elliot Chen (OSS) (2026-07-23 07:57:36)
> > Hi Nicolas Dufresne,
> > 1. Thank you for your comments. The issue occurs when using ap1302 camera on NXP i.MX95 board when using libcamerasrc as a camera source in camerabin. 
> > 2. First, I create a capsfilter to configure the expected output format and resolution, then configure video-source-filter property of wrappercamerabinsrc. 
> > 3. I wrote a demo to showcase my configuration flow as below. The final pipeline diagram can refer to the attachment.
> >      libcamerasrc will select NV12 instead of YUY2 format in this flow. But in fact, camera can support YUY2 format. 
> > 
> > //step1: configure the expected video format and resolution gchar *output_video_format_str = "YUY2"; gint image_width = 1280; gint image_height = 800; camera_output_caps  = gst_caps_new_full (gst_structure_new ("video/x-raw",
> >                 "width", G_TYPE_INT, image_width,
> >                 "height", G_TYPE_INT, image_height,
> >                 "format", G_TYPE_STRING, output_video_format_str,
> >                 NULL), NULL);
> > gchar *video_filter_str = g_strdup_printf ("%s\"%s\"", "capsfilter 
> > caps=", gst_caps_to_string (camera_output_caps)); videosrc_filter = 
> > gst_parse_bin_from_description (video_filter_str, TRUE, NULL);
> > 
> > //step2: set video source filter
> > GstElement *wrapper = gst_element_factory_make 
> > ("wrappercamerabinsrc", NULL); g_object_set (wrapper, 
> > "video-source-filter", videosrc_filter, NULL);
> > 
> > -----Original Message-----
> > From: Nicolas Dufresne <nicolas@ndufresne.ca>
> > Sent: Thursday, July 23, 2026 2:46 AM
> > To: Elliot Chen (OSS) <elliot.chen@oss.nxp.com>; 
> > libcamera-devel@lists.libcamera.org; Elliot Chen 
> > <elliot.chen@nxp.com>
> > Subject: Re: [PATCH v2] gstreamer: Honor downstream format 
> > preferences in caps negotiation
> > 
> > Hi Elliot,
> > 
> > Le mercredi 22 juillet 2026 à 12:40 +0900, elliot.chen@oss.nxp.com a écrit :
> > > From: Elliot Chen <elliot.chen@nxp.com>
> > > 
> > > When using libcamerasrc as a camera source in camerabin with a 
> > > video-source-filter that specifies a particular output format, the 
> > > negotiated format was not respecting the downstream preference.
> > > This occurred because gst_pad_peer_query_caps() was called with 
> > > the libcamera stream format filter directly as its filter 
> > > argument, which caused the returned caps to be ordered by the 
> > > libcamera source format list rather than by the downstream element's preference.
> > 
> > Can you extend on how you have set your filter, and what you described as a preference ?
> > 
> > Nicolas
> > 
> > > 
> > > Fix this by splitting the caps query into two steps: first query 
> > > the full set of caps supported by the downstream peer without any 
> > > filter, then intersect with the libcamera stream formats using 
> > > GST_CAPS_INTERSECT_FIRST. This intersection mode preserves the 
> > > order of the first operand (the downstream peer caps), so when the 
> > > caps are subsequently fixated, the format preferred by the 
> > > downstream element (e.g. the video-source-filter in camerabin) is selected.
> > > 
> > > Signed-off-by: Elliot Chen <elliot.chen@nxp.com>
> > > ---
> > >  src/gstreamer/gstlibcamerasrc.cpp | 7 ++++++-
> > >  1 file changed, 6 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/src/gstreamer/gstlibcamerasrc.cpp
> > > b/src/gstreamer/gstlibcamerasrc.cpp
> > > index 9061f9163..579bd466d 100644
> > > --- a/src/gstreamer/gstlibcamerasrc.cpp
> > > +++ b/src/gstreamer/gstlibcamerasrc.cpp
> > > @@ -598,8 +598,13 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)
> > >               StreamConfiguration &stream_cfg = 
> > > state->config_->at(i);
> > >  
> > >               /* Retrieve the supported caps. */
> > > +             g_autoptr(GstCaps) peer_caps =
> > > gst_pad_peer_query_caps(srcpad, NULL);
> > > +             if (!peer_caps || gst_caps_is_empty(peer_caps))
> > > +                     return false;
> > > +
> > >               g_autoptr(GstCaps) filter = 
> > > gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
> > > -             g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad,
> > > filter);
> > > +             /* Intersect with downstream's preferred format order first.
> > > */
> > > +             g_autoptr(GstCaps) caps = 
> > > + gst_caps_intersect_full(peer_caps,
> > > filter, GST_CAPS_INTERSECT_FIRST);
> > >               if (gst_caps_is_empty(caps))
> > >                       return false;
> > >

--
Regards,

Laurent Pinchart

Patch
diff mbox series 

diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp
index 9061f9163..579bd466d 100644
--- a/src/gstreamer/gstlibcamerasrc.cpp
+++ b/src/gstreamer/gstlibcamerasrc.cpp
@@ -598,8 +598,13 @@  gst_libcamera_src_negotiate(GstLibcameraSrc *self)
 		StreamConfiguration &stream_cfg = state->config_->at(i);
 
 		/* Retrieve the supported caps. */
+		g_autoptr(GstCaps) peer_caps = gst_pad_peer_query_caps(srcpad, NULL);
+		if (!peer_caps || gst_caps_is_empty(peer_caps))
+			return false;
+
 		g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats());
-		g_autoptr(GstCaps) caps = gst_pad_peer_query_caps(srcpad, filter);
+		/* Intersect with downstream's preferred format order first. */
+		g_autoptr(GstCaps) caps = gst_caps_intersect_full(peer_caps, filter, GST_CAPS_INTERSECT_FIRST);
 		if (gst_caps_is_empty(caps))
 			return false;