[{"id":39565,"web_url":"https://patchwork.libcamera.org/comment/39565/","msgid":"<8d59fb03154497a008a7c7761e13d17df22074db.camel@ndufresne.ca>","date":"2026-07-02T19:58:54","subject":"Re: [PATCH 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats\n\tin caps negotiation","submitter":{"id":30,"url":"https://patchwork.libcamera.org/api/people/30/","name":"Nicolas Dufresne","email":"nicolas@ndufresne.ca"},"content":"Hi David,\n\nthanks again for working on this.\n\nLe jeudi 02 juillet 2026 à 15:39 +0100, David Plowman a écrit :\n> Existing code was preferring fixed size outputs rather than ones with\n> ranges, resulting in raw output formats being preferred even when\n> applications (\"cheese\" is one such example) aren't expecting them and\n> subsequently fail.\n> \n> The revised version looks at raw (in the Bayer sense) and non-raw\n> formats separately, preferring fixed size outputs within each\n> category, but then will opt for the non-raw formats if any were\n> available. This still allows applications that explicitly request raw\n> formats to get them.\n> \n> Fixes, for example, \"cheese\" on Raspberry Pi.\n> \n> Signed-off-by: David Plowman \n> ---\n>  src/gstreamer/gstlibcamera-utils.cpp | 96 +++++++++++++++++++++-------\n>  src/gstreamer/gstlibcamera-utils.h   |  2 +-\n>  src/gstreamer/gstlibcamerasrc.cpp    |  3 +-\n>  3 files changed, 75 insertions(+), 26 deletions(-)\n> \n> diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp\n> index 6541d478..4f87525f 100644\n> --- a/src/gstreamer/gstlibcamera-utils.cpp\n> +++ b/src/gstreamer/gstlibcamera-utils.cpp\n> @@ -435,20 +435,54 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg\n>  \treturn caps;\n>  }\n>  \n> -void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n> +/*\n> + * We will want to distinguish between caps structures corresponding to\n> + * raw and non-raw (processed) output images, as this will help inform\n> + * our choice of preferred format.\n> + *\n> + * We identify Bayer as being the principle \"raw\" format here, but note\n> + * that we are considering greyscale to be raw too (e.g. a raw\n> + * monochrome sensor).\n> + */\n> +static bool\n> +gst_libcamera_structure_is_raw_capture(const GstStructure *s)\n> +{\n> +\tif (gst_structure_has_name(s, \"video/x-bayer\"))\n> +\t\treturn true;\n> +\n> +\tif (gst_structure_has_name(s, \"video/x-raw\")) {\n> +\t\tconst gchar *format = gst_structure_get_string(s, \"format\");\n> +\n> +\t\tif (format && (!strcmp(format, \"GRAY8\") || !strcmp(format, \"GRAY16_LE\") ||\n> +\t\t\t       !strcmp(format, \"GRAY10_LE16\")))\n> +\t\t\treturn true;\n\nA more generic solution could be something like:\n\nfinfo = gst_video_format_get_info(gst_video_format_from_string(format));\nif (finfo->n_components == 1)\n\treturn true;\n\n> +\t}\n> +\n> +\treturn false;\n> +}\n> +\n> +bool gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n>  \t\t\t\t\t      GstCaps *caps, GstVideoTransferFunction *transfer)\n>  {\n>  \tGstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat);\n>  \tguint i;\n> -\tgint best_fixed = -1, best_in_range = -1;\n>  \tGstStructure *s;\n>  \n>  \t/*\n> -\t * These are delta weight computed from:\n> -\t *   ABS(width - stream_cfg.size.width) * ABS(height - stream_cfg.size.height)\n> +\t * Track the best size match (by delta weight, see below) separately\n> +\t * for raw sensor capture structures (Bayer/mono) and non-raw ones.\n> +\t * The previous logic that preferred fixed sizes over a range is\n> +\t * preserved within each of these \"families\".\n>  \t */\n> -\tguint best_fixed_delta = G_MAXUINT;\n> -\tguint best_in_range_delta = G_MAXUINT;\n> +\tstruct FormatMatch {\n> +\t\tbool seen = false;\n> +\t\tgint fixed = -1;\n> +\t\tguint fixed_delta = G_MAXUINT;\n> +\t\tgint in_range = -1;\n> +\t\tguint in_range_delta = G_MAXUINT;\n> +\n> +\t\tgint best() const { return fixed >= 0 ? fixed : in_range; }\n> +\t} raw, non_raw;\n>  \n>  \t/* First fixate the caps using default configuration value. */\n>  \tg_assert(gst_caps_is_writable(caps));\n> @@ -458,38 +492,50 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n>  \t\ts = gst_caps_get_structure(caps, i);\n>  \t\tgint width, height;\n>  \t\tguint delta;\n> +\t\tbool is_fixed = gst_structure_has_field_typed(s, \"width\", G_TYPE_INT) &&\n> +\t\t\t\tgst_structure_has_field_typed(s, \"height\", G_TYPE_INT);\n>  \n> -\t\tif (gst_structure_has_field_typed(s, \"width\", G_TYPE_INT) &&\n> -\t\t    gst_structure_has_field_typed(s, \"height\", G_TYPE_INT)) {\n> +\t\tif (is_fixed) {\n>  \t\t\tgst_structure_get_int(s, \"width\", &width);\n>  \t\t\tgst_structure_get_int(s, \"height\", &height);\n> -\n> -\t\t\tdelta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height);\n> -\n> -\t\t\tif (delta < best_fixed_delta) {\n> -\t\t\t\tbest_fixed_delta = delta;\n> -\t\t\t\tbest_fixed = i;\n> -\t\t\t}\n>  \t\t} else {\n>  \t\t\tgst_structure_fixate_field_nearest_int(s, \"width\", stream_cfg.size.width);\n>  \t\t\tgst_structure_fixate_field_nearest_int(s, \"height\", stream_cfg.size.height);\n>  \t\t\tgst_structure_get_int(s, \"width\", &width);\n>  \t\t\tgst_structure_get_int(s, \"height\", &height);\n> +\t\t}\n>  \n> -\t\t\tdelta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height);\n> +\t\tdelta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height);\n>  \n> -\t\t\tif (delta < best_in_range_delta) {\n> -\t\t\t\tbest_in_range_delta = delta;\n> -\t\t\t\tbest_in_range = i;\n> +\t\tFormatMatch &match = gst_libcamera_structure_is_raw_capture(s) ? raw : non_raw;\n> +\t\tmatch.seen = true;\n> +\t\tif (is_fixed) {\n> +\t\t\tif (delta < match.fixed_delta) {\n> +\t\t\t\tmatch.fixed_delta = delta;\n> +\t\t\t\tmatch.fixed = i;\n>  \t\t\t}\n> +\t\t} else if (delta < match.in_range_delta) {\n> +\t\t\tmatch.in_range_delta = delta;\n> +\t\t\tmatch.in_range = i;\n>  \t\t}\n>  \t}\n>  \n> -\t/* Prefer reliable fixed value over ranges */\n> -\tif (best_fixed >= 0)\n> -\t\ts = gst_caps_get_structure(caps, best_fixed);\n> -\telse\n> -\t\ts = gst_caps_get_structure(caps, best_in_range);\n> +\t/*\n> +\t * Raw foramts are likely to be useful only to applications that\n> +\t * specifically ask for them. Other applications will typically be\n> +\t * unable to handle them and fail. Therefore return a non-raw match\n> +\t * if we have one, falling back to raw formats if that's all that\n> +\t * was requested. Note how both raw and non-raw variants preserve\n> +\t * the previous \"prefer fixed\" behaviour in their own right.\n> +\t */\n> +\tgint best = non_raw.best() >= 0 ? non_raw.best() : raw.best();\n> +\n> +\tif (best < 0) {\n> +\t\tGST_WARNING(\"Failed to find a suitable caps structure to configure the stream\");\n> +\t\treturn false;\n> +\t}\n> +\n> +\ts = gst_caps_get_structure(caps, best);\n\nI believe I understand the heuristic, but its getting complicated and will be\nhard to extend int the future. Again, not against anything, takes this softly\nbased on the time you have.\n\nWhat we do in v4lsrc, is that we gather each structures into a sorted list, you\ncan search for gst_v4l2src_fixed_caps_compare().\n\nNicolas\n\n>  \n>  \tif (gst_structure_has_name(s, \"video/x-raw\")) {\n>  \t\tconst gchar *format = gst_video_format_to_string(gst_format);\n> @@ -529,6 +575,8 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n>  \n>  \t\tstream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry, transfer);\n>  \t}\n> +\n> +\treturn true;\n>  }\n>  \n>  void gst_libcamera_get_framerate_from_caps(GstCaps *caps,\n> diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h\n> index 35df56fb..5ce0e839 100644\n> --- a/src/gstreamer/gstlibcamera-utils.h\n> +++ b/src/gstreamer/gstlibcamera-utils.h\n> @@ -18,7 +18,7 @@\n>  GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats);\n>  GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg,\n>  \t\t\t\t\t\t    GstVideoTransferFunction transfer);\n> -void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg,\n> +bool gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg,\n>  \t\t\t\t\t      GstCaps *caps, GstVideoTransferFunction *transfer);\n>  void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *element_caps);\n>  void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls,\n> diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp\n> index 9061f916..66b31ce8 100644\n> --- a/src/gstreamer/gstlibcamerasrc.cpp\n> +++ b/src/gstreamer/gstlibcamerasrc.cpp\n> @@ -605,7 +605,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)\n>  \n>  \t\t/* Fixate caps and configure the stream. */\n>  \t\tcaps = gst_caps_make_writable(caps);\n> -\t\tgst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]);\n> +\t\tif (!gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]))\n> +\t\t\treturn false;\n>  \t\tgst_libcamera_get_framerate_from_caps(caps, element_caps);\n>  \t}\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3FB8AC330A\n\tfor ;\n\tThu, 2 Jul 2026 19:59:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 3382965FBC;\n\tThu, 2 Jul 2026 21:59:01 +0200 (CEST)","from mail-ua1-x929.google.com (mail-ua1-x929.google.com\n\t[IPv6:2607:f8b0:4864:20::929])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 99A4D65FB5\n\tfor ;\n\tThu, 2 Jul 2026 21:58:58 +0200 (CEST)","by mail-ua1-x929.google.com with SMTP id\n\ta1e0cc1a2514c-963a722ed59so688515241.1\n\tfor ;\n\tThu, 02 Jul 2026 12:58:58 -0700 (PDT)","from ?IPv6:2606:6d00:15:e06b::5ac? ([2606:6d00:15:e06b::5ac])\n\tby smtp.gmail.com with ESMTPSA id\n\ta1e0cc1a2514c-96983a0b1efsm1486206241.12.2026.07.02.12.58.55\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 02 Jul 2026 12:58:56 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=ndufresne-ca.20251104.gappssmtp.com\n\theader.i=@ndufresne-ca.20251104.gappssmtp.com\n\theader.b=\"a2mIhibX\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ndufresne-ca.20251104.gappssmtp.com; s=20251104; t=1783022337;\n\tx=1783627137; darn=lists.libcamera.org; \n\th=mime-version:user-agent:content-type:autocrypt:references\n\t:in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date\n\t:message-id:reply-to:content-type;\n\tbh=ycQ+8oMLgXn6C1ZCXNf82fzPd+t/8LYQAGTrpL3NXrQ=;\n\tb=a2mIhibXzPOnifFnfmHttwFy9kNt4OnhE426bv9yayGdF+GaekzHp34X1ruDbDZ1eO\n\t+tPfKce2LJFxyHb6tKhDNSwz/N+oK1Ikn/ElsTdzkhC3OPhYI7AkAVzQYXakgOjaPUiA\n\twoLzcXnYmFUfF5i58+3KW2zD+9GfSsQxVgVx7euiM0dq/CvzRgGcjtmYp7W0Q3HJSrP2\n\tl0XXaT/kqsiqC02C30Ufximv0EVyqKPrY3C1rDGhfIMI+eueCsEK/eQ4VBaAl3FxV2Z3\n\tPuUFpvsS6LZ+MQcELsBfIGrhfu65TzsViMRYix2jVYINjDmgtZMf8avwxuxVTC/8GFLr\n\tK6/Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20251104; t=1783022337; x=1783627137;\n\th=mime-version:user-agent:content-type:autocrypt:references\n\t:in-reply-to:date:to:from:subject:message-id:x-gm-gg\n\t:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to\n\t:content-type;\n\tbh=ycQ+8oMLgXn6C1ZCXNf82fzPd+t/8LYQAGTrpL3NXrQ=;\n\tb=SLzH2tMkLHDKOA0jqgSS/04VK+ysHt/OJqE9b1ptVsTm8HVVZ64F4PnZ87eN4jNubc\n\t2IWyy+ZYVJI3u1d20uSoljbwMIB9H63Lp3my+WooZ9//XbThLFQZsSQc3JxKzSw3ejSA\n\tuLHgf8g+2Hc5G8T6tzUfHmjcIWTKwwJ5GVWl/5kYcfXGHnRk7LSmMF3WQXJpn1pj68Ug\n\tQ1xdX+VmpmGZLarn0giWxdDvom9JIza458nraf6Mym6zATfQZfTTRBmxvNFq4fkTBW8e\n\tC+iDv5xHb2QANHCsBqRJ67QEA+fP4bzhZUhdizhoCVir7/5cY2fvZUhLhQQU2J4RY2WM\n\tz1tw==","X-Forwarded-Encrypted":"i=1;\n\tAHgh+Rp9cOGofM5giGg86Umy5qoWUY/4Dl0bge/CNg/tuZOXOXwUpq1frFP52UMbRreuu4ssqsYJSZepty/vmRZciYs=@lists.libcamera.org","X-Gm-Message-State":"AOJu0YxfiPQVZ7+I7+u0uqqRNIUTABmeljmqk4GcA48oEUPbMO1c5IsP\n\tnXGlNCsqrBpIKbEJmBEQ30Dma9WQwJhUTiTxKGQ1rrkhaUa9aF3m3K2PGcYFzIaV+3U=","X-Gm-Gg":"AfdE7cmk4l2kJW20sZ5DuFojE3JBJiw8zn0fiDi7bZAfhGK9r5xwD9CfNDC5/S7ZQLI\n\thzC8pMGUbIE/yrgMLJS5IltY6r249oyPL0e0Uh8+eUbMnFgE3Ea+ddv1xh919ygMgOnghaOfvjv\n\tCVLlwDvxQ/W9E0ratrcKF3fUqu69LI7DDJA98RCEIFNmvBxNtm7Dxqst3pffZJUA9t/ZofzM1ta\n\tS+yKvqYo8e73hUv3rCqPHAzVc3jhqn2BF4GLPdKIihB5GoQCooK+lc5MCJNj2xIuNM3cH1Lklbr\n\tYjEGjDlpSxlltGJV7UMjCyeluE26is3+z5aAgL52P8iLf9CuicLVYLccf8PUg/0GKhd8pt4pw9U\n\tUQ1Ei70Za1WV0wXoHF0U/a1OqIRoQoqNGHA5I+ehBCt1B44VdsQgMf3WVRXbX2btplJlp3TkfTL\n\t9ZQ3obUMPt1r/GmGXe8g==","X-Received":"by 2002:a05:6102:41ac:b0:737:e354:4092 with SMTP id\n\tada2fe7eead31-73da7cc9befmr3097624137.11.1783022336952; \n\tThu, 02 Jul 2026 12:58:56 -0700 (PDT)","Message-ID":"<8d59fb03154497a008a7c7761e13d17df22074db.camel@ndufresne.ca>","Subject":"Re: [PATCH 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats\n\tin caps negotiation","From":"Nicolas Dufresne ","To":"David Plowman , \n\tlibcamera-devel@lists.libcamera.org","Date":"Thu, 02 Jul 2026 15:58:54 -0400","In-Reply-To":"<20260702145317.86482-2-david.plowman@raspberrypi.com>","References":"<20260702145317.86482-1-david.plowman@raspberrypi.com>\n\t<20260702145317.86482-2-david.plowman@raspberrypi.com>","Autocrypt":"addr=nicolas@ndufresne.ca; prefer-encrypt=mutual;\n\tkeydata=mDMEaCN2ixYJKwYBBAHaRw8BAQdAM0EHepTful3JOIzcPv6ekHOenE1u0vDG1gdHFrChD\n\t/e0J05pY29sYXMgRHVmcmVzbmUgPG5pY29sYXNAbmR1ZnJlc25lLmNhPoicBBMWCgBEAhsDBQsJCA\n\tcCAiICBhUKCQgLAgQWAgMBAh4HAheABQkJZfd1FiEE7w1SgRXEw8IaBG8S2UGUUSlgcvQFAmibrjo\n\tCGQEACgkQ2UGUUSlgcvQlQwD/RjpU1SZYcKG6pnfnQ8ivgtTkGDRUJ8gP3fK7+XUjRNIA/iXfhXMN\n\tabIWxO2oCXKf3TdD7aQ4070KO6zSxIcxgNQFtDFOaWNvbGFzIER1ZnJlc25lIDxuaWNvbGFzLmR1Z\n\tnJlc25lQGNvbGxhYm9yYS5jb20+iJkEExYKAEECGwMFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4\n\tAWIQTvDVKBFcTDwhoEbxLZQZRRKWBy9AUCaCyyxgUJCWX3dQAKCRDZQZRRKWBy9ARJAP96pFmLffZ\n\tsmBUpkyVBfFAf+zq6BJt769R0al3kHvUKdgD9G7KAHuioxD2v6SX7idpIazjzx8b8rfzwTWyOQWHC\n\tAAS0LU5pY29sYXMgRHVmcmVzbmUgPG5pY29sYXMuZHVmcmVzbmVAZ21haWwuY29tPoiZBBMWCgBBF\n\tiEE7w1SgRXEw8IaBG8S2UGUUSlgcvQFAmibrGYCGwMFCQll93UFCwkIBwICIgIGFQoJCAsCBBYCAw\n\tECHgcCF4AACgkQ2UGUUSlgcvRObgD/YnQjfi4+L8f4fI7p1pPMTwRTcaRdy6aqkKEmKsCArzQBAK8\n\tbRLv9QjuqsE6oQZra/RB4widZPvphs78H0P6NmpIJ","Content-Type":"multipart/signed; micalg=\"pgp-sha512\";\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"=-GfhjBhiOdkHhxGgralZh\"","User-Agent":"Evolution 3.60.2 (3.60.2-1.fc44) ","MIME-Version":"1.0","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":39570,"web_url":"https://patchwork.libcamera.org/comment/39570/","msgid":"","date":"2026-07-03T15:21:13","subject":"Re: [PATCH 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats\n\tin caps negotiation","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Thanks for the suggestions.\n\nYes, I think keeping a sorted list might be clearer, and all the\ncomparison logic is then explicitly in one place, in the compare\nfunction. I'll have a look at that when I'm back from holiday.\n\nThanks\nDavid\n\nOn Thu, 2 Jul 2026 at 20:58, Nicolas Dufresne wrote:\n>\n> Hi David,\n>\n> thanks again for working on this.\n>\n> Le jeudi 02 juillet 2026 à 15:39 +0100, David Plowman a écrit :\n> > Existing code was preferring fixed size outputs rather than ones with\n> > ranges, resulting in raw output formats being preferred even when\n> > applications (\"cheese\" is one such example) aren't expecting them and\n> > subsequently fail.\n> >\n> > The revised version looks at raw (in the Bayer sense) and non-raw\n> > formats separately, preferring fixed size outputs within each\n> > category, but then will opt for the non-raw formats if any were\n> > available. This still allows applications that explicitly request raw\n> > formats to get them.\n> >\n> > Fixes, for example, \"cheese\" on Raspberry Pi.\n> >\n> > Signed-off-by: David Plowman \n> > ---\n> > src/gstreamer/gstlibcamera-utils.cpp | 96 +++++++++++++++++++++-------\n> > src/gstreamer/gstlibcamera-utils.h | 2 +-\n> > src/gstreamer/gstlibcamerasrc.cpp | 3 +-\n> > 3 files changed, 75 insertions(+), 26 deletions(-)\n> >\n> > diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp\n> > index 6541d478..4f87525f 100644\n> > --- a/src/gstreamer/gstlibcamera-utils.cpp\n> > +++ b/src/gstreamer/gstlibcamera-utils.cpp\n> > @@ -435,20 +435,54 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg\n> > return caps;\n> > }\n> >\n> > -void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n> > +/*\n> > + * We will want to distinguish between caps structures corresponding to\n> > + * raw and non-raw (processed) output images, as this will help inform\n> > + * our choice of preferred format.\n> > + *\n> > + * We identify Bayer as being the principle \"raw\" format here, but note\n> > + * that we are considering greyscale to be raw too (e.g. a raw\n> > + * monochrome sensor).\n> > + */\n> > +static bool\n> > +gst_libcamera_structure_is_raw_capture(const GstStructure *s)\n> > +{\n> > + if (gst_structure_has_name(s, \"video/x-bayer\"))\n> > + return true;\n> > +\n> > + if (gst_structure_has_name(s, \"video/x-raw\")) {\n> > + const gchar *format = gst_structure_get_string(s, \"format\");\n> > +\n> > + if (format && (!strcmp(format, \"GRAY8\") || !strcmp(format, \"GRAY16_LE\") ||\n> > + !strcmp(format, \"GRAY10_LE16\")))\n> > + return true;\n>\n> A more generic solution could be something like:\n>\n> finfo = gst_video_format_get_info(gst_video_format_from_string(format));\n> if (finfo->n_components == 1)\n> return true;\n>\n> > + }\n> > +\n> > + return false;\n> > +}\n> > +\n> > +bool gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n> > GstCaps *caps, GstVideoTransferFunction *transfer)\n> > {\n> > GstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat);\n> > guint i;\n> > - gint best_fixed = -1, best_in_range = -1;\n> > GstStructure *s;\n> >\n> > /*\n> > - * These are delta weight computed from:\n> > - * ABS(width - stream_cfg.size.width) * ABS(height - stream_cfg.size.height)\n> > + * Track the best size match (by delta weight, see below) separately\n> > + * for raw sensor capture structures (Bayer/mono) and non-raw ones.\n> > + * The previous logic that preferred fixed sizes over a range is\n> > + * preserved within each of these \"families\".\n> > */\n> > - guint best_fixed_delta = G_MAXUINT;\n> > - guint best_in_range_delta = G_MAXUINT;\n> > + struct FormatMatch {\n> > + bool seen = false;\n> > + gint fixed = -1;\n> > + guint fixed_delta = G_MAXUINT;\n> > + gint in_range = -1;\n> > + guint in_range_delta = G_MAXUINT;\n> > +\n> > + gint best() const { return fixed >= 0 ? fixed : in_range; }\n> > + } raw, non_raw;\n> >\n> > /* First fixate the caps using default configuration value. */\n> > g_assert(gst_caps_is_writable(caps));\n> > @@ -458,38 +492,50 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n> > s = gst_caps_get_structure(caps, i);\n> > gint width, height;\n> > guint delta;\n> > + bool is_fixed = gst_structure_has_field_typed(s, \"width\", G_TYPE_INT) &&\n> > + gst_structure_has_field_typed(s, \"height\", G_TYPE_INT);\n> >\n> > - if (gst_structure_has_field_typed(s, \"width\", G_TYPE_INT) &&\n> > - gst_structure_has_field_typed(s, \"height\", G_TYPE_INT)) {\n> > + if (is_fixed) {\n> > gst_structure_get_int(s, \"width\", &width);\n> > gst_structure_get_int(s, \"height\", &height);\n> > -\n> > - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height);\n> > -\n> > - if (delta < best_fixed_delta) {\n> > - best_fixed_delta = delta;\n> > - best_fixed = i;\n> > - }\n> > } else {\n> > gst_structure_fixate_field_nearest_int(s, \"width\", stream_cfg.size.width);\n> > gst_structure_fixate_field_nearest_int(s, \"height\", stream_cfg.size.height);\n> > gst_structure_get_int(s, \"width\", &width);\n> > gst_structure_get_int(s, \"height\", &height);\n> > + }\n> >\n> > - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height);\n> > + delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height);\n> >\n> > - if (delta < best_in_range_delta) {\n> > - best_in_range_delta = delta;\n> > - best_in_range = i;\n> > + FormatMatch &match = gst_libcamera_structure_is_raw_capture(s) ? raw : non_raw;\n> > + match.seen = true;\n> > + if (is_fixed) {\n> > + if (delta < match.fixed_delta) {\n> > + match.fixed_delta = delta;\n> > + match.fixed = i;\n> > }\n> > + } else if (delta < match.in_range_delta) {\n> > + match.in_range_delta = delta;\n> > + match.in_range = i;\n> > }\n> > }\n> >\n> > - /* Prefer reliable fixed value over ranges */\n> > - if (best_fixed >= 0)\n> > - s = gst_caps_get_structure(caps, best_fixed);\n> > - else\n> > - s = gst_caps_get_structure(caps, best_in_range);\n> > + /*\n> > + * Raw foramts are likely to be useful only to applications that\n> > + * specifically ask for them. Other applications will typically be\n> > + * unable to handle them and fail. Therefore return a non-raw match\n> > + * if we have one, falling back to raw formats if that's all that\n> > + * was requested. Note how both raw and non-raw variants preserve\n> > + * the previous \"prefer fixed\" behaviour in their own right.\n> > + */\n> > + gint best = non_raw.best() >= 0 ? non_raw.best() : raw.best();\n> > +\n> > + if (best < 0) {\n> > + GST_WARNING(\"Failed to find a suitable caps structure to configure the stream\");\n> > + return false;\n> > + }\n> > +\n> > + s = gst_caps_get_structure(caps, best);\n>\n> I believe I understand the heuristic, but its getting complicated and will be\n> hard to extend int the future. Again, not against anything, takes this softly\n> based on the time you have.\n>\n> What we do in v4lsrc, is that we gather each structures into a sorted list, you\n> can search for gst_v4l2src_fixed_caps_compare().\n>\n> Nicolas\n>\n> >\n> > if (gst_structure_has_name(s, \"video/x-raw\")) {\n> > const gchar *format = gst_video_format_to_string(gst_format);\n> > @@ -529,6 +575,8 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n> >\n> > stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry, transfer);\n> > }\n> > +\n> > + return true;\n> > }\n> >\n> > void gst_libcamera_get_framerate_from_caps(GstCaps *caps,\n> > diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h\n> > index 35df56fb..5ce0e839 100644\n> > --- a/src/gstreamer/gstlibcamera-utils.h\n> > +++ b/src/gstreamer/gstlibcamera-utils.h\n> > @@ -18,7 +18,7 @@\n> > GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats);\n> > GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg,\n> > GstVideoTransferFunction transfer);\n> > -void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg,\n> > +bool gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg,\n> > GstCaps *caps, GstVideoTransferFunction *transfer);\n> > void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *element_caps);\n> > void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls,\n> > diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp\n> > index 9061f916..66b31ce8 100644\n> > --- a/src/gstreamer/gstlibcamerasrc.cpp\n> > +++ b/src/gstreamer/gstlibcamerasrc.cpp\n> > @@ -605,7 +605,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)\n> >\n> > /* Fixate caps and configure the stream. */\n> > caps = gst_caps_make_writable(caps);\n> > - gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]);\n> > + if (!gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]))\n> > + return false;\n> > gst_libcamera_get_framerate_from_caps(caps, element_caps);\n> > }\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 86194C328C\n\tfor ;\n\tFri, 3 Jul 2026 15:21:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 696D165FC1;\n\tFri, 3 Jul 2026 17:21:27 +0200 (CEST)","from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com\n\t[IPv6:2a00:1450:4864:20::52a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C433B65F9F\n\tfor ;\n\tFri, 3 Jul 2026 17:21:25 +0200 (CEST)","by mail-ed1-x52a.google.com with SMTP id\n\t4fb4d7f45d1cf-698b899b4c8so657798a12.0\n\tfor ;\n\tFri, 03 Jul 2026 08:21:25 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"Ct5lywN+\"; dkim-atps=neutral","ARC-Seal":"i=1; a=rsa-sha256; t=1783092085; cv=none;\n\td=google.com; s=arc-20260327;\n\tb=V0Z3+SrPuwhiG7h1aNn6SzPINAVayjEC3jlMngapQaCQewkuEV/cZgilfqshTq+5JX\n\tNcFPrVFKgdZ4TvKoMEDDPGLpQORgOyL4d8CrZEbPYUbssR4qDEnvBJEwNWkimhyEI74H\n\tRabjeWafjwNsP/r2B71in1HlgkwtTt2sH9EqNECO6+JoesCKmuZlwToAC5k95/hng3vq\n\tUwJ3uoKNMzf68g3isqb5PuneIM8TmHG6EY6RS6SgoXLSPpXGXp71nejMCkm9WM7Txmif\n\tN7sbfpph+tueSgEngnw7fhGatLX+Wkt6rHbCmI29Xq0VDURO7m/ZYT/kfQ69yp8auwgN\n\tkJkQ==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=arc-20260327; \n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:dkim-signature;\n\tbh=P45ItppFfJRqPqT+YXczZt9Ad97jqYnbIkuAspv1srs=;\n\tfh=d8+zo8pV0/m/b60f8il+KyfgQD9k6syIaaxOgkmf2uI=;\n\tb=kK7QrXBVw39NtJJvwoznpr24aglaTX5M76Iv6ol0M2jhphOWVihNz7QPTxitxszjf5\n\taWPWKOCO1MHWd2+vCB+p5DAlkDsH0u2oF5j6bhWJvNpowTBaeC3PadB8+ejF5ks6e9zI\n\tzRsWY08IXLfDcCae7xypvTFBGyClaJ/zdqT8tatSZ6Ex/Y/beDo3bbZ0Uv+VqoSBPvCb\n\tTTcistLgv4cS3vb8e1rAMZYkJ5QI5LqcJcMIMLiSVpdc9b6kwvOYmeh748wcdKkT9bhN\n\t0oH5OA5g9z0XpbQ0+I/MnDpPeJ48McmUXaSrI/BzmMdQ65WIxx5DHuMehSUfh95iuibO\n\tO4hg==; darn=lists.libcamera.org","ARC-Authentication-Results":"i=1; mx.google.com; arc=none","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1783092085; x=1783696885;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=P45ItppFfJRqPqT+YXczZt9Ad97jqYnbIkuAspv1srs=;\n\tb=Ct5lywN+XXsZz2NIHNshnKWG15Af+mAMhKdGcmltClWIf5Vbb+3DJ8Kzpi/vJCc1X8\n\tuXrU2RKAwWKrfyd2MyeZd+LQTBnDrcyAEDqwMp5JDuSDGbkGtLeXsgWtyiJYREhId1vz\n\t3D3A9+2qBljkV+f4qGUhbhTJtr8qCbmH9m/OTnkCETBgoMYI5flcP3fwLZrCTImjaVVV\n\tUp0SmsIv5inXeotfjD44CeyDyU81xuMUR1amxtKcZiqLwdGyJHvt8j5BP98ZE2XFb+xf\n\tUwzLKeZE6qc2H35BF3FU5nZj9lKvZ0g6zakH4pDxrM0+XOnkuh8U2RUTvaKmOTWbCYPa\n\tRt4g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20251104; t=1783092085; x=1783696885;\n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=P45ItppFfJRqPqT+YXczZt9Ad97jqYnbIkuAspv1srs=;\n\tb=FHCH+5bZMIQPxY7fXbbUwGFTBxBtJ57Zi5SfaM3kjZmSQbQM4HPKIu7WS4UQsrFhmH\n\tL87D3gyc5KzhgVY5CIOPtMAZGbM0yTBxMtU5WRi/EmaMoMdbNlHKKcAj9O2tBm1gJXD/\n\tdM4gKTvZ05/fDG8eouQ+pvJazVjkXnq/2O5w6RVbGzwukDPoMHHC6UKwltJjN8NK8mag\n\t+QB2ScaUSG53wXn3JEEunQY6YB6lx62g+LpdAPqVkC3MIiNAyIWkvFTR/Q7P0b1Qq0XL\n\tTfNhm02oEpQTXR845lHVQ6CPam10tl24m8hcmwsvMzLizRB23u3Mz4543CnSi3TONoLE\n\t5H7g==","X-Gm-Message-State":"AOJu0YyoLBGE4e299yzzMbo6s0tWgHSNPDTxdrBlWs5OLVrI+GPHwxhM\n\tI837sxFGctrEojXq0uIuz4u5/sY11qoy6IXgFZp5Jx2nG5xzlcBqVTRvOfUf5EW57tJ8lFssSnc\n\tWTr9TRu4gkFJGY29eCY2wUkJ9K0zHtxGPCCYwzxbg9g==","X-Gm-Gg":"AfdE7clTe1UTYtM/DIPWEgt8SNl3LDsCutSMd6h3fLCjuXR/nQBiqYvydO8GVdXCjKc\n\t30jhbzOxY7G0FM/Lcp8JdCMrEqAd/JTeR3ZoVbZsljgT0CpuD52qywctP+OpmheA5+Ru6kMEZ46\n\t+zei5f0RJ8DOPbRBkFjtd/bR8ANRl0jID8KfcmBL0anvnPIzgT6gvTxBiPfypKgRZQqgquqKf2D\n\tSiYFPGigmygDnIyrosvwL8AybjTR9/4MXkdZBQ1Kw6TWNUCZQakxctrkIA/m+ySxGzoa23a8Rom\n\te2IpqrRdZz5kn9Z8FqPCbJHWgdKFH9ub8CwoRglbtfqWbYyUbcB8J9IiWsQ+YGSZ2KJPPdOEK0a\n\tcLsPz3YJB3UKCPg==","X-Received":"by 2002:a05:6402:5347:10b0:69a:f82:ccdc with SMTP id\n\t4fb4d7f45d1cf-69a175b4c99mr101318a12.19.1783092085046;\n\tFri, 03 Jul 2026 08:21:25 -0700 (PDT)","MIME-Version":"1.0","References":"<20260702145317.86482-1-david.plowman@raspberrypi.com>\n\t<20260702145317.86482-2-david.plowman@raspberrypi.com>\n\t<8d59fb03154497a008a7c7761e13d17df22074db.camel@ndufresne.ca>","In-Reply-To":"<8d59fb03154497a008a7c7761e13d17df22074db.camel@ndufresne.ca>","From":"David Plowman ","Date":"Fri, 3 Jul 2026 16:21:13 +0100","X-Gm-Features":"AVVi8Ccpe4Ao04u0DB6PtlMS7NPKgV0FGeFgJU9OEWV0hA0SC8E934d_Xs9uyZo","Message-ID":"","Subject":"Re: [PATCH 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats\n\tin caps negotiation","To":"Nicolas Dufresne ","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]