From patchwork Thu Jul 2 14:39:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 27157 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 601B3C3303 for ; Thu, 2 Jul 2026 14:53:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 608A165FBD; Thu, 2 Jul 2026 16:53:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kVA3/jGy"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E04A65FB5 for ; Thu, 2 Jul 2026 16:53:20 +0200 (CEST) Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-475cb71a4ebso1846093f8f.0 for ; Thu, 02 Jul 2026 07:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1783004000; x=1783608800; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kGqRU7MvJXzSfoJQ6NDjSBTGu44ltje+wZp2HcrA8qQ=; b=kVA3/jGyEhcUtOppnvP1RB431jcY9w/SWvObdnncnGcKJ1d6ikbBjymLiq1Gv+QJJE sgBygHfeMx9cMsIKWmRGMZ+a+MfmF0Ld1YZKCp6g/89Ynqqs+JFJgNLKjI9syr/U0Pwh wH7dt+ln1iKhBOri2h2FhHQBHXwgxIqLFszJLDsHdJQC/QtKP87xytFzt7z1DZtldUVR MZQmpj/CvLPqVspALvWLhfGaqr8xjdTRe69zmzfjjO4sNYeV6ve61D5M81tjOS+M++aI BxHVr1VLd7lLJ4GnJ032W6N8I90gIKFLx9lXGQT5sD/ij3zK2SX+VskHGFvvJPABBNib XfQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783004000; x=1783608800; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=kGqRU7MvJXzSfoJQ6NDjSBTGu44ltje+wZp2HcrA8qQ=; b=Vmwqw9iA2AG0SYZD5OS/MM96vsOw/yW5zVN9fL24s5IgQ3ASaKJosGnQkwBwxJcac9 n//6z+ikroyI55pKzWkuZfaLCTFOJLBGsroNhGJewyeSzVnIboRhq6JhYP1YfxKLEWKB oxf/2FERP2fVFxnIaQ6N3+ckSA/Q6oeOTQBdmRdWyxkTigd0wqfN8TreZzieJGmBDr8v jkL6nDd/fZDLB1O50npnDzovqZ9P+WK8g7Bs7kBwaynwOjAG1nlj8WR4D1iNKERVMrGx +EXOxeEMpB+jcyZCG24mgXK3qQk9M4K9jFelOM0OfDU1baRifex441m6n5zm3P7bZEAD Wp9Q== X-Gm-Message-State: AOJu0YzMnJcInGcp7QA884tkuLWfG5QRI0MSoTt+vMqt2gO+anoA5DlR wyOGkscrlJ3S1qSUwfsgXNL367vwzuaYVhmA17RmyjQTtaoJ0ls/d098ZcZ8Thki2Db+AUzza7Q 1Y4TZcnc= X-Gm-Gg: AfdE7clzt117MFbiXxr1koD88wrD+yDNRMnXRlrU9pSLqXS5GAIaSKP7imr0hmthEO9 NdggiKZ1UiT3RF654Ja+X8Dxz8Vo9COhqzagxll9vhTfn62wwkKvKPI0zLFz/Xm9h0B4+11P5QV 099WZq4losQnBMl+Aca2JnHz41GfYJXKWLiJtFeQPuLsP2QZI2rlVqj7JPV1VAt0TnvJJVTIQtl hueGPUfgV5fIVpJXk7Jc63MfQowbXbKJeklbizlgxVhui1MpKO1CA8cTUTMHtWm/XuIigPOYZUI T6R9ODjQL/DUcTjE9RSt+sQxKgDuYqfYcJPaVl/vQlq4P/1m+xsif7xNFMP9sGf4+OHa9wNpwt7 9ZxCtWyEu4PGp/s4CeLgSPtR+ZHoBD/q3/uIS+FZVZYahVTqGGATgyJZHzw8yxP3ZfcLXDPrabQ FRp4s6+PIIuk3LHAP9AFBJ4n6wk8tZaFS23IZqj/orUwCAOFiRHEI8tWwINDYUS7QcLxgK2prTc cxgEejKkQaylAk82CxA3Rv3dKZW44vZ X-Received: by 2002:a05:6000:2406:b0:45e:f684:7347 with SMTP id ffacd0b85a97d-47757e577f6mr11045672f8f.12.1783003999867; Thu, 02 Jul 2026 07:53:19 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:4706:89ee:c47e:1087]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-477db3db964sm9419149f8f.8.2026.07.02.07.53.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jul 2026 07:53:19 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats in caps negotiation Date: Thu, 2 Jul 2026 15:39:17 +0100 Message-ID: <20260702145317.86482-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260702145317.86482-1-david.plowman@raspberrypi.com> References: <20260702145317.86482-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Existing code was preferring fixed size outputs rather than ones with ranges, resulting in raw output formats being preferred even when applications ("cheese" is one such example) aren't expecting them and subsequently fail. The revised version looks at raw (in the Bayer sense) and non-raw formats separately, preferring fixed size outputs within each category, but then will opt for the non-raw formats if any were available. This still allows applications that explicitly request raw formats to get them. Fixes, for example, "cheese" on Raspberry Pi. Signed-off-by: David Plowman --- src/gstreamer/gstlibcamera-utils.cpp | 96 +++++++++++++++++++++------- src/gstreamer/gstlibcamera-utils.h | 2 +- src/gstreamer/gstlibcamerasrc.cpp | 3 +- 3 files changed, 75 insertions(+), 26 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 6541d478..4f87525f 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -435,20 +435,54 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg return caps; } -void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, +/* + * We will want to distinguish between caps structures corresponding to + * raw and non-raw (processed) output images, as this will help inform + * our choice of preferred format. + * + * We identify Bayer as being the principle "raw" format here, but note + * that we are considering greyscale to be raw too (e.g. a raw + * monochrome sensor). + */ +static bool +gst_libcamera_structure_is_raw_capture(const GstStructure *s) +{ + if (gst_structure_has_name(s, "video/x-bayer")) + return true; + + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_structure_get_string(s, "format"); + + if (format && (!strcmp(format, "GRAY8") || !strcmp(format, "GRAY16_LE") || + !strcmp(format, "GRAY10_LE16"))) + return true; + } + + return false; +} + +bool gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, GstCaps *caps, GstVideoTransferFunction *transfer) { GstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat); guint i; - gint best_fixed = -1, best_in_range = -1; GstStructure *s; /* - * These are delta weight computed from: - * ABS(width - stream_cfg.size.width) * ABS(height - stream_cfg.size.height) + * Track the best size match (by delta weight, see below) separately + * for raw sensor capture structures (Bayer/mono) and non-raw ones. + * The previous logic that preferred fixed sizes over a range is + * preserved within each of these "families". */ - guint best_fixed_delta = G_MAXUINT; - guint best_in_range_delta = G_MAXUINT; + struct FormatMatch { + bool seen = false; + gint fixed = -1; + guint fixed_delta = G_MAXUINT; + gint in_range = -1; + guint in_range_delta = G_MAXUINT; + + gint best() const { return fixed >= 0 ? fixed : in_range; } + } raw, non_raw; /* First fixate the caps using default configuration value. */ g_assert(gst_caps_is_writable(caps)); @@ -458,38 +492,50 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, s = gst_caps_get_structure(caps, i); gint width, height; guint delta; + bool is_fixed = gst_structure_has_field_typed(s, "width", G_TYPE_INT) && + gst_structure_has_field_typed(s, "height", G_TYPE_INT); - if (gst_structure_has_field_typed(s, "width", G_TYPE_INT) && - gst_structure_has_field_typed(s, "height", G_TYPE_INT)) { + if (is_fixed) { gst_structure_get_int(s, "width", &width); gst_structure_get_int(s, "height", &height); - - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); - - if (delta < best_fixed_delta) { - best_fixed_delta = delta; - best_fixed = i; - } } else { gst_structure_fixate_field_nearest_int(s, "width", stream_cfg.size.width); gst_structure_fixate_field_nearest_int(s, "height", stream_cfg.size.height); gst_structure_get_int(s, "width", &width); gst_structure_get_int(s, "height", &height); + } - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); + delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); - if (delta < best_in_range_delta) { - best_in_range_delta = delta; - best_in_range = i; + FormatMatch &match = gst_libcamera_structure_is_raw_capture(s) ? raw : non_raw; + match.seen = true; + if (is_fixed) { + if (delta < match.fixed_delta) { + match.fixed_delta = delta; + match.fixed = i; } + } else if (delta < match.in_range_delta) { + match.in_range_delta = delta; + match.in_range = i; } } - /* Prefer reliable fixed value over ranges */ - if (best_fixed >= 0) - s = gst_caps_get_structure(caps, best_fixed); - else - s = gst_caps_get_structure(caps, best_in_range); + /* + * Raw foramts are likely to be useful only to applications that + * specifically ask for them. Other applications will typically be + * unable to handle them and fail. Therefore return a non-raw match + * if we have one, falling back to raw formats if that's all that + * was requested. Note how both raw and non-raw variants preserve + * the previous "prefer fixed" behaviour in their own right. + */ + gint best = non_raw.best() >= 0 ? non_raw.best() : raw.best(); + + if (best < 0) { + GST_WARNING("Failed to find a suitable caps structure to configure the stream"); + return false; + } + + s = gst_caps_get_structure(caps, best); if (gst_structure_has_name(s, "video/x-raw")) { const gchar *format = gst_video_format_to_string(gst_format); @@ -529,6 +575,8 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry, transfer); } + + return true; } void gst_libcamera_get_framerate_from_caps(GstCaps *caps, diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 35df56fb..5ce0e839 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -18,7 +18,7 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg, GstVideoTransferFunction transfer); -void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, +bool gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, GstCaps *caps, GstVideoTransferFunction *transfer); void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *element_caps); void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls, diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 9061f916..66b31ce8 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -605,7 +605,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) /* Fixate caps and configure the stream. */ caps = gst_caps_make_writable(caps); - gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]); + if (!gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i])) + return false; gst_libcamera_get_framerate_from_caps(caps, element_caps); }