From patchwork Mon Jul 20 11:10:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 27388 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DE692C32FE for ; Mon, 20 Jul 2026 11:51:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6153667E37; Mon, 20 Jul 2026 13:51:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="M2Fjrmgu"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DD0366122 for ; Mon, 20 Jul 2026 13:51:49 +0200 (CEST) Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-49550ec592cso8132375e9.0 for ; Mon, 20 Jul 2026 04:51:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1784548309; x=1785153109; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fwgRAPskwfGif3jCVguiElbgNIjD4EIyAwHqT2FZV0s=; b=M2FjrmguOJW6wrZ3nNCYRoDu89ymC7eNiJNEBLeuKvlfkApaamHlg79Jiv6tofO2mQ 0dW8XhwVhMzOgU9U0z/Y81sW/4A38ywTiKYs/L2n2BJzijNVrYrcGoWkGPGI89LO8W++ 29ajs0u0ldq89aZCtjFIieAxHIyHLdKxHq5EzjrhA0yQm0mjCQ9d2MhD5tK/sky3XKq5 4ZTrYgm4qAgqG/rA6hPdAlYd6X5KX1oaM/yCq+zQ4pfYzG8M7m5LBF8mbwOOR7hf6fMk LhaIbDzkd5oMDpjSdYeK3eUkqH7H8saP0KvWdLB2sAJEep8cAVUOuUQMnhlHwUOw/+DP O6EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784548309; x=1785153109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fwgRAPskwfGif3jCVguiElbgNIjD4EIyAwHqT2FZV0s=; b=outHGcASvVbwg3ABaAY+Vt43zFuyPSOlcBWTz0v4vsA2jOp+44ZMNVIcQNANYUppJ2 UXJxbk9o0f3v/SJU+ELe6BBwtlSNYTrWHqMsWGCU1AI+/XuHc8nTnVyDKIgcWF+gDYOS I6rJUJrTGnN4NAzkpQiPTUKbBYPGcsn3T1hKl5Vg4n+08pxXpTNLsm26ZCVo4H98fkeg OoAhBXASCQ/W72pHR/FYpvNeLntAWAy4j/KKPH7SpRriUI3qriSXvAUVqSUi6VsaQlQa fz1cc2ZumtDiwZdQA/oxbFTGTmtCyb1DNxJROoNtBYdq7fKqEtAw8SAytHFNwyCuCWX+ Nztg== X-Gm-Message-State: AOJu0Yz8O6OdQnkTnQ/Nx2DmeYXfsdTFwq/KS3dGfxOirsdSgnNv60CV zM35msQYlqgBXxE0rZAz/nphCn/s9p23MHQYJqxpxR4PnfvaeU96UeZkh4GZnzPWy0QoyL2ol3T W+EjB X-Gm-Gg: AR+sD13BmFh+hXApRTj9PRWdOk5qrd4WkEcUOnOoLU2CE9OaT+ET2P9HOPtLNStn8y9 rB8jIm+wOmsi0XCOvQdJxz+o/aY6Riq6/R7tZxqn+HpFGBpzHuan93yNymFNk6UBHD5xMfL6OHf gdAIjdXmj4nnVmC58cS9eC4F6ZOo1tv70lq9lWA52HK3DJ6pOA5pJWMepZI2tsqxjstzdMhPT2H /M4avqAzb+EyuZrmgOzFGM2ETdI2vN7J2AyG7kK/7s3wJVVOm1UnUyU9UHXLXBeBw7JbbvT4KrV 7Ce16aoydOPs3nCaxprGcryqgho56L6OGT0OjhxLIbfagJZFsrHjVvXaHfwwkEjSAitJKHop3Xd wPk0P0Csv/RD3uHdq5jmwg43I9IwL4ecZRlU1ZkdIvW3t541Ypk/xHFM2o+fjZUEGp7jtko3Wtz 9jKxx9hUtM8azHdiRHBowfbb90KUUFpvjMLcsO5Xw5mvrj1w+xLVrl9l0AOE32rJ1TO7evm4xGY 5W5HUM97gKnuHLohPv9Mw== X-Received: by 2002:a05:6000:46d9:b0:47f:6e8b:699c with SMTP id ffacd0b85a97d-47f6e8b6bcdmr7226929f8f.14.1784548308540; Mon, 20 Jul 2026 04:51:48 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:4706:89ee:c47e:1087]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47f63ec7c69sm31270120f8f.17.2026.07.20.04.51.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 04:51:48 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v2 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats in caps negotiation Date: Mon, 20 Jul 2026 12:10:07 +0100 Message-ID: <20260720115144.4392-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260720115144.4392-1-david.plowman@raspberrypi.com> References: <20260720115144.4392-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Existing code was preferring fixed size outputs over ones with ranges, resulting in raw output formats being preferred even when applications ("cheese" is one such example) aren't expecting them and subsequently fail. The revised version still prefers fixed size outputs over ones with ranges, but will prefer non-raw (meaning non-Bayer) formats where these are available. This still allows applications that explicitly request raw formats to get them, but other applications will normally get non-raw formats which they handle better. Fixes, for example, "cheese" on Raspberry Pi. Signed-off-by: David Plowman --- src/gstreamer/gstlibcamera-utils.cpp | 125 ++++++++++++++++++++------- src/gstreamer/gstlibcamera-utils.h | 2 +- src/gstreamer/gstlibcamerasrc.cpp | 3 +- 3 files changed, 97 insertions(+), 33 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 6541d478c..f0f128880 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -9,6 +9,7 @@ #include "gstlibcamera-utils.h" #include +#include #include #include @@ -435,61 +436,121 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg return caps; } -void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, +/* + * We will want to distinguish between caps structures corresponding to + * raw and non-raw (processed) output images, as this will help inform + * our choice of preferred format. + * + * We identify Bayer as being the principle "raw" format here, but note + * that we are considering greyscale (images with one component) to be + * raw too (e.g. a raw monochrome sensor). + */ +static bool +gst_libcamera_structure_is_raw_capture(const GstStructure *s) +{ + if (gst_structure_has_name(s, "video/x-bayer")) + return true; + + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_structure_get_string(s, "format"); + if (!format) + return false; + + const GstVideoFormatInfo *finfo = + gst_video_format_get_info(gst_video_format_from_string(format)); + + return finfo->n_components == 1; + } + + return false; +} + +/* + * Data recorded for each caps structure candidate, used to rank them + * against each other in gst_libcamera_compare_caps_candidates(). + */ +struct CapsCandidate { + guint index; + bool is_raw; + bool is_fixed; + guint delta; +}; + +static gint +gst_libcamera_compare_caps_candidates(gconstpointer a, gconstpointer b) +{ + const CapsCandidate *ca = static_cast(a); + const CapsCandidate *cb = static_cast(b); + + /* + * Raw formats are likely to be useful only to applications that + * specifically ask for them. Other applications will typically be + * unable to handle them and fail, so prefer non-raw candidates. + */ + if (ca->is_raw != cb->is_raw) + return ca->is_raw ? 1 : -1; + + /* Prefer a reliable fixed value over a range. */ + if (ca->is_fixed != cb->is_fixed) + return ca->is_fixed ? -1 : 1; + + /* Otherwise the closest size match wins. */ + if (ca->delta != cb->delta) + return ca->delta < cb->delta ? -1 : 1; + + return 0; +} + +bool gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, GstCaps *caps, GstVideoTransferFunction *transfer) { GstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat); guint i; - gint best_fixed = -1, best_in_range = -1; GstStructure *s; - /* - * These are delta weight computed from: - * ABS(width - stream_cfg.size.width) * ABS(height - stream_cfg.size.height) - */ - guint best_fixed_delta = G_MAXUINT; - guint best_in_range_delta = G_MAXUINT; - /* First fixate the caps using default configuration value. */ g_assert(gst_caps_is_writable(caps)); - /* Lookup the structure for a close match to the stream_cfg.size */ + /* + * Build a candidate for every available caps structure, and insert + * it into a list kept sorted by preference as we go. The vector + * is constructed at its final size immediately, so it's safe to use + * pointers to the elements in it. + */ + std::vector candidates(gst_caps_get_size(caps)); + GList *sorted = nullptr; + for (i = 0; i < gst_caps_get_size(caps); i++) { s = gst_caps_get_structure(caps, i); gint width, height; - guint delta; + bool is_fixed = gst_structure_has_field_typed(s, "width", G_TYPE_INT) && + gst_structure_has_field_typed(s, "height", G_TYPE_INT); - if (gst_structure_has_field_typed(s, "width", G_TYPE_INT) && - gst_structure_has_field_typed(s, "height", G_TYPE_INT)) { + if (is_fixed) { gst_structure_get_int(s, "width", &width); gst_structure_get_int(s, "height", &height); - - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); - - if (delta < best_fixed_delta) { - best_fixed_delta = delta; - best_fixed = i; - } } else { gst_structure_fixate_field_nearest_int(s, "width", stream_cfg.size.width); gst_structure_fixate_field_nearest_int(s, "height", stream_cfg.size.height); gst_structure_get_int(s, "width", &width); gst_structure_get_int(s, "height", &height); + } - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); + guint delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); - if (delta < best_in_range_delta) { - best_in_range_delta = delta; - best_in_range = i; - } - } + candidates[i] = { i, gst_libcamera_structure_is_raw_capture(s), is_fixed, delta }; + sorted = g_list_insert_sorted(sorted, &candidates[i], gst_libcamera_compare_caps_candidates); + } + + if (!sorted) { + GST_WARNING("Failed to find a suitable caps structure to configure the stream"); + return false; } - /* Prefer reliable fixed value over ranges */ - if (best_fixed >= 0) - s = gst_caps_get_structure(caps, best_fixed); - else - s = gst_caps_get_structure(caps, best_in_range); + guint best = static_cast(sorted->data)->index; + g_list_free(sorted); + + s = gst_caps_get_structure(caps, best); if (gst_structure_has_name(s, "video/x-raw")) { const gchar *format = gst_video_format_to_string(gst_format); @@ -529,6 +590,8 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry, transfer); } + + return true; } void gst_libcamera_get_framerate_from_caps(GstCaps *caps, diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 35df56fb9..5ce0e839c 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -18,7 +18,7 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg, GstVideoTransferFunction transfer); -void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, +bool gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, GstCaps *caps, GstVideoTransferFunction *transfer); void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *element_caps); void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls, diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 9061f9163..66b31ce82 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -605,7 +605,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) /* Fixate caps and configure the stream. */ caps = gst_caps_make_writable(caps); - gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]); + if (!gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i])) + return false; gst_libcamera_get_framerate_from_caps(caps, element_caps); }