From patchwork Mon Aug 3 13:07:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 27601 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7D974C32CE for ; Mon, 3 Aug 2026 13:16:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E0A9E680FC; Mon, 3 Aug 2026 15:16:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gg/Bag1Z"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3BD9B68067 for ; Mon, 3 Aug 2026 15:16:19 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-47c6e9a694bso2215183f8f.1 for ; Mon, 03 Aug 2026 06:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1785762979; x=1786367779; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=0Xjvh+J+Hv9zC8LKMx3w8YUUSZW2rg9SjCfV5TpeCGw=; b=gg/Bag1Zonya94+r7wBX32hx1oyklEMSxPIzLyrKlmkFUyDVKMoL+naSrprQl7N9hw pJWO/jxPVaIrpwixsw7TXNmoopd2U6qQhXwWpuKEiv1u4KJxy4LtgtJYznIl2CRuE0+t G2FIVNONTTrPlBBnyvcLnmgMl7sk7L29wPeDVui1ZSkug8XqYH+po/Gaz+9UJMkgQa17 0fzrFjDj6Uuotmh+WGz5u7ojnqwL9E2wJ/cBtZb0kdtGlJFCHpXHlua0LnQgJE1FmS7J izDXH27YaPHPwN2RcJQZyAeDPbrP6fusDQcfxT6jPBGVBZO/J+xSTuCe+yXFqoBNcXg3 WH4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785762979; x=1786367779; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=0Xjvh+J+Hv9zC8LKMx3w8YUUSZW2rg9SjCfV5TpeCGw=; b=kiKspdy8kG2QuUv/B2GVzLIcN+h7rec5nf+oZd/MDfxJP6BPDYX7V6rkKFjAUJFqDz 4HmX7LffBSTl2juDhyTcYQxMW4KM7pjkoY/yDzlE9sQlwE9tIHl95eHTXLWGfqMxtXa0 ejr9Lh48UjM1sBHyFKH9lohsXaaxgHlBiVCXGNHbmM1gBQFAtxVUdB+u6OcOnDW9W64o OlZzFUnFWR+GXJMAly5+ZfLlaom1SeKTUBUyGTf4S5lUj33mnXjHWuCATAG/aI8sWmmr G7dZrbEb17Rp91xtDxtOqrssRuQ7grj2IlPMfF3EKA6YMmm0hFgZJz/yVVc0eMtSNRSx WAxQ== X-Gm-Message-State: AOJu0YxcsmcHyEKNmGFGOvlB6w7irjlihU/+uEAcLbQeHp5QH32N4a5a gXcF/bs+iHRVbbLD3bAANOV7h6uAwyiDOjh8LTzn7DzBV7WdvAStL9Km+NAkDK0BmqC4gWDplBC i0JiZ X-Gm-Gg: AR+sD13UfXz/Wklsjvw8kLTEblyKeYuHD/oS1D8qw/AiS4fQ8yuG+wnzJj2kzyFDPyd oPIBWDTfkiQVbz3ICbtv3D5fQC/6wIrGehiKheXzeAzIZlVirUFBSrzvPsItGFtuQcxQUdFQ8Sx h6ZVpQxlSa2uSeFtPpSvvtqB/lO3sFoviMixR+40SbJLmO1Yv7E8vYRf/qQ3pA9dbYl12BtJb28 Qe2BCutIwOfVztgtjM3HI3TnGQ6tdNtXKLdjan46cZ6QVo3YKqopICKEq8MGi0XHUUhJbfF1yP8 Xxh23l7OA41hUru8WJ11Elp1zSw+o+NSWutyoejsefsac+ApFbBpOZI0QN5pWuHHeqczfGTVCU8 BzCfVRwvbf1VuGhX86iQPjGzR6PpZK085HGrff3ZCdnXij6cjCZhsV/lq/PollzaEATOhnPCO+8 wqCoss42qyCXbjKVsvapRhq9c5IzpI/SOuoA+s3FG/ltSU4EK2FTfLX5eJPOZ7DG/kLOell22Rm fEGxkCAJtVXkKTltoNChsbQoldBAJsZ7Zhku47GL6cYzQoK5szOaWf3ekdkbh+bjDN1J7LPEYfc X4Qu X-Received: by 2002:a05:6000:2c08:b0:47f:8d71:210a with SMTP id ffacd0b85a97d-47fd72b8dc4mr26597063f8f.15.1785762978543; Mon, 03 Aug 2026 06:16:18 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:4706:89ee:c47e:1087]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47fd45895b6sm33725944f8f.26.2026.08.03.06.16.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Aug 2026 06:16:18 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v3 1/1] gstreamer: Prefer non-raw (i.e. non-Bayer) formats in caps negotiation Date: Mon, 3 Aug 2026 14:07:47 +0100 Message-ID: <20260803131616.11292-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260803131616.11292-1-david.plowman@raspberrypi.com> References: <20260803131616.11292-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Existing code was preferring fixed size outputs over ones with ranges, resulting in raw output formats being preferred even when applications ("cheese" is one such example) aren't expecting them and subsequently fail. The revised version still prefers fixed size outputs over ones with ranges, but will prefer non-raw (meaning non-Bayer) formats where these are available. This still allows applications that explicitly request raw formats to get them, but other applications will normally get non-raw formats which they handle better. Fixes, for example, "cheese" on Raspberry Pi. Signed-off-by: David Plowman --- src/gstreamer/gstlibcamera-utils.cpp | 115 +++++++++++++++++++-------- src/gstreamer/gstlibcamera-utils.h | 2 +- src/gstreamer/gstlibcamerasrc.cpp | 3 +- 3 files changed, 87 insertions(+), 33 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 6541d478c..996467ef4 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -8,6 +8,7 @@ #include "gstlibcamera-utils.h" +#include #include #include @@ -435,61 +436,111 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg return caps; } -void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, +/* + * We will want to distinguish between caps structures corresponding to + * raw and non-raw (processed) output images, as this will help inform + * our choice of preferred format. + * + * We identify Bayer as being the principle "raw" format here, but note + * that we are considering greyscale (images with one component) to be + * raw too (e.g. a raw monochrome sensor). + */ +static bool +gst_libcamera_structure_is_raw_capture(const GstStructure *s) +{ + if (gst_structure_has_name(s, "video/x-bayer")) + return true; + + if (gst_structure_has_name(s, "video/x-raw")) { + const gchar *format = gst_structure_get_string(s, "format"); + if (!format) + return false; + + const GstVideoFormatInfo *finfo = + gst_video_format_get_info(gst_video_format_from_string(format)); + + return finfo->n_components == 1; + } + + return false; +} + +/* + * Data recorded for each caps structure candidate, used to rank them + * against each other via operator<(). + */ +struct CapsCandidate { + guint index; + bool is_raw; + bool is_fixed; + guint delta; + + bool operator<(const CapsCandidate &other) const + { + /* + * Raw formats are likely to be useful only to applications that + * specifically ask for them. Other applications will typically be + * unable to handle them and fail, so prefer non-raw candidates. + */ + if (is_raw != other.is_raw) + return !is_raw; + + /* Prefer a reliable fixed value over a range. */ + if (is_fixed != other.is_fixed) + return is_fixed; + + /* Otherwise the closest size match wins. */ + return delta < other.delta; + } +}; + +bool gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, GstCaps *caps, GstVideoTransferFunction *transfer) { GstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat); guint i; - gint best_fixed = -1, best_in_range = -1; GstStructure *s; - /* - * These are delta weight computed from: - * ABS(width - stream_cfg.size.width) * ABS(height - stream_cfg.size.height) - */ - guint best_fixed_delta = G_MAXUINT; - guint best_in_range_delta = G_MAXUINT; - /* First fixate the caps using default configuration value. */ g_assert(gst_caps_is_writable(caps)); - /* Lookup the structure for a close match to the stream_cfg.size */ + /* + * Build a candidate for every available caps structure, and keep + * track of the best one seen so far, as ranked by + * CapsCandidate::operator<(). + */ + std::optional best; + for (i = 0; i < gst_caps_get_size(caps); i++) { s = gst_caps_get_structure(caps, i); gint width, height; - guint delta; + bool is_fixed = gst_structure_has_field_typed(s, "width", G_TYPE_INT) && + gst_structure_has_field_typed(s, "height", G_TYPE_INT); - if (gst_structure_has_field_typed(s, "width", G_TYPE_INT) && - gst_structure_has_field_typed(s, "height", G_TYPE_INT)) { + if (is_fixed) { gst_structure_get_int(s, "width", &width); gst_structure_get_int(s, "height", &height); - - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); - - if (delta < best_fixed_delta) { - best_fixed_delta = delta; - best_fixed = i; - } } else { gst_structure_fixate_field_nearest_int(s, "width", stream_cfg.size.width); gst_structure_fixate_field_nearest_int(s, "height", stream_cfg.size.height); gst_structure_get_int(s, "width", &width); gst_structure_get_int(s, "height", &height); + } - delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); + guint delta = ABS(width - (gint)stream_cfg.size.width) * ABS(height - (gint)stream_cfg.size.height); - if (delta < best_in_range_delta) { - best_in_range_delta = delta; - best_in_range = i; - } - } + CapsCandidate candidate{ i, gst_libcamera_structure_is_raw_capture(s), is_fixed, delta }; + + if (!best || candidate < *best) + best = candidate; + } + + if (!best) { + GST_WARNING("Failed to find a suitable caps structure to configure the stream"); + return false; } - /* Prefer reliable fixed value over ranges */ - if (best_fixed >= 0) - s = gst_caps_get_structure(caps, best_fixed); - else - s = gst_caps_get_structure(caps, best_in_range); + s = gst_caps_get_structure(caps, best->index); if (gst_structure_has_name(s, "video/x-raw")) { const gchar *format = gst_video_format_to_string(gst_format); @@ -529,6 +580,8 @@ void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry, transfer); } + + return true; } void gst_libcamera_get_framerate_from_caps(GstCaps *caps, diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 35df56fb9..5ce0e839c 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -18,7 +18,7 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg, GstVideoTransferFunction transfer); -void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, +bool gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, GstCaps *caps, GstVideoTransferFunction *transfer); void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *element_caps); void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls, diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 9061f9163..66b31ce82 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -605,7 +605,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) /* Fixate caps and configure the stream. */ caps = gst_caps_make_writable(caps); - gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]); + if (!gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i])) + return false; gst_libcamera_get_framerate_from_caps(caps, element_caps); }