From patchwork Mon Dec 16 03:13:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hou Qi X-Patchwork-Id: 22329 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECBE0C32F9 for ; Mon, 16 Dec 2024 03:14:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2693F67F2F; Mon, 16 Dec 2024 04:14:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=nxp.com header.i=@nxp.com header.b="L4Qg8g/m"; dkim-atps=neutral Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazlp170130007.outbound.protection.outlook.com [IPv6:2a01:111:f403:c20a::7]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D87AF62C8A for ; Mon, 16 Dec 2024 04:14:13 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Hp0y0VJchAJaZBkYHh2NoyYZVxO93wexv0VgdySqUSHFGGQObk3uCP0i7p07xSWCn6tBXId7wmJ9CbxLEf88v2T8ulaTkRNpQV7DTHDAOmOEQF+bspj8+aatJzpjjqTgSn8gdTFKoISSqXvpW9xGTkY0v0tioPPK8CxTHJ6gb6UAoI1IaRwtihSKi26Dze3H660SrmuB0NdrcAxflo+m2MLG9953Ya1WeJh2QNsorNdT5vVrddKOOoaauLfgmiWQ8ddVdGEwxrWrgTbRljIyP257UnzhirV/RYLLviBqB9SsAxlvkT8zxkGMb2Ea80eT1gVc496RS4s5Wk3XBREWIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z3eFlYQ7iqgHYa4/A9lVD/hoVokUIAqQtb9OTvYUl0I=; b=y5b2klOfcRahKX38+y3HQrUbf5a6OmeKaFboQg5kbpfUhQs98n9aqNgIm0g2TH6zxQsT8fUXnotZIVQPoNPnjtH+c3VeR0Slch8LWAsU8M5yPNxOJFfuexEOk5htTLOv+KG2cSbw9RCT4Zfz79UXtT3t+GZQY1xQgGyXemSvbZpfW7faPaB+c2I7zjFlFl26VrKxRM2C7Yadttye5JFCi7/aOp6o/E0vPZ55/emEraSFh3RD0E+U3urYsgZDjFP38KmjBWotVIsL1eMWFWrAy5XNCNKWmMwVdzFgqvt1ENXueYCOiDmSEg6l4uEO0xFRZOrpFyeUcwueSqZJU2hqkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z3eFlYQ7iqgHYa4/A9lVD/hoVokUIAqQtb9OTvYUl0I=; b=L4Qg8g/m2LIn/4jsUHoCYp9z2ZPlC3fS4D0isZcSqFqJPkURA6lUTyFbYWqhTIgUfEoe3TEjVUKeTScL00b9DmxsXUMtl6nVSU5gY1CC46j/dMEd/rmPZe9DKtg5N88MonRJSu302Vj/bk0TufkR4I1I6FTYG0Qq+4bxObgPTcIg9F3k0LGqkPkdza0P8+llE/uLDGX5wCIEtIlX6sDDzmThvXMglqcmpKh/E2Ly4SgGTHrgJa0dDn1wUGX4+PMHWtTDgA0keUjhlcUwNKsO8f7yMP+r4/waFPVYtrUd5c57TQ6ySxFLJAanTC9Elji8Efw8RJdAUCzrf9607sGtCg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from PAXPR04MB8285.eurprd04.prod.outlook.com (2603:10a6:102:1ca::15) by DU0PR04MB9493.eurprd04.prod.outlook.com (2603:10a6:10:350::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.20; Mon, 16 Dec 2024 03:14:12 +0000 Received: from PAXPR04MB8285.eurprd04.prod.outlook.com ([fe80::e003:8fb:64ea:acfd]) by PAXPR04MB8285.eurprd04.prod.outlook.com ([fe80::e003:8fb:64ea:acfd%6]) with mapi id 15.20.8251.015; Mon, 16 Dec 2024 03:14:12 +0000 From: Hou Qi To: libcamera-devel@lists.libcamera.org Cc: jared.hu@nxp.com, qi.hou@nxp.com, julien.vuillaumier@nxp.com Subject: [PATCH v3] gstreamer: keep same transfer with that in negotiated caps Date: Mon, 16 Dec 2024 12:13:53 +0900 Message-Id: <20241216031353.1622924-1-qi.hou@nxp.com> X-Mailer: git-send-email 2.34.1 X-ClientProxiedBy: SI1PR02CA0060.apcprd02.prod.outlook.com (2603:1096:4:1f5::17) To PAXPR04MB8285.eurprd04.prod.outlook.com (2603:10a6:102:1ca::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8285:EE_|DU0PR04MB9493:EE_ X-MS-Office365-Filtering-Correlation-Id: 81300e2c-c8ba-4952-a5ff-08dd1d7fb690 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|1800799024|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: G1FvDzSYUTKLXfK8e3fgu/XDUpx0fkqvpWS5PGX5XsfMnnAXzww1qjYh9RfdIHxiZ9xkSryxYgEtDxOPdS5zQp+tQemNz87zjHWe0BNBhA0ttytUn8SFlywkp+9UaBxEPW/5/+nGWpcb8588waEURpYeaGJ7Bb+/abPMcA0i3bFRJcDXRV6zMuNyMo3va6FaFrsWtxRRQw9QsiXgYpizQJd3wWan4UeQkr10bZLu/ibA53/ub4xbxA0IDC20tbPdThH2Nf/u2HtILRRMiiZ9T1ccvMncLGjMUoeidvECVsI/MbT3kzwkDWamLAlOrbtJUlPPP9G0BiWzRaj0ICh0QiPf4qrGXceVRvyErur40jQ1XetXZSLqTxuEtPp0KT4DCaBeVyMKhUF/+tus0Amu/49hi2zNkDtvlZjQEyJni5e+UFDQF1rIaGWzVWIT5GRUhJVv/fCpeEaHPxrzl83j7aZTfPAu26fQDzac3i4xmiPwa32gtW6HctgSBCFuwH2QIZzadCxte8vhB2vwPL/uuJCHqNy31+rJygubRHfEzQkkVItlLlwUOBU562NFDrOYc8ftouUfMRGV31nQyKjgbgWgaOSimLYJo3tlH8izMpAQwfH+3Ep9bm7DqRsYJ7yQ5PY7csY5V+IcUcYeFRmLhaJBGrdcJgsOxXPzYSL4DEdUO3+kQ1Zfp3ZcYJIrQFwm+2UX6ntshLZGw6MhObUnCSaWN8i4KIg6m6YFIT8EnMhD1Z9R60WLg/e1JnUW0cAYl5BM86OUdT+ZxwOPVJZWIKkeUbmk7Co5LNDy8kHos0SPIjsYsvEmMGNU+ciID0z76ntnIxFlzJdu+HHax9mNF1iaeJXczp0PFLORqwclsCJbUdaijlduFIMapmuveIqBF3JbWb6cmjN3BTIqUsOdnGn2G+RphX2Sre/jrikY5C4fOt7qpD4v0yVW/JRP7GeJ2bFeXp9L2hT5AizJB1fbdmQ9gmnClAOWlRRWnte9Btvu6+SydUn6L2RneKJnCWDHTiWBf32DnSsh1DqabhRZHXZiKJPhkJc5nDfi4L6PraL0XEvOzSPmfYd4QAuxD0iVQK3zx0jzCGpEbf8A7wL3k9vTwmM5LR+xwv2AX14wfdarJtNCIobXfhKX1X+NkYFPt6zg5gBrhCj8PLXNz9suBh4Kdv+nkhA2FDu+WX/qIT5Rh1pT5HtdAwrED68EAx8IdYH1d1gagQhp37HwqxjAVWugd3TP6sXvhlXeKkjr6YqnVKY9XcZoWHjIECInzFsKYlDngzQwokB8/obCTqDNHA7gJSHRZfF5ZWihzu7gB/vJMgEi4go1oqpX7xAsr9kmuvDLXnsvkBFqZZBzvyPnu2N1uFpHhjGn9ngeDzKmh8V9qpJtRJTlTCoCdK4je43p X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR04MB8285.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(52116014)(366016)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: GYHevX0f1lxkRpHYUwDtGOIwFmfORmYiGbd+wgFYmV51OhPw19ZL8e3HK6lA6A4sheLKFmYxirzpJmcx7S67vQQcFSyHOalHbpPWTvCwnsxqY+x05obRx3PMyxtOswV82ObkwSLycs89nt8u10bALpqUsQ+7Z5kRhYBx+HyAdhbYEnt2POrbw85l2BRGYztcmNYY1dxtaw2arGvM67O/Egy4B8WBgK+nRPaM6AC5kS5QDF+DqPIPoSam/hyBPJt8Onin4sFeHIUgA1qZFug31T0uzuG5SjugWCNxh3jlb016EflWJDClFVOZIlPkardxeVOy1LV2K935Z4IKm1/J0FTe7vjwgs7z6xBXysjyPomC6vzQtrpWQwYGEwWH/s63bMk7BvirMNJ3vmb3bwJDO92omKgjjE/TW5kRyd5AGXiZ5FJaHBsZopvL69EeKji1WrJE2+T3pVmlLtGciY9IMUVRCEB0uEKkV0z1VvPA67E2F9AHcEH4caePF87Wxbl5QT9u102gN3Rte5W7AxUikAddPS55hVxi0zI5M8hXvjq4RuHVeuh+2PMm+a9usgQEyXhkMJ4cvXWB2ACbiEnyAMmXgWh7tBGRlBMW870bm0u5Jbv2rTueGDy5xlvlMQ1M1SjOYQALe6EweIcNj3NndihqvEaQqhnj/FzxjMGbyl4GWOHEzzQIKQJ1wBOBa9Qv9OCtBnp2jIdtjcpT6wT6+kAKse49B/jdZx1PCPXNjnLI/ybgg/FSQ28Nnhner/6hpitZZwO/7l6alx0TCh//FE6WkhZaOFVypERprpoL5/fO5XSqWy4TKD49yyJNdQrJQYVeAl6iff9Woc52C43TMB4gdpIJjnjZnn5Q6eQmAUm+6a4OPNQl2FsyjBEM5JpAJfHXYAlG+d+n4uVfXxzIqLTjMz9Divx7RiF9YBLEgdHe1llezSNsXCgWz3laQXsetyBLy5FMO+YQw1ib2EuTvD1w+JxQ/FlGY4XGF9gJWBhHT+NVW2N8T1L7U8YIzE4gTX9w2L7BeEMHMFn7m7I2KdnHxz3xq8B2Eq98sbtWiTKiWC/BXDJee++AtMWPu6HukcTZAmxbBUt8fFBjDnAe0F5cUtZWAVCEaHTUdzBJHUX8UU8sDXpdENxook+k+lAm742aYYzZ9lD3clJXIf//NnVC1hGqs+14KDtsV+mMO4yeWPJKkoMmIQfauBJew+eBkHCLbNfsLG3Vt3+NQNHJ4PWsolA+owgx/LD8b9OGaynIJZS80ua0Ug+dI77EaH4z8boUuN70XBr2L9Vpbg6RWnQuqQre107yN+h46wS8HYpmiJRRE5g5hjMRbdHwvQ0TGIPXpK4+DOWvB8fkYgZFmaTtDONO3R0qIEcvE+KdF3WyhuDPqdBfUpl9x457qeID4enTyBJkNMArH54q0adoZ7FfUzyXbcHsPoyrC6oFwfiTPFl3qPgBxYXjnFnQOaUO5sQv66xpnQ7SYT4OBSQUPMpHtKGT0sB8CvTQv4t3f6mChyh36qfvYZ8n07YXv2Z7Pw439PHHTPN/w4PgVioRAULzczwXEAju+b2BCMqWKS8= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81300e2c-c8ba-4952-a5ff-08dd1d7fb690 X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8285.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Dec 2024 03:14:11.9515 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: I0Xd4M46npUpFmmYrAaXeMsobkoDiu4/BL7HjxTJFtVg5cPByMUq5G9vNhTQWAWB X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR04MB9493 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The conversions back and forth between GStreamer colorimetry and libcamera color space are not invariant for the bt601 colorimetry. The reason is that Rec709 transfer function defined in GStreamer as GST_VIDEO_TRANSFER_BT709 (5), is to be replaced by its alias GST_VIDEO_TRANSFER_BT601 (16) only for the case of bt601 (aka 2:4:16:4) colorimetry - see [1]. Currently the composition of the GStreamer/libcamera conversions: colorimetry_from_colorspace (colorspace_from_colorimetry (bt601)) returns 2:4:5:4 instead of the expected 2:4:16:4 (bt601). This causes negotiation error when the downstream element explicitly expects bt601 colorimetry. Minimal example to reproduce the issue is with a pipeline handler that do not set the optional color space in the stream configuration, for instance vimc or imx8-isi: export LIBCAMERA_PIPELINES_MATCH_LIST="vimc,imx8-isi" gst-launch-1.0 -v libcamerasrc ! video/x-raw,colorimetry=bt601 ! fakesink Above pipeline fails to start. This change memorizes downstream required transfer function when mapped libcamera transfer is Rec709 in gst_libcamera_configure_stream_from_caps(), and restores the transfer function in gst_libcamera_stream_formats_to_caps(). [1] https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/724 Bug: https://bugs.libcamera.org/show_bug.cgi?id=150 Signed-off-by: Hou Qi --- src/gstreamer/gstlibcamera-utils.cpp | 20 ++++++++++++-------- src/gstreamer/gstlibcamera-utils.h | 5 +++-- src/gstreamer/gstlibcamerasrc.cpp | 7 +++++-- 3 files changed, 20 insertions(+), 12 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 732987ef..a466b305 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -85,7 +85,7 @@ static struct { }; static GstVideoColorimetry -colorimetry_from_colorspace(const ColorSpace &colorSpace) +colorimetry_from_colorspace(const ColorSpace &colorSpace, GstVideoTransferFunction transfer) { GstVideoColorimetry colorimetry; @@ -113,6 +113,8 @@ colorimetry_from_colorspace(const ColorSpace &colorSpace) break; case ColorSpace::TransferFunction::Rec709: colorimetry.transfer = GST_VIDEO_TRANSFER_BT709; + if (transfer != GST_VIDEO_TRANSFER_UNKNOWN) + colorimetry.transfer = transfer; break; } @@ -144,7 +146,8 @@ colorimetry_from_colorspace(const ColorSpace &colorSpace) } static std::optional -colorspace_from_colorimetry(const GstVideoColorimetry &colorimetry) +colorspace_from_colorimetry(const GstVideoColorimetry &colorimetry, + GstVideoTransferFunction *transfer) { std::optional colorspace = ColorSpace::Raw; @@ -188,6 +191,7 @@ colorspace_from_colorimetry(const GstVideoColorimetry &colorimetry) case GST_VIDEO_TRANSFER_BT2020_12: case GST_VIDEO_TRANSFER_BT709: colorspace->transferFunction = ColorSpace::TransferFunction::Rec709; + *transfer = colorimetry.transfer; break; default: GST_WARNING("Colorimetry transfer function %d not mapped in gstlibcamera", @@ -379,7 +383,8 @@ gst_libcamera_stream_formats_to_caps(const StreamFormats &formats) } GstCaps * -gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg) +gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg, + GstVideoTransferFunction transfer) { GstCaps *caps = gst_caps_new_empty(); GstStructure *s = bare_structure_from_format(stream_cfg.pixelFormat); @@ -390,7 +395,7 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg nullptr); if (stream_cfg.colorSpace) { - GstVideoColorimetry colorimetry = colorimetry_from_colorspace(stream_cfg.colorSpace.value()); + GstVideoColorimetry colorimetry = colorimetry_from_colorspace(stream_cfg.colorSpace.value(), transfer); g_autofree gchar *colorimetry_str = gst_video_colorimetry_to_string(&colorimetry); if (colorimetry_str) @@ -405,9 +410,8 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg return caps; } -void -gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, - GstCaps *caps) +void gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, + GstCaps *caps, GstVideoTransferFunction *transfer) { GstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat); guint i; @@ -495,7 +499,7 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, if (!gst_video_colorimetry_from_string(&colorimetry, colorimetry_str)) g_critical("Invalid colorimetry %s", colorimetry_str); - stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry); + stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry, transfer); } } diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index cab1c814..4978987c 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -16,9 +16,10 @@ #include GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); -GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); +GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg, + GstVideoTransferFunction transfer); void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, - GstCaps *caps); + GstCaps *caps, GstVideoTransferFunction *transfer); void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *element_caps); void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls, const libcamera::ControlInfoMap &camera_controls, diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 8efa25f4..c701b3cf 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -433,12 +433,15 @@ static bool gst_libcamera_src_negotiate(GstLibcameraSrc *self) { GstLibcameraSrcState *state = self->state; + gsize srcpads_num = state->srcpads_.size(); + GstVideoTransferFunction transfer[srcpads_num]; g_autoptr(GstStructure) element_caps = gst_structure_new_empty("caps"); for (gsize i = 0; i < state->srcpads_.size(); i++) { GstPad *srcpad = state->srcpads_[i]; StreamConfiguration &stream_cfg = state->config_->at(i); + transfer[i] = GST_VIDEO_TRANSFER_UNKNOWN; /* Retrieve the supported caps. */ g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); @@ -448,7 +451,7 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) /* Fixate caps and configure the stream. */ caps = gst_caps_make_writable(caps); - gst_libcamera_configure_stream_from_caps(stream_cfg, caps); + gst_libcamera_configure_stream_from_caps(stream_cfg, caps, &transfer[i]); gst_libcamera_get_framerate_from_caps(caps, element_caps); } @@ -476,7 +479,7 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) GstPad *srcpad = state->srcpads_[i]; const StreamConfiguration &stream_cfg = state->config_->at(i); - g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg); + g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg, transfer[i]); gst_libcamera_framerate_to_caps(caps, element_caps); if (!gst_pad_push_event(srcpad, gst_event_new_caps(caps)))