{"id":21863,"url":"https://patchwork.libcamera.org/api/patches/21863/?format=json","web_url":"https://patchwork.libcamera.org/patch/21863/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241108092113.3769844-1-qi.hou@nxp.com>","date":"2024-11-08T09:21:13","name":"[v2] gstreamer: Add GstVideoMeta support","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"3f0f7beedf4c29dbbc70d616c83ee3a047ac2056","submitter":{"id":195,"url":"https://patchwork.libcamera.org/api/people/195/?format=json","name":"Qi Hou","email":"qi.hou@nxp.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21863/mbox/","series":[{"id":4781,"url":"https://patchwork.libcamera.org/api/series/4781/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4781","date":"2024-11-08T09:21:13","name":"[v2] gstreamer: Add GstVideoMeta support","version":2,"mbox":"https://patchwork.libcamera.org/series/4781/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21863/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21863/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 084B3BE173\n\tfor ;\n\tFri, 8 Nov 2024 09:21:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1124965495;\n\tFri, 8 Nov 2024 10:21:33 +0100 (CET)","from EUR05-AM6-obe.outbound.protection.outlook.com\n\t(mail-am6eur05on20629.outbound.protection.outlook.com\n\t[IPv6:2a01:111:f403:2612::629])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 25B1D60589\n\tfor ;\n\tFri, 8 Nov 2024 10:21:31 +0100 (CET)","from PAXPR04MB8285.eurprd04.prod.outlook.com\n\t(2603:10a6:102:1ca::15)\n\tby GVXPR04MB9851.eurprd04.prod.outlook.com (2603:10a6:150:118::17)\n\twith Microsoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8137.22;\n\tFri, 8 Nov 2024 09:21:29 +0000","from PAXPR04MB8285.eurprd04.prod.outlook.com\n\t([fe80::e003:8fb:64ea:acfd]) by\n\tPAXPR04MB8285.eurprd04.prod.outlook.com\n\t([fe80::e003:8fb:64ea:acfd%3]) with mapi id 15.20.8137.018;\n\tFri, 8 Nov 2024 09:21:29 +0000"],"Authentication-Results":["lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=nxp.com header.i=@nxp.com header.b=\"DqiTFb3X\";\n\tdkim-atps=neutral","dkim=none (message not signed)\n\theader.d=none;dmarc=none action=none header.from=nxp.com;"],"ARC-Seal":"i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n\tb=gA9Vu4VAK1LJ+SbIfRwgZ9EM+33PBFHrEk0yz3Mq4ha+MKM4y9LlKI8mPbo7+7JcIydHEL/gJPV5G3gmODKvDY/C0XXUvcyvEFYTTQwgxmdeSePpPqs6FfGIOrJogd7Dj2Upp7/+4F2mL+JA96EnzoDb30hwWmaMcgWN3OUp2TYVEw0Oq9FfjsrHvTUnBqI8zhvtcZ171o0W6voP7KL0T8vgtdoDFjCTQeO470XrBnbJbj46IFCLFAZq4ooTOgbMlP9IqXN5j9YxDQmAR//zGzg8FewUSkrkN3skMsm4cQgQGJ0eFtY1OzwB4KvDQnvhmPPHG40duT3NEpS/JzMKqQ==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n\ts=arcselector10001;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n\tbh=kaIU6ts2O45I8fJK8umwmc6jwO1wDNvY0khcSPjfjlI=;\n\tb=r/LxjqHFJny8TYQwC8q73mX4pLsfu+jlxZAc34MLZmrl+LgacvTMUpWoEjr7cmgnN/UK0Jbmaab7j2HU047pvOw4A1jbDdOUV+zD1uzomyQOZPEcQvtgx4RSxTIBRfrgWI4f7PqGBcRGIka4roRjHLSnS5wR8dm1Q6iTOApNJ3pd0h/XhXEN3XDIO06LyaFUxWqTpFLhlQU2gac2eWUA9rDvCpKJx+1FAzC0DjpAnQDnfPBbPvSRE1pOYSqeowx1ahFIGdZrSN/0tXoTCbWlnXAjoOvKhoHiFs865d4Mtm1CuSmeOPDJFNVOXjFedWlw95tHTWItH1sybO1C/299Ag==","ARC-Authentication-Results":"i=1; mx.microsoft.com 1; spf=pass\n\tsmtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com;\n\tdkim=pass header.d=nxp.com; arc=none","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n\tbh=kaIU6ts2O45I8fJK8umwmc6jwO1wDNvY0khcSPjfjlI=;\n\tb=DqiTFb3XQq7dK0MYbFIGh6wOj6M0fdAOMFDTMXSGehm12MiRTayEW+/MplOmnaWSwKc3ovHLlV4d8nF2v2saeXwfhqwQh1d9c5lqMbn0/uGAnPsCRIA2/M9/7F7I+GpYDORWBQfee8FMh7GCDg7ojeTVbX1XmkmuFWFKdHrZO9V8XAiVRukpmjSEANMc93AILLKDLlLjnpz1M9hjyjEIab1jc4qhZx7umEh9OGrn3o7LpjjxqB/zawm4wUgKiU/JSvNvMNoy14tIPYAds8f9hGoZUd7YLWyI2HMOKnZPO/Q5qoesD47CHwlUGy/0JjkgRnoJcwm3vOVsLdnk0Z9q5g==","From":"Hou Qi ","To":"libcamera-devel@lists.libcamera.org","Cc":"jared.hu@nxp.com,\n\tqi.hou@nxp.com,\n\tjulien.vuillaumier@nxp.com","Subject":"[PATCH v2] gstreamer: Add GstVideoMeta support","Date":"Fri, 8 Nov 2024 18:21:13 +0900","Message-Id":"<20241108092113.3769844-1-qi.hou@nxp.com>","X-Mailer":"git-send-email 2.34.1","Content-Transfer-Encoding":"8bit","Content-Type":"text/plain","X-ClientProxiedBy":"SI1PR02CA0002.apcprd02.prod.outlook.com\n\t(2603:1096:4:1f7::16) To PAXPR04MB8285.eurprd04.prod.outlook.com\n\t(2603:10a6:102:1ca::15)","MIME-Version":"1.0","X-MS-PublicTrafficType":"Email","X-MS-TrafficTypeDiagnostic":"PAXPR04MB8285:EE_|GVXPR04MB9851:EE_","X-MS-Office365-Filtering-Correlation-Id":"2e618583-279c-46a4-02e9-08dcffd6b913","X-MS-Exchange-SenderADCheck":"1","X-MS-Exchange-AntiSpam-Relay":"0","X-Microsoft-Antispam":"BCL:0;\n\tARA:13230040|1800799024|366016|376014|52116014|38350700014; ","X-Microsoft-Antispam-Message-Info":"LonvvLH2qpGizXtYEjCHd9VrIwWq3GpGkjrnm7maxr0pM6zWntZJPzacf781cnx70MTzw6Qp1pCZZvG4dQhoxl58LpSYTgiyzMk5dtSfO4Vhfki8gKxs/rFH/Wyyrg/FpOI3bZBZT2yYbwW/VCJPKWmYXDYqQ6mOdggMPxY+2tCcy5NFCFEnymaWuy476pmMaHia+kGD9XZ2e13n//QiQTUP2Zfo4Hq4CakLqIDXl8cQtDmMN3w9xmaAG5OcO27FyAYd9mbzIF3Zv0/OgKsQ5UTgLZG3Dwq45FSa0AboJr/PpJPlbdkRbf27dOJlTxtQwGIurZP/N0AE0foESdhDPb9hOw0ldyj14WplM2UENdwhW9lk0bfNDLRQ7rog+y8bB6rN8Qzko9fIdUy8Y/iuTQMu3LMmkbFsXuFL633bJt1zWM5yReViecAXYaXX9J3yfulC09M5x3HqRlvOkgElA9JZ9+61Sh/mwJ6bP/rTbstOGXnklZXssk2Y0lNv3Dt7T/esVLEy1jboXz/0erdUYNZoyNRrkaqEOyi/umRBjG0m0ddNqmfMMtNemWjN+nApbFd1AVCin9H/3sGgqYeHll0YrpnpYCXQGD7IuOIveFPudNAEJCsyrJlSi+aXvQK0cg29AbQD+4ZRU3xY+5H38WYmuS8SlEB2fjKAt2iEt1RMnskRIlIQDFuxZrGMnyh5Dq+x5PjBj1geyE3MhJnePGPbCO6smecOlZVfpomMX4GU7tfcDG0C7Ea1UKyXoeG2Pqy84xvVgCElzWBccne6pFZCSeCF+7cVr9WK4xRG1FqV7yaaQbqBmyJSByA3NWyjK11qyQU8krrJ2CHER+cWd1/YYs7vZoC1WeJeYdKU6yz08uVUQpx49B2U0/pDnLf8au1i29/sYNLc5YRtCBU/VcIdFXUyIHmkTXf9ROvoBZoUN9U/ha8E1LazAd168LLc1SmQCZyZueCom3FwCOdchYCU0bGg5icOzmpXobJeDKK+kwnCJikzmtY7HTzfhIyRS+xqaqwb4xOIbOcK10O6BUvfaAdii+DKeCon13je7sjxwjvFsxyGicX+cM5PMwlh6h7bu1+ACtqpZirHPJTNI8Z/zEJfx8tO+Vumq+0D2yv3VEKh43Ogavv1w5J/ZVbcrbXOe6BuUjyHRkCuYY8alRsRV/5baLJwpyMza23Xrvwj9QB6kthYLH55qoQyjN1penK3jQ38gEFtOU70+bT0zcisebY1fkU7fvBWw/csKRygc3Jt+OD717i2wZ7c8KH13fnqbbZk9RHG5ZHMbGB0pXxTHFIBA5ebfVXsYkSW7fhMD1Sny6ybiEORrs/8zK51rv8a6toBkaS/KORItJR8cCjlEx7Le7aqKR7pTyVDyuQ=","X-Forefront-Antispam-Report":"CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n\tIPV:NLI; SFV:NSPM; H:PAXPR04MB8285.eurprd04.prod.outlook.com; PTR:;\n\tCAT:NONE; \n\tSFS:(13230040)(1800799024)(366016)(376014)(52116014)(38350700014);\n\tDIR:OUT; SFP:1101; ","X-MS-Exchange-AntiSpam-MessageData-ChunkCount":"1","X-MS-Exchange-AntiSpam-MessageData-0":"2qX5IAkV4d8YoW5aiFlRGCBGhtTa1eYJ9PHEh0amqbbZNPqu46y+37DXtsai9w6gOSRWy7bf5a3LlgYp+0DTon03MjCy+QxyFhdL53TlfSU49vb3iJTeGesVKwQC87I0Y1vxLZF6UQmUd3Len98r0n7zCEl5F+dvnpGKWf61i5OBw03HJEpfGGICijEMfpcDXqUlIkzV8327fms3lNmVN3NhV6krxww67PH//EOI1ox1Y/zE8Q6O0zYHFd6i6636Os4atjBo3yYgzk6Sva60h4MNrpkv6Hh4VmUZg9yvFHUzQrIbcX3z5OqEHzM8aMkKC5z51Wfdq9hsEUMSkJArewmlVUR4/tcA9C4JKjNaZtkXzwxZOHiPZC/xO3yzNBTd4WODWO44x0OmNtr4ZPzsOI623vDOacDxELEkeej1N4O/uPMSzFNebAASeRSsDf47iwEHO4Vn9LNGYKR3rOkzE5Xpk+vQk87mDICAzeUSXbY+99c3h8w8b78Driv5ieK0DIyYSWTPhmuHlBlUcC+Q4tMKwX0a0/TlX9N0CDR57gu+MnD8RHueOGnEJP2H1JSSmwlq1ULh+mw6XR+K0WZBMQHl4qTzqAHVqz2+DJ/YRCoSx387tShkcpH8K2OXbU/NDNEPK/0K/NdRP9/VL6FZC3Jdv9dR3Dmw6BuIOVhJ2W6Rj6QUxtXHOUbhk3DSkWgIFVVTw+uMSTYBtnQ3A77mEUpwbNXU7sphmnJNawjTteEvboXnuBSDMcnFiWWWRf6d8+138aqVAaBCqKEgY8zphNWWD1s9MMyyCyiK7gDw9i+SXUiQdVEcIbQbJyGct7PNoJjM/S0B9EAfnsuQQbo/O7F+f6frB+6Ll6m5tmAeUtBQcsKFijSdfMyg2qZgBm7BYeMg/wTJIfjQf/mNq1GE8zRXhcu6ylHRFSgErVwFiP/rsNml1Po1ZBBt3RG3GMU8xbcpBNiYGpsB3YSgyadJDHCEA8pmu2ZhMvg4yPeQXSL4Nbmt9MmImnlUZqV9ERu6wcYgvkJd553YtWYQauaNbaz8/O3Ojk56edZzshrR3SgJ0teXkQbC+YXd5ewzIUecO0htuyUHchwxD5FbpVts74Ipu4/mmFDRv3QfOixhK+/G0klUJ4VF1Fy3/A698Vaf0rBwNkkqjn7rMjv3gNJSx3B+Z7OvlDVbx57V+/7/BvShlAmueBIm5Q9bpIxPbUssE/Tyezpesl2xCW52lPU4iSKY+yWsvy9RoGWDaKsasqzXQHLKMQXs1mtqga+hLBOGVJHkVF2yJumAtdAv4MVFBlrhATDJteG1tS5tLO9KBJ9IexKk+QuhqEkzPyWX/zTnbyAiDZLzJ1TB2Bmn94gY0VICOeS60fTBwUgCStgAS07P6XHU2eAj34d64M3A2tBbbllJUo4c0YdLDtn3FyhCg04K+/G51dOF83BADYuIR2m6yiUbMmUSBb2cegByYMxwZ6LRYCps4HBkO70FqurIw10xhbBr7q2V7BMnCgdI+bGf5/enQgKT2nj1dly2jdu5AKllWR/7kF/t6DyMIcpAPcE+ate35NAX693rz8qOYLneRfscz+6cmWXUUlelxtUP","X-OriginatorOrg":"nxp.com","X-MS-Exchange-CrossTenant-Network-Message-Id":"2e618583-279c-46a4-02e9-08dcffd6b913","X-MS-Exchange-CrossTenant-AuthSource":"PAXPR04MB8285.eurprd04.prod.outlook.com","X-MS-Exchange-CrossTenant-AuthAs":"Internal","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"08 Nov 2024 09:21:29.0233\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"686ea1d3-bc2b-4c6f-a92c-d99c5c301635","X-MS-Exchange-CrossTenant-MailboxType":"HOSTED","X-MS-Exchange-CrossTenant-UserPrincipalName":"a1G87TjsGPzkhOFr0mB00z/TWXnSpecDXgP8z+TD8TvYR1RNc/+P1HGE5fkBNm9d","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"GVXPR04MB9851","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"GStreamer video-info calculated stride and offset may differ from\nthose used by the camera.\n\nThis patch enhances downstream plugin's support for videometa by\nadding videometa support for libcamerasrc. It ensures that when\ndownstream plugin supports videometa by allocation query, libcamerasrc\nalso attaches videometa to buffer, preventing discrepancies\nbetween the stride and offset calculated by video-info and camera.\n\nSigned-off-by: Hou Qi \n---\n src/gstreamer/gstlibcamerapool.cpp | 28 ++++++++++++++++++++++++----\n src/gstreamer/gstlibcamerapool.h | 2 +-\n src/gstreamer/gstlibcamerasrc.cpp | 14 +++++++++++++-\n 3 files changed, 38 insertions(+), 6 deletions(-)","diff":"diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp\nindex 9cd7eccb..6593b3ca 100644\n--- a/src/gstreamer/gstlibcamerapool.cpp\n+++ b/src/gstreamer/gstlibcamerapool.cpp\n@@ -135,16 +135,36 @@ gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass)\n }\n \n GstLibcameraPool *\n-gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream)\n+gst_libcamera_pool_new(GstLibcameraAllocator *allocator, const StreamConfiguration &stream_cfg,\n+\t\t GstCaps *caps, gboolean add_video_meta)\n {\n \tauto *pool = GST_LIBCAMERA_POOL(g_object_new(GST_TYPE_LIBCAMERA_POOL, nullptr));\n+\tGstVideoInfo info;\n \n \tpool->allocator = GST_LIBCAMERA_ALLOCATOR(g_object_ref(allocator));\n-\tpool->stream = stream;\n-\n-\tgsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream);\n+\tpool->stream = stream_cfg.stream();\n+\n+\tif (caps && gst_video_info_from_caps(&info, caps)) {\n+\t\tguint k, stride;\n+\t\tgsize offset = 0;\n+\t\tfor (k = 0; k < GST_VIDEO_INFO_N_PLANES(&info); k++) {\n+\t\t\tstride = gst_video_format_info_extrapolate_stride(info.finfo, k, stream_cfg.stride);\n+\t\t\tinfo.stride[k] = stride;\n+\t\t\tinfo.offset[k] = offset;\n+\t\t\toffset += stride * GST_VIDEO_FORMAT_INFO_SCALE_HEIGHT(info.finfo, k, GST_VIDEO_INFO_HEIGHT(&info));\n+\t\t}\n+\t} else\n+\t\tadd_video_meta = false;\n+\n+\tgsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream_cfg.stream());\n \tfor (gsize i = 0; i < pool_size; i++) {\n \t\tGstBuffer *buffer = gst_buffer_new();\n+\t\tif (add_video_meta) {\n+\t\t\tgst_buffer_add_video_meta_full(buffer, GST_VIDEO_FRAME_FLAG_NONE,\n+\t\t\t\t\t\t GST_VIDEO_INFO_FORMAT(&info), GST_VIDEO_INFO_WIDTH(&info),\n+\t\t\t\t\t\t GST_VIDEO_INFO_HEIGHT(&info), GST_VIDEO_INFO_N_PLANES(&info),\n+\t\t\t\t\t\t info.offset, info.stride);\n+\t\t}\n \t\tpool->queue->push_back(buffer);\n \t}\n \ndiff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h\nindex 2a7a9c77..8ad87cab 100644\n--- a/src/gstreamer/gstlibcamerapool.h\n+++ b/src/gstreamer/gstlibcamerapool.h\n@@ -21,7 +21,7 @@\n G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstBufferPool)\n \n GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator,\n-\t\t\t\t\t libcamera::Stream *stream);\n+\t\t\t\t\t const libcamera::StreamConfiguration &stream_cfg, GstCaps *caps, gboolean add_video_meta);\n \n libcamera::Stream *gst_libcamera_pool_get_stream(GstLibcameraPool *self);\n \ndiff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp\nindex 8efa25f4..c05a31e7 100644\n--- a/src/gstreamer/gstlibcamerasrc.cpp\n+++ b/src/gstreamer/gstlibcamerasrc.cpp\n@@ -497,9 +497,21 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)\n \tfor (gsize i = 0; i < state->srcpads_.size(); i++) {\n \t\tGstPad *srcpad = state->srcpads_[i];\n \t\tconst StreamConfiguration &stream_cfg = state->config_->at(i);\n+\t\tGstQuery *query = NULL;\n+\t\tgboolean add_video_meta = false;\n+\n+\t\tg_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg);\n+\t\tgst_libcamera_framerate_to_caps(caps, element_caps);\n+\n+\t\tquery = gst_query_new_allocation(caps, false);\n+\t\tif (!gst_pad_peer_query(srcpad, query))\n+\t\t\tGST_DEBUG_OBJECT(self, \"didn't get downstream ALLOCATION hints\");\n+\t\telse\n+\t\t\tadd_video_meta = gst_query_find_allocation_meta(query, GST_VIDEO_META_API_TYPE, NULL);\n+\t\tgst_query_unref(query);\n \n \t\tGstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator,\n-\t\t\t\t\t\t\t\tstream_cfg.stream());\n+\t\t\t\t\t\t\t\tstream_cfg, caps, add_video_meta);\n \t\tg_signal_connect_swapped(pool, \"buffer-notify\",\n \t\t\t\t\t G_CALLBACK(gst_task_resume), self->task);\n \n","prefixes":["v2"]}