From patchwork Wed Nov 20 10:04:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hou Qi X-Patchwork-Id: 22030 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5CDA5C0F2A for ; Wed, 20 Nov 2024 10:04:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5340365F41; Wed, 20 Nov 2024 11:04:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=nxp.com header.i=@nxp.com header.b="Bb61wKc/"; dkim-atps=neutral Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on20608.outbound.protection.outlook.com [IPv6:2a01:111:f403:2613::608]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5B1D4658B6 for ; Wed, 20 Nov 2024 11:04:38 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OAbn+fP16UKnP2aycmPz6cuY9rcsAXRkWQ+QlmDK7sKpJP/nwMTBz6DC3NOEF3bU3c6UNRHkeU53X0+Pcu9VFRMW7hD4kcIdOrMtWtf8Rs9s3ns+2tlLWvFMuAOV3nuyk7TKL0wZP6WtzhnFnnExi5SI3rPpIm0OLk2mZdr7NMBKUcfaMUX112lDh8UPoqRHsTF0toctnd2LihC+dRrzTxoiymmBu2xCFFOyNDzYgg9Y5XAJNorO91dYAhbICVvlL7mDSUHtyzODbBPjd6hyK8aVVid5C0YXb4xcG5P7yoHC89H82AdR8iKnPmJpRf9xKsrF0dRKDtEMqW5tFJ2FKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Wvoes9BW8z58Gkri40IXGS+eVNDutUnx+pXzTbLBcAY=; b=SLkuEHMGQOJievZMk1ZEZ4W2iT2NEiSL24Z9QzUMnY2YTaxAboxp60SfqYkHv8FW1fdiP2l3Kb5saJYauWgSvx6mJ7r1f3uCUO9yGXCkmSQybobp7cS0IcM8qQ76O2snCjJAsT4zamAYViozV/xGMoHnJEMV8LU9MwPFAIPkQHS4VS/DfFSW4pUFp2wfrlF+iQclERh9bH6GwFaT3i1FaaxC1kgbOWNNRWj2n+HgtiHfnSQiFxzvFoC7gEAMIQzEuEXa6uXVXrGzkdaFTyM+Cj70ToH/NMD7z0JsRjZUXgY+Ws5cE9duFS2LbBlwAtkOV08c+WN5Fz52cBoVdwORnQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Wvoes9BW8z58Gkri40IXGS+eVNDutUnx+pXzTbLBcAY=; b=Bb61wKc/lK0GRtEO9vYaWcl8rA2/6o+9ytIPWAXQRl5F29qKdPafvawr6u22YLPeM/xcMefuXOm65DGQZfN3IO+e5Rbai0hy9ZlOSkNfVwze0J2RQvPT51h8t944TmzerX4zy3MlkYetj9rqMkRWH0iCNF1dj+ehxluDWUqllXjpePtMDeNJdI3pdMs3mmRVO0XXOShGpxD8NYO6wPDyPKR+oPqFRN1HmOIZJWxCuRvHkiVtdb4/Wtjwk47Uc1K2xSz0IyeBGdhfYn3eh5eNgX4FB4ZC4jW9H9NcY6fFDG8ZnQRLWnSm4qV0pXNNAuxHYGDBrg1IQPLPGaVCA0/r4Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from PAXPR04MB8285.eurprd04.prod.outlook.com (2603:10a6:102:1ca::15) by AM8PR04MB7779.eurprd04.prod.outlook.com (2603:10a6:20b:24b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8158.23; Wed, 20 Nov 2024 10:04:35 +0000 Received: from PAXPR04MB8285.eurprd04.prod.outlook.com ([fe80::e003:8fb:64ea:acfd]) by PAXPR04MB8285.eurprd04.prod.outlook.com ([fe80::e003:8fb:64ea:acfd%3]) with mapi id 15.20.8158.023; Wed, 20 Nov 2024 10:04:35 +0000 From: Hou Qi To: libcamera-devel@lists.libcamera.org Cc: jared.hu@nxp.com, qi.hou@nxp.com, julien.vuillaumier@nxp.com Subject: [PATCH v4] gstreamer: Add GstVideoMeta support Date: Wed, 20 Nov 2024 19:04:14 +0900 Message-Id: <20241120100414.219130-1-qi.hou@nxp.com> X-Mailer: git-send-email 2.34.1 X-ClientProxiedBy: SI2PR01CA0049.apcprd01.prod.exchangelabs.com (2603:1096:4:193::13) To PAXPR04MB8285.eurprd04.prod.outlook.com (2603:10a6:102:1ca::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8285:EE_|AM8PR04MB7779:EE_ X-MS-Office365-Filtering-Correlation-Id: 29a382e6-2f89-44bf-fb91-08dd094abc7e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|366016|52116014|376014|38350700014; X-Microsoft-Antispam-Message-Info: U+AU+U9JYVDk1Gbv3dZFLNjfSiiKrr2NyUA8Al8KzzV1tQKQLYu3NF/tmWOItf1SkPUyPdRh+5PXEFAUMusUpgGMvuV34G8SezAfObwn+EUhl2P5txk7Wf19b3X+ljJ0dKS+a4RMG83nfsjCbFo91QvH6+g8eE0viHPPbeB4+ZqjuvXKd4lCOrQlDwXrzkOWOAtyN8+TNbeC0k1KETodgfTsYcv421o/S0lvYjbQQIJEJ7rIVODXyUuaAz1KzPnKAithi1gR9194MLLJ6nToJnh3Hgahu3ePjR9bi5J78az5CiGymqKXm4ihcZyqPlMusgWjbWzquKMMETGqeyPDgiZoMT3Ed+lfLf81CesIu+ZWvgHyDTKIjSc13CEf7Td/A7mcYyJxmzOtJTCC3EKPM29cbHT1aYhfcppWAE3IUcIlAe+nTUkokR1g4C++0JvizLYU5XSBzo4CJ1OsBQG1E9jpbB69iMpJPFnHKgT4qlIk5vwbhl+1WnWeNqjjqwuiCFnNmG9UMoxFFrifpwBvYQgy8HnNqfJ2wd4z8uJNhmwcockcXsJsbTjZUJwUuA46riDLXdYoyMp8+6RKuqSqBELgin/FSmkV0nKMVpfEq6gSu8FDCX4cYm91s1LKWAE8ufetbjz+bReL3X7AtijHEEyaDkpA6tnC7Wh3wr+IegAE6GspNtgOMrGbDyHS3xdrBDtnIauz5U5IXJvPIvHKzbixqtSV8nspTci/LpoNhIdalgWxI82r0Y6JNZ5UmVCN6UroUf1oQ9ey57lUubMrMX+r3LtVNQieSicw1DBKToHfRU/d1eb0z/ttVIC3xVv1HMm2kKOQsiQtLEdXPSHw1JmynUeu8A/SR+XVdhr/T0nGyuykgY+2XleByuC2Gnst6azMTunPdh4kB46HirqW5FTDtRBfX8cGAr+GIBlWmoR6rvWbo6U33iN7VVyl9SCuHt8JwrVrBahaGuazJLkbceg3pJjnHVL2/MaMv6d/WBle/5Q9iAEIWPrpgb3BlyFKBqWjU2htRXRZM9JYVrugcBDEIrCiiXrPIM3l+Vwqele6qACb2+cUwHwAO9z3Hzh11idXIydvQ6lVBhr0jRKRHB/uCFrn9uQexOhKiYLsBIVELeWksgl644Us8WxvNvwAIb33bvFjDrIsEPwYdsjlUK+pRJjtzr1SRnEzuct6kv1QyeDV7Lmekiqce5QEW6ClY378auYtXcoj65vWKaoEyjPnOJsb+ZwLTbhrWPtcp6Xkb+EwtNE5cAw6iEnu8E1rPJ0j7v37jhLEQGHxu/3Xn9BTpKL8WAf+LIenf4k84GBnOa9MCJ5kDeq8wYBO3KX0kkzR2a3xtNntAznXBfGVhHFel4abUl/9VsTVyTLe3xA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR04MB8285.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(52116014)(376014)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +6AZ0F8g9w7mBCLw0vRmxlKMJuXfVpZ1H9t5iaKpZ8OOmTjofrWADUwAJxPyaM2lo5vH6dCkHa8GyMY7i2NrPP+kRgkvScOX21IJc0pQ6NaARfWbs+tVsLiCymI7kkUqMRB/8zA/p45V7c0L1AZ8zloKQAVY1oJRnyFhi4YWnI/tZeIdb50G+Do3U3LK+bIoiDEbib1bbPiCxKJoroFk9s4Yv+/YMH8aCdzfVaWSH3wCgDIo5dN+zFT/0dq0kFziE2fMQNyAoaXEEohYb4Xvolls1yTS5mLQn8yLfmVRhGnNkRRhpmKUU1E7NyBs0yyI323rtpEo/WVddOlis1boNAlsfVMBGTibGzwZLrcx2ASDl2bT6YZ9rhMwfZ4bihd+RHTzxgLDA/4jKCC+I2WGNWZD/f8oeGxC4eU0eu/ltEa3jHeDsbxKEU2tfScPcAlk5Jb6mvGgru7KCJ/qep4drjfZJtB0fh0zi05SIUkfP12BTVY5rQK6aRAmjhQ6/1R2rblCL1Lpush4Wr3D+uF9Jj8zVY8PO5Z9xsikrN/g3r6mSMkdywiMHBcE6tYNNgHrXO0Y32pjIY4iccD1/r3KxoULwmkSBQERjd29mE/HdqfgEBQFYMp/BQrir1npg4bOfnBMol3BuZ90nC9CpK/pJO7BNHKp9TtRVqwwSzLvKai6D1tJ0cA5McH5BCbCEUbZCaZSlpHzlVMYNfFxaq8vq/lMRMsNQVGxO14HtSquSliB0UMNjur0J0RV9jKEbDp1RGgA18JMj+MMHgI6ig5/byDEIGC6tx3fR44QUlWHLExo8dW2MPa96zsa0FU8nwI/eEDGZ4DJA+F/Pf0764eztI6DQGMia0c9SZ1gTZnDv+uhEu0z+HaYbPmyJIH7sxcWzEj7tdElPRBzHG+IL0iwaytD+vUUrHYjLUIjGOMc8wEBcVV6vg9XESr0cfTa4iAgZOrdGMRVDdu6pP+8TDHFp0ah/9WuBHJRr5GdLcq9rzX7VBb0KFJMN7NhLmMy6zWS+DW5tVSaBvBTM4IDUqMsn+CQQbg6m9HhkXaXGjTzT/UfZv7m9h2c8t2O5wJCLVmoyRZ4BgygJw+HebD8Czf6vR0Tjt7S5KXguC6qcLBGQdEveeXip5fjvfC+xPZhQHqHTTN6V3tmkso3KJLFQMjMDLkTgQh/OzpUf0n2Oz0J/4wcbVRkC3DkXQ0fhsxk2QiL9HpSguxq/z7Ctz0m9Vf9FwQ5x/jmeAE4qikaoSgPgV4P8nlrJHfsPZ1JfHvmiulxe82kU6rjBzwb/JRhI3GEK/5HBM0QF868d/qMKkVLEVZDlshUU9mYZ3XQUfTjou+PLEA6hkXzR6MWu65N/cwYzZyhNDxIACdZNNzK5LqDIS8ZEmXPWJfmyl/EVy2EaWdAVvRhr/T+R3YyBNwBYl/0cUr/UgZW/MM6QFYtKUb7Z8rVkCYz+YQVbYCbf7Byz49LknOqcwLuFaTM5Fdb5m5oRmdAgZxzMqVeneNPYEq4nu9w3MIuPW67SP9N0oNm6N9p23y8teCoKHvGxzQpc0GLQaA5QJ6xZMcUZGU2wwN3P3Q= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 29a382e6-2f89-44bf-fb91-08dd094abc7e X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8285.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Nov 2024 10:04:35.3571 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: y1dQCdD8P+5h4SvS94Ec0Hoog3sjPgI2hNo0nQUREMJ820T3jOmaiKOajLSoUYyQ X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR04MB7779 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" GStreamer video-info calculated stride and offset may differ from those used by the camera. This patch enhances downstream plugin's support for videometa by adding videometa support for libcamerasrc. It ensures that when downstream plugin supports videometa by allocation query, libcamerasrc also attaches videometa to buffer, preventing discrepancies between the stride and offset calculated by video-info and camera. Signed-off-by: Hou Qi --- src/gstreamer/gstlibcamera-utils.cpp | 33 ++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 4 ++ src/gstreamer/gstlibcamerapool.cpp | 57 +++++++++++++++++++++++++++- src/gstreamer/gstlibcamerapool.h | 5 ++- src/gstreamer/gstlibcamerasrc.cpp | 47 +++++++++++++++++++++-- 5 files changed, 141 insertions(+), 5 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 732987ef..09af9204 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -591,6 +591,39 @@ gst_task_resume(GstTask *task) } #endif +#if !GST_CHECK_VERSION(1, 22, 0) +/* + * Copyright (C) <1999> Erik Walthinsen + * Library <2002> Ronald Bultje + * Copyright (C) <2007> David A. Schleef + */ +/* This function has been imported directly from the gstreamer project to + * support backwards compatibility and should be removed when the older + * version is no longer supported. */ +gint gst_video_format_info_extrapolate_stride(const GstVideoFormatInfo *finfo, gint plane, gint stride) +{ + gint estride; + gint comp[GST_VIDEO_MAX_COMPONENTS]; + gint i; + + /* there is nothing to extrapolate on first plane */ + if (plane == 0) + return stride; + + gst_video_format_info_component(finfo, plane, comp); + + /* For now, all planar formats have a single component on first plane, but + * if there was a planar format with more, we'd have to make a ratio of the + * number of component on the first plane against the number of component on + * the current plane. */ + estride = 0; + for (i = 0; i < GST_VIDEO_MAX_COMPONENTS && comp[i] >= 0; i++) + estride += GST_VIDEO_FORMAT_INFO_SCALE_WIDTH(finfo, comp[i], stride); + + return estride; +} +#endif + G_LOCK_DEFINE_STATIC(cm_singleton_lock); static std::weak_ptr cm_singleton_ptr; diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index cab1c814..81149280 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -35,6 +35,10 @@ static inline void gst_clear_event(GstEvent **event_ptr) #if !GST_CHECK_VERSION(1, 17, 1) gboolean gst_task_resume(GstTask *task); #endif + +#if !GST_CHECK_VERSION(1, 22, 0) +gint gst_video_format_info_extrapolate_stride(const GstVideoFormatInfo *finfo, gint plane, gint stride); +#endif std::shared_ptr gst_libcamera_get_camera_manager(int &ret); /** diff --git a/src/gstreamer/gstlibcamerapool.cpp b/src/gstreamer/gstlibcamerapool.cpp index 9cd7eccb..bd1cc473 100644 --- a/src/gstreamer/gstlibcamerapool.cpp +++ b/src/gstreamer/gstlibcamerapool.cpp @@ -29,6 +29,8 @@ struct _GstLibcameraPool { std::deque *queue; GstLibcameraAllocator *allocator; Stream *stream; + gboolean frame_copy; + GstVideoInfo info; }; G_DEFINE_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_TYPE_BUFFER_POOL) @@ -135,16 +137,29 @@ gst_libcamera_pool_class_init(GstLibcameraPoolClass *klass) } GstLibcameraPool * -gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream) +gst_libcamera_pool_new(GstLibcameraAllocator *allocator, Stream *stream, + GstVideoInfo *info, gboolean stride_mismatch, gboolean has_video_meta) { auto *pool = GST_LIBCAMERA_POOL(g_object_new(GST_TYPE_LIBCAMERA_POOL, nullptr)); pool->allocator = GST_LIBCAMERA_ALLOCATOR(g_object_ref(allocator)); pool->stream = stream; + pool->info = *info; + + if (stride_mismatch && !has_video_meta) + pool->frame_copy = true; + else + pool->frame_copy = false; gsize pool_size = gst_libcamera_allocator_get_pool_size(allocator, stream); for (gsize i = 0; i < pool_size; i++) { GstBuffer *buffer = gst_buffer_new(); + if (stride_mismatch && has_video_meta) { + gst_buffer_add_video_meta_full(buffer, GST_VIDEO_FRAME_FLAG_NONE, + GST_VIDEO_INFO_FORMAT(info), GST_VIDEO_INFO_WIDTH(info), + GST_VIDEO_INFO_HEIGHT(info), GST_VIDEO_INFO_N_PLANES(info), + info->offset, info->stride); + } pool->queue->push_back(buffer); } @@ -163,3 +178,43 @@ gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer) GstMemory *mem = gst_buffer_peek_memory(buffer, 0); return gst_libcamera_memory_get_frame_buffer(mem); } + +GstBuffer * +gst_libcamera_copy_buffer(GstLibcameraPool *self, GstBuffer *src, FrameBuffer *fb, guint32 stride) +{ + if (self->frame_copy) { + GstVideoInfo src_info = self->info; + gsize size = GST_VIDEO_INFO_SIZE(&self->info); + GstBuffer *dest = gst_buffer_new_allocate(NULL, size, NULL); + GstVideoFrame src_frame, dest_frame; + int i = 0; + + for (const FrameBuffer::Plane &plane : fb->planes()) { + src_info.stride[i] = stride; + src_info.offset[i] = plane.offset; + i++; + } + src_info.size = gst_buffer_get_size(src); + + if (!gst_video_frame_map(&src_frame, &src_info, src, GST_MAP_READ)) { + GST_WARNING("fail to map src_frame"); + return src; + } + + if (!gst_video_frame_map(&dest_frame, &self->info, dest, GST_MAP_WRITE)) { + gst_video_frame_unmap(&src_frame); + GST_WARNING("fail to map dest_frame"); + return src; + } + + gst_video_frame_copy(&dest_frame, &src_frame); + + gst_video_frame_unmap(&src_frame); + gst_video_frame_unmap(&dest_frame); + + gst_buffer_unref(src); + return dest; + } + + return src; +} diff --git a/src/gstreamer/gstlibcamerapool.h b/src/gstreamer/gstlibcamerapool.h index 2a7a9c77..ffbd3173 100644 --- a/src/gstreamer/gstlibcamerapool.h +++ b/src/gstreamer/gstlibcamerapool.h @@ -14,6 +14,7 @@ #include "gstlibcameraallocator.h" #include +#include #include @@ -21,8 +22,10 @@ G_DECLARE_FINAL_TYPE(GstLibcameraPool, gst_libcamera_pool, GST_LIBCAMERA, POOL, GstBufferPool) GstLibcameraPool *gst_libcamera_pool_new(GstLibcameraAllocator *allocator, - libcamera::Stream *stream); + libcamera::Stream *stream, GstVideoInfo *info, gboolean stride_mismatch, gboolean has_video_meta); libcamera::Stream *gst_libcamera_pool_get_stream(GstLibcameraPool *self); libcamera::FrameBuffer *gst_libcamera_buffer_get_frame_buffer(GstBuffer *buffer); + +GstBuffer *gst_libcamera_copy_buffer(GstLibcameraPool *self, GstBuffer *buffer, libcamera::FrameBuffer *fb, guint32 stride); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 8efa25f4..1d46536a 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -292,12 +292,17 @@ int GstLibcameraSrcState::processRequest() GstFlowReturn ret = GST_FLOW_OK; gst_flow_combiner_reset(src_->flow_combiner); - for (GstPad *srcpad : srcpads_) { + for (gsize i = 0; i < src_->state->srcpads_.size(); i++) { + GstPad *srcpad = src_->state->srcpads_[i]; Stream *stream = gst_libcamera_pad_get_stream(srcpad); GstBuffer *buffer = wrap->detachBuffer(stream); + const StreamConfiguration &stream_cfg = src_->state->config_->at(i); + GstLibcameraPool *pool = gst_libcamera_pad_get_pool(srcpad); FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); + buffer = gst_libcamera_copy_buffer(pool, buffer, fb, stream_cfg.stride); + if (GST_CLOCK_TIME_IS_VALID(wrap->pts_)) { GST_BUFFER_PTS(buffer) = wrap->pts_; gst_libcamera_pad_set_latency(srcpad, wrap->latency_); @@ -497,9 +502,45 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) for (gsize i = 0; i < state->srcpads_.size(); i++) { GstPad *srcpad = state->srcpads_[i]; const StreamConfiguration &stream_cfg = state->config_->at(i); + gboolean stride_mismatch = false, has_video_meta = false; + GstVideoInfo info; + + g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg); + gst_libcamera_framerate_to_caps(caps, element_caps); + + gst_video_info_init(&info); + gst_video_info_from_caps(&info, caps); + + /* stride mismatch between camera stride and that calculated by video-info */ + if (static_cast(info.stride[0]) != stream_cfg.stride && + GST_VIDEO_INFO_FORMAT(&info) != GST_VIDEO_FORMAT_ENCODED) { + GstQuery *query = NULL; + gboolean need_pool = false; + stride_mismatch = true; + + query = gst_query_new_allocation(caps, need_pool); + if (!gst_pad_peer_query(srcpad, query)) + GST_DEBUG_OBJECT(self, "didn't get downstream ALLOCATION hints"); + else + has_video_meta = gst_query_find_allocation_meta(query, GST_VIDEO_META_API_TYPE, NULL); + gst_query_unref(query); + + if (has_video_meta) { + guint k, stride; + gsize offset = 0; + + /* this should be updated if tiled formats get added in the future. */ + for (k = 0; k < GST_VIDEO_INFO_N_PLANES(&info); k++) { + stride = gst_video_format_info_extrapolate_stride(info.finfo, k, stream_cfg.stride); + info.stride[k] = stride; + info.offset[k] = offset; + offset += stride * GST_VIDEO_FORMAT_INFO_SCALE_HEIGHT(info.finfo, k, GST_VIDEO_INFO_HEIGHT(&info)); + } + } + } - GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, - stream_cfg.stream()); + GstLibcameraPool *pool = gst_libcamera_pool_new(self->allocator, stream_cfg.stream(), + &info, stride_mismatch, has_video_meta); g_signal_connect_swapped(pool, "buffer-notify", G_CALLBACK(gst_task_resume), self->task);