From patchwork Fri Oct 25 03:37:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Hou X-Patchwork-Id: 21750 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D92E6C3220 for ; Fri, 25 Oct 2024 03:37:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 135D465394; Fri, 25 Oct 2024 05:37:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=nxp.com header.i=@nxp.com header.b="PhJ7vIn6"; dkim-atps=neutral Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02on20624.outbound.protection.outlook.com [IPv6:2a01:111:f403:2607::624]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AF72C60366 for ; Fri, 25 Oct 2024 05:37:51 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Mu7ocDykFwPogryXZUTOoEt2IO7L6R6NVqRYx1zapmZ64Skxt3OIkuVmTdDEUiK+tNW9F+eQfG7zXie6f2mAbQHvs/JXa6htQ70KpWDpK98dJzMAyIX24WmW0S+EosYszBMcT+GhBh50nzoqDhqyhmRd5O9l4L1iP0QJlRKGeSwkU/TWLKacIDS/Eq5qLI88zI7OyT1pMhZCEi0pxZImSuemXw2D9ZmdfPpPpPu7ZLvDifb4foCrqoUGIP2MRuLTT+c5k55OBCdHEKmhLAKXvrcHeoclV8BNEOtbvSWMJuVT976IztgZN2ir+A+EAQOjNjFwZY8a+Z2+ntBv6Po32w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R9tGn50xdZ6PJMkTSLKg/I3sCkS9sUV1TNkp+dF7MxI=; b=dl2oSRDu8V7C/R0FuAUotQ+aM6LW/Nqw1n4yTdkomq9ZEg6bzVwoNZnif+ZntgXKde1JvyX+8oLoxoaLiTvT9MbwLOhr5YX0sw1y0FmJggRuslH4Lbj45LBuEZ4BIUzulPiRlE7CuJYFXNDXV0rqlJnmrq7q1/9NMn+T7B7CNRgaqM14KexSzPAOpL8fzg+Ay7wjHkwoRjXFGOzbFm9LkYyxy3yN7VWgolwbtsIz3tPfhTJielI3Ln8mjUId2LW3YWSQViMiQLV6yT9hqJXtwGEoZF9wKx3mNYWIf+T9Aq8HbBdAx6w5a8D+dvD/TqcxvHuX/ZjcjZG0fqI8UmXVHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R9tGn50xdZ6PJMkTSLKg/I3sCkS9sUV1TNkp+dF7MxI=; b=PhJ7vIn6+k8OJITfI6MFL072hq9D2S38hK8Nk8P8jTKE53EkFLezfASuf/uZ475y4P4Jqtz4t2aDg3JwQ0i3cTbjwQRvHREqn/8KfXJOdeINnf39J6S5WH2XrwgwukvH+PkLoLQhX2wF/+n6ODryHV/aRqIWvFQefzX00naA55qABhlj+xlBvMPk6IXAickhZUw7xH7oj02DA4jbBLwqNtFuTCkftVUneoHa4ksbNECZhB/0pubnFauTJ/TWZRIkoiai3jbUBYLhdP9qPvKeXYZdXsfFS2cBErzQUDFmqTAFjemekY5WeVnU5LktvNaccMUt0uLGIprplJoa/u7kJQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from PAXPR04MB8285.eurprd04.prod.outlook.com (2603:10a6:102:1ca::15) by AS8PR04MB7717.eurprd04.prod.outlook.com (2603:10a6:20b:292::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Fri, 25 Oct 2024 03:37:48 +0000 Received: from PAXPR04MB8285.eurprd04.prod.outlook.com ([fe80::e003:8fb:64ea:acfd]) by PAXPR04MB8285.eurprd04.prod.outlook.com ([fe80::e003:8fb:64ea:acfd%6]) with mapi id 15.20.8093.018; Fri, 25 Oct 2024 03:37:48 +0000 From: Hou Qi To: libcamera-devel@lists.libcamera.org Cc: jared.hu@nxp.com, qi.hou@nxp.com, julien.vuillaumier@nxp.com Subject: [PATCH] gstreamer: Add GstVideoMeta support Date: Fri, 25 Oct 2024 12:37:18 +0900 Message-Id: <20241025033718.2387920-1-qi.hou@nxp.com> X-Mailer: git-send-email 2.34.1 X-ClientProxiedBy: SI2PR02CA0051.apcprd02.prod.outlook.com (2603:1096:4:196::10) To PAXPR04MB8285.eurprd04.prod.outlook.com (2603:10a6:102:1ca::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8285:EE_|AS8PR04MB7717:EE_ X-MS-Office365-Filtering-Correlation-Id: a9571e1e-3e5b-4925-22c2-08dcf4a6658d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: RcbD6/6IzBCrtKb6aOwNvmGzWN1RhjAd1MIfoJ0Q1f9gXKKDGuwQUm99ASI8r/SQQKdP929gphUoNXlwTuKkwOkXRIrNPrX0JOuvMiXFePe8STL2jBIfeviUy/5HWtP5TUXNEYe9QdE9pVleDgJ9IWuplsCoPMkDtGbJ7NVegpJUloTZfTgL9F0rfx45VqMP8kUveT7pS6CblnCsfhm6YVVrs/EAlQ6wfnzaI/b7W94AfTdODl/lgyj9D17v4QI2SHWcvUThc86j+rux1vx82SnmOJj3nYAgBtWtdFBH7T1T5p33AGRC5DUc1D0enRN+RlCAwDm+aU8eOWYys0FmQymQ2D6mCzJgkDk7da0b0HP58ZoCyMlGM/IHTxOBKEaF0N5Cq2A6LPhG/OBSJK1XIBVS7TJvUs1Dhg4pwAKq2MRB+G8HLXyrZ75tveEwiDjT2HKTwECG2sZPMc1KW9M3KCSffnYrRscYC3B4KimMnMaK7Jlk7uP3BAyp81mDA0zJwfuLni6N5gRBAI52mrEW1wVb3kdrTTaxxQtpb1LrlR0LxMig8mfISijBAFUJTmNtgPmfUcIMG5hZeTHWJfYQgyNuRekPI2FHJewj7ayMiDb+SvPzq2NRAWXYH1Fq5TUCJIzT2bK3Vvu6guUkAltaO3BLXVvtzywuKQpMrX4Cios2g11jN1wOY+rpnReSTv39l029BnoiHSmK+pLaMIswf0QRmSU55LsbZzYcWTWLIxy+LjL8vxss/o0ObLadkGt1c2Qy5ZgcWIrGIoPFF9IUFxbIfUrNwLGlcrooppbWXZYz5LihSXIYJ2ObOVHWqKiRxLm22ptViJSZyB+92O9bh1zx1p2yuGeGaqjt7YyKnI7pHUM5DlNiqzHh3yESzIv1nwhyJnayM8HpUDHzzkIhO13TmSMLsw94M9mS0QgxKuxhI2TSt69dlsKgvykW/EksDVuJD3ANFWc8Hx3iCAzyA/6Pq+9By0uBCUiF65BNtdpFWhD27cz1vF8pkEEYHh0jpr4H0PgtmdFFm/KEb7wvC5jiVeJEjzbZEtCXT8vcmGViVb9bOhmIkOrDw2RY24NELBnucqj4utiayMsGpd3aObVqfPIRdJQPwKaOnf51sLU5coy3G8Huiu66du/S5UZoBLsTG2MOXl5BIgYd64bsg30DfGo6GMuFryYCKn2HyaFQx/WXKUIl59yxw0uUa6tAB7hYBODj6GCcQlW+rUJS9wvZl7Vc4LULvyYD3eNXAUzRBKmu4zAgazvpVg6Uv8Xq/t9XZ792yQvYfvZPNs+fvwGTIiF+iEcmjrEfVxEne1b/BGoxbTldSahbOTSTi6mrk3v1cNnmyS7sqKnDXwyCHq5I7J8kGURxlymybcsFJdI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR04MB8285.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: O5lwQo9aRjS9xOmcsQTvypf5d6atH/HQy3l++u4MYcKjBXxl1dhQkoFZixJ+1oj+nr5CCoxjjcbEo5RPlkBoAwOOzHo78lPTxnNLJJtZFfJbMjqWO2po4DuycRuWpXI+y2PdFP0EazWKKdEITjQoJu0LwaQZ3KFS+qziR2h+/3ObwK8Q9nZ87JUzAAqBHzIl4JR20xhNyB6r9GpSPk85rzUEWO4DL8ttsSk72a5bsC4/CXHMDzfeKwSSXs7RbHiEHrxme0GTgT4TI8TXguNpZUil5SFetE+Y4Cvxyw6Ua0JW3WpQVmMFG6jDf1IiODNwe9FS7/fHnlGtKMRoA4wSK77sm51qblN/GQXRcao0DdJRAzV+11MGD8W67P/zUunD6eKBTJZGfueRcT469c62X7Zgct5VyN4SCUhVjU0zLWWO1dZlz7l5crmBV6tmZarS7LWl22qA0trclXjf9kNjicjkOy79YMs+L7nT2rsvs51AdEofNfiocmTqYjr5JcdsnutyvQHIwpTxBnfsF/uxoGc9R6dughXGYD/YddB9n6NYpGHKtIQp/54paYjdzwcWrX7MPIrq4ZsoY9w5bOdEenLSw+t6jsn23GH2UvPRHRVQ8fqayXdEJO2SMs3xWppCwBXvNYlx+lFhuEkBxeV2gN/hqcgBnkgXbm3UYey9//GALTMbDMIrcKdNhtLsO3J86/a2LeCa1P2zqLxTQ85/R1mOvEX6BElu7QXSwMQfseQ0rIM60PqHXEI0XmCSjC88ODVoAcYKpzV3Uem9TkrX+HAV9mm0zlL//iU4JESaOOdc7DwN3UkNDvs+zXrrbVRlwrOqbXCZ1j6OE0JBEawUWZvDOZqdc4U9zMV/CH3pjwzozq+7pffT1wBSkTp79AjhNhovlS2lAnV0/eOWtRwDwKWQ7Yog9T36P6c304R5WmdazRCRqsB7PKD24O8NprTXbn1ssG1cCEQIjGYi/BpligVGEs615uRqzvJ0opR02Ev878O7SQ91pYWcuMA67fzkZO1CgA+iZ0GUqZ55y1rddYfYVjVg2LsVbp6+irYGpogTvIBoCIfUDM0K/Iqf6d8KoQiy5vmjYoLDMWA2bouJTAM+Een5667WabE+axTGkhLFJO5JJWhMxzwlSTy/V4r5HwSxLKZPpvwUND+77h0LrpBFCcDgkaZWpOpxD0W9DYLwcAjYNK/+WG6oldsvNuIwVbebE5M91hpLAIAS3pmI9LRppvCaLAEwOnYZFFZ29ZFFZTisFGQ4yvRNlMrryV0AmRXQH8s8laOj7EsDMMYeboySMRsW6u10pBXRFSVYumEp3Fv90ApqvaYP1dehoCy3GVO7TqSTx0Ng9fWrmUWtgNmiqdyIrJv6UN/iCJipECvEC8m9LPLSBTDwInQjGBvHprkBid5Le+7ncU53Zfe/IKpRsPjtZ+peRw3hWuP8HE+kot/eN3A+Ff6CVbKOB1aH5pwQvMlSvjgTkB9NGCLnJl6ww4Y6P2o4+ycsQvoIoYIpMiYFddUDs/hlSIIghFiNYPjS/Hnm21CoatSvoaq5iWKcliY5WEyECTMBk9AmJMb0fXXDZ7DzzJpWj87G1O6v X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: a9571e1e-3e5b-4925-22c2-08dcf4a6658d X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8285.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 03:37:48.7047 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5F1yPthWXYLsQLWSGrFStZ+hzpMog+dDAXUqbKGw6BNQZ4FAL18/Da6lqEjA70dH X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB7717 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This change is to provide correct stride and offset for downstream plugin who needs GstVideoMeta when camera is connected to waylandsink or others. Without GstVideoMeta, waylandsink uses video-info calculated stride and offset to create wlbuffer. The calculated stride is 4 aligned with width. While camera stride got from driver reported bytesperline maybe the same size with width. There will be mismatch between camera output buffer and wlbuffer if camera generated size is not 4 aligned for multi-plane formats such as NV12 and NV16. It causes display abnormal. --- src/gstreamer/gstlibcamerasrc.cpp | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 912a8d55..c357dda1 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -285,10 +285,30 @@ int GstLibcameraSrcState::processRequest() GstFlowReturn ret = GST_FLOW_OK; gst_flow_combiner_reset(src_->flow_combiner); - for (GstPad *srcpad : srcpads_) { + for (gsize i = 0; i < srcpads_.size(); i++) { + GstPad *srcpad = srcpads_[i]; Stream *stream = gst_libcamera_pad_get_stream(srcpad); GstBuffer *buffer = wrap->detachBuffer(stream); + StreamConfiguration &stream_cfg = src_->state->config_->at(i); + GstVideoInfo info; + g_autoptr(GstCaps) caps = gst_pad_get_current_caps(srcpad); + if (!gst_video_info_from_caps(&info, caps)) { + GST_WARNING("Failed to create video info"); + } else { + guint k, stride; + gsize offset = 0; + for (k = 0; k < GST_VIDEO_INFO_N_PLANES(&info); k++) { + stride = gst_video_format_info_extrapolate_stride(info.finfo, k, stream_cfg.stride); + info.stride[k] = stride; + info.offset[k] = offset; + offset += stride * GST_VIDEO_FORMAT_INFO_SCALE_HEIGHT(info.finfo, k, GST_VIDEO_INFO_HEIGHT(&info)); + } + gst_buffer_add_video_meta_full(buffer, GST_VIDEO_FRAME_FLAG_NONE, + GST_VIDEO_INFO_FORMAT(&info), GST_VIDEO_INFO_WIDTH(&info), + GST_VIDEO_INFO_HEIGHT(&info), GST_VIDEO_INFO_N_PLANES(&info), + info.offset, info.stride); + } FrameBuffer *fb = gst_libcamera_buffer_get_frame_buffer(buffer); if (GST_CLOCK_TIME_IS_VALID(wrap->pts_)) {