From patchwork Thu Jun 5 06:37:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Hou X-Patchwork-Id: 23471 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C83B1C31E9 for ; Thu, 5 Jun 2025 06:38:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8111F68DB3; Thu, 5 Jun 2025 08:38:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=nxp.com header.i=@nxp.com header.b="bclzELB0"; dkim-atps=neutral Received: from AS8PR03CU001.outbound.protection.outlook.com (mail-westeuropeazlp170120005.outbound.protection.outlook.com [IPv6:2a01:111:f403:c201::5]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AC02B60342 for ; Thu, 5 Jun 2025 08:38:05 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q8wnjby9gkTaztK8Aq8bUdx4QmsmwM/ZtGrGRqzkX/Vxby4BCcSunc3toJmZvoHW4suskz88iXLeDKeINEZC7mdzYQD7H+VX2glOYBCDRco5mn+i8qA12TgdSyWq2ukahlHbQ+Z4ZKZhkjb58oeuQ4w0XgvPuzoGnhQdggfuT2PGPitl/hVGYWYIEpvYEFvhT5O4l18V+7E7NFIgZPe+Z5UVvWzLWbujnfPQj0eDNgu4HP3QaN6+gak378mCwHEPd4hWOjuU9mcK4RHmk9q/K3P7NUoMSfyykQT4fi57OcyPFxpj33JeKUv8L1CeKf2hBb9J26dz/Tpd/ltPoCA+oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EqmXcjAYZVL5m+QPFUIxvZOq6t5HSzACQvfffgCWyxI=; b=YqPz1aCPHzE6vfs3RkxVUIA8UlHXKmGyi7U7F4GJwgkuVcwlAP8pw/3cJLdy4UKWu3uchFwFTl6brYlnOBQt4ktT9ouicbWdnY9cTvhLZjpKnYH9hdhStLS/X3zy16MtAe5ji5udac2khRP8piPjFTKR8691AEou63dVR3X7IKrgJOeh+9mmGdwPobJw/wTzvWr5u9Vy5FcHR6OvbMjZaRwT4vzOkH6Uj2oZ9pth/NS0bt0Z0TBlWyWXmAeKfHqKySRyhkQzX/22a/UlgRpLWaceqEQvtDrgnTVPRRuAqkNAgRdEsFyKNc/c3ec5fz0hLBPZv1vC1KKle7FzWYj3rQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EqmXcjAYZVL5m+QPFUIxvZOq6t5HSzACQvfffgCWyxI=; b=bclzELB09kDjI7NKWCcEUQ7wfLxDusNQdEWCJKYtVxbRhV46z/KYg7W8PKYdo5jJ/C8i/s0tbWbVe3Z2Wl8yDBO23BObrWJUe93EXiUtjlRWnfX/VisXBu1s6IkkHpTS4SumZlBx9SmxIiEdGZw0Sg4nMgw6NueoO0qbNnXLZKCN/dBzic1FKY0drp+heWVaaM0KasJhtKs/xQCguncvrygil7Tu3zIcQ8COkT4qwAcWZRu2YAnfOPCovqyKqqEjqm8WHrpkH0z5fjFACqbjtYpu85UmPDfp3h2As9UC5gWst3zuA288pGtSXx5Y2O3P4eVvO0mMHDxe65vqS6JKzQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AS8PR04MB8278.eurprd04.prod.outlook.com (2603:10a6:20b:3ff::22) by DB9PR04MB8169.eurprd04.prod.outlook.com (2603:10a6:10:25d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8813.20; Thu, 5 Jun 2025 06:38:01 +0000 Received: from AS8PR04MB8278.eurprd04.prod.outlook.com ([fe80::8e33:9371:b4c5:c059]) by AS8PR04MB8278.eurprd04.prod.outlook.com ([fe80::8e33:9371:b4c5:c059%4]) with mapi id 15.20.8813.018; Thu, 5 Jun 2025 06:38:01 +0000 From: Hou Qi To: libcamera-devel@lists.libcamera.org Cc: nicolas@ndufresne.ca, jared.hu@nxp.com, qi.hou@nxp.com, julien.vuillaumier@nxp.com Subject: [PATCH] gstreamer: Fix libcamerasrc responding latency before setting caps Date: Thu, 5 Jun 2025 15:37:41 +0900 Message-Id: <20250605063741.1527483-1-qi.hou@nxp.com> X-Mailer: git-send-email 2.34.1 X-ClientProxiedBy: SI2PR02CA0051.apcprd02.prod.outlook.com (2603:1096:4:196::10) To AS8PR04MB8278.eurprd04.prod.outlook.com (2603:10a6:20b:3ff::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8PR04MB8278:EE_|DB9PR04MB8169:EE_ X-MS-Office365-Filtering-Correlation-Id: 17d255b5-9ca9-46ee-3487-08dda3fb843d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|52116014|366016|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: 9VYxP7uxQ3YgMKKLelNXCLsT4Pqg1NAJZlXZJE7i2B4FPyidv4DlP84bwuz6kfHHoMxUntnvwRas/yONwbKgkhfa6Z1+XudBI0Hzw74dGFCHkwtor9CSrkJMWDrION87EEK3ChS2s21V+EYzEWoi5uf05eSqtQCH3JZXqmf37ymGPVDzh7f5bSsxPtoD8+pTbvBK442s4CFDPGd30R7+YiDa/jdnztAJkD1wwXF1BE6M5i3sInMUm3R8d6AQgstYToJmgdeDReryqqlTY9YwtwhPsXN16Der7ngyj5EBnfLb0fdRKpKLQI3Z48eyhu78uZbZO5bP7eOXXoMESx3Mxs6Po3PmVM5LubnWrIcarpuZKSCDYJpV8K/fv/FFZHGNRzQiMw+qaIMz7bdViwl1DZluzyMqPYGZanHEsty8yWG6dRLHc8h9FFkzB92901lRERmf8kmVdFfjqHQ2ofEtEy3tUlPDRmg3JcdGkALS6MlyInkKSSlR1i27uAckbhiK5xMyh3RzjRBnHfynL1QxOw1VH+F2ArgwHY8HeXqqy0i0JJqdpszDAHvcOFS8RxdirRkkYMbVfBP0J3s8fir+VXhHAaZTyj3JPZR+9YUju5n1Obmj5kSMW+172PSwgJNJPR2PSEct547QQD5x2HNkxVncz20O8ffHhLCK5fCS5Z1q69MzYBcbd11PZcqunsUKPPcNaXTtBaxNu01IdR7swk1scrT+P4tjZtfCGBo10D6yh9zm9P2+SqJ+CYRmF1ppxA5RvltDGVNwcHYiSIpuhujxHDqwMsOUYrvm/CRnx1UIBNRm+h3Gx30Lk2K/P9Sm7mje2CM2XJIjbnYKBusQtoo0sEK8dnWWmSVUsWDzpAaMvP4w1GJrSk+GRxs0w1py0UOBHegTMQfD9EEcOhbsdjmBog/Un1r3l6mHsb6K2566s77Rlt86hu/9U82twTECp0fmPEuOZRc1/Lz3mD0u4Nvs6uptAULo0s3EUfczi2aFsQDlOnMaZ7+MvGqDEjQbqncoLIg4IWzcTywQAEhwWkVVG/bUYZS5gKyfEwL/5cKVxfXc6IvxEnJNLzf2Ut5SRWMnTxfLJEleCKrlS99fmolKdsp5X1lqADhyFsxpOj4jZ/jonfpDvb5f1luEevm40vewG67J8Xj4G+2Eu41+OvoLi1ZegYe9Klb45FAr9c7Xl7zk4k3+dLfpwOgDR3nz7wRFtk0nOwhUUD/Uj8ZQLrP3xCeC0Tl0ArKQYXzl1vCkSdgwhXeC1IqNu4iO+X/9HK7uyUxI6x9NIyIyVc28KWK8IQH7cM0eiVlRxwgXfbPHQVqW/IyBI0JSo7ac8rhbAIbTJypH2VrtHqxw1aswU5mroQeLrvobX3jJi45olE/mvsJgAE6RvaM/BQxjqrrUVJ2sbD0aLQbmlhRDUA648db7qOmVqjfeiQ691vfORkr2oGqTvENFSX1UeDL295q0GEsBpVtCkULjkizpls68ZXhx7cStsMz6zRykjN1gqsw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR04MB8278.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(52116014)(366016)(1800799024)(38350700014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6g/39FDjkO1MCd8PUF7iNFnGIS8vlOMRX8njWxX2pFX5dsFyah3VrSKeIE8TpYHywi9l0pgS1PHR4ritVSDD3TRkBLQmNeuERAN0ZaOsCd/1OI1yrwIfKc0y0VvFaF3g6l4pOrgfLc2mfHP68MLmHvEgpJchUEJoPbycC/TI8gI6gOwfKxkLKRrY9Hdroj8OQ80DmFtgu0pwSpGt+v8IIizUJ4z+/8t7SLP6Y2WVcjSA6mhgUomNk/8XHeXalOhVymoqY3TxKPqNm7ne5Iwn4atYUmzjmRpDniO0Kj2TLjruR1B/ZmnU+vzGF02nlOUU/2UiyFrDusvH07i8zK4we8LHbL2SCVYwAMoQEpqB5+YSjtqMlcNQ3+Pc/cbhY/sO+VTyr0OeFDsVlgIFJ1Id9UoKfb3b1z7+tdOcJIE/IJQOXg+vyUj7bMbocaZ124mEHl+SCMcF68KRi5UrWckTvkGAosJz1Yj4X3xdLLVuNVBr3o5EABAfDKYOZPrH9Q/v21RMLskfl1qmdHXdqKljYpQpoF3Coq2kPcIwHLSWtA/sU+LxQ4b/glg73Nq1BC84ibXA0WB3OM8q/XzNsHySRJUTrBYSB+AVWWVXmdndPmlLjaghC/fBMhZic5MyG0S6yAGD8zQV9raG3WBh14Y/PQeoxZZexeSrYyIgFOTRvY3nfLkb218VewGUBoMyC1MBVn8Y6AdnenJNUfHPR2JxtYpZRdukAgn5phAFkwGZ6E/cT6RAzCCKEsXPbFu4zsPNBsRTOZ/AZSAnf5E21RLYRSOfAWSPFYAK5yun8ehfCFlOzyWBTakSPi0yRiB4emGdwElBT3lbkB7FiKCYcbF7DuRJ3GBeQAAf8I7kypPI+RjIYeVaTwMqVVekQCU8AOLwmXcTP64RCAQsPOMjiPw5f0ajxeeNrAeh5mYqkJJ/oF6FwE4jk46/otKoa/ygtA8qFvvjjn8qaDCy3VQQWEJ+PbhxggDpjR3IXCCZw5LqB1fmaLWRSwFWJqq+TPwBuqwr1aoEJ2lxpx5dJyGWl11j6fEffxfd4y0+G4ah66niBCFIqO63mtN2lUpaF9Wz0nYhFXEvi5x8WVzr7Jhkfi9+KW7Lat8vNHuMRkkabHanvIl4rGvGqHKEgDPMr6+a4zGKKEGEU7Vbja+PXmnEWPnNVnZImKOIp1fOWdRgM6xUJIpps3SffuzLOssurOFlsX/LWaVX4rsjGZxLZS0OISNMGiwd+1QlY4czqnwuT0ER6coDnICRPolMUD9GHLF5VmxD5pfSIGBsXk54kYUfLNna+b+1ZRsVqwC4mdIFoxx53NkJPu8YD4bsSMte+vUQNkNZZoRDoXPbZ7oFbS/yUh13j0qUssWg31W8sYm+uHmImspqUnO6nvBqgQbe34ij4DDRABVvdzaPa5qs6HVgT96JpfPXgjnfScU9yE40FkP44kH2RekwaOGqzKZWe09MK8HdMVtg5I/3pg5f1eJrzA2TwD1SnHmnbNpPFhaL240yEnpi5sihuxjgAfNBP3csoi/hlugWHqgKCsAJhpLoce7F/hq9ZI8dg6gO0o6NvU/6HiwqB8DlLkRcU17VroC9rwyq X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 17d255b5-9ca9-46ee-3487-08dda3fb843d X-MS-Exchange-CrossTenant-AuthSource: AS8PR04MB8278.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2025 06:38:00.9867 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hY/iz9AUmj3X9sJUA3HjEo8ub1dgpP/fEwkPKz7BVK8P7nkmNIzgTKLo3NgB7jDr X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR04MB8169 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Whenever a downstream element queries latency, libcamerasrc will always reply, even though it has not yet determined the latency. However some downstream elements (e.g. glvideomixer/aggregator) will query the latency before libcamerasrc sets the caps. When these elements get the latency, they will start the caps negotiation. Since libcamerasrc has not yet determined caps, invalid negotiation is performed and workflow is disrupted. So, set latency to 'GST_CLOCK_TIME_NONE' during initialization, and reply to the query after libcamerasrc confirms the latency. At this time, libcamerasrc has also completed caps negotiation and downstream elements work fine. In addition, every time the src pad task stops, we reset the latency to GST_CLOCK_TIME_NONE to ensure that when next time task starts, the downstream elements can generate out buffers after receiving the effective latency. Signed-off-by: Hou Qi Reviewed-by: Nicolas Dufresne Reviewed-by: Nicolas Dufresne --- src/gstreamer/gstlibcamerapad.cpp | 5 +++++ src/gstreamer/gstlibcamerasrc.cpp | 4 +++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamerapad.cpp b/src/gstreamer/gstlibcamerapad.cpp index 3bc2bc87..37a4689f 100644 --- a/src/gstreamer/gstlibcamerapad.cpp +++ b/src/gstreamer/gstlibcamerapad.cpp @@ -72,6 +72,10 @@ gst_libcamera_pad_query(GstPad *pad, GstObject *parent, GstQuery *query) if (query->type != GST_QUERY_LATENCY) return gst_pad_query_default(pad, parent, query); + GLibLocker lock(GST_OBJECT(self)); + if (self->latency == GST_CLOCK_TIME_NONE) + return FALSE; + /* TRUE here means live, we assumes that max latency is the same as min * as we have no idea that duration of frames. */ gst_query_set_latency(query, TRUE, self->latency, self->latency); @@ -81,6 +85,7 @@ gst_libcamera_pad_query(GstPad *pad, GstObject *parent, GstQuery *query) static void gst_libcamera_pad_init(GstLibcameraPad *self) { + self->latency = GST_CLOCK_TIME_NONE; GST_PAD_QUERYFUNC(self) = gst_libcamera_pad_query; } diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index b34f0897..b57d4a55 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -835,8 +835,10 @@ gst_libcamera_src_task_leave([[maybe_unused]] GstTask *task, { GLibRecLocker locker(&self->stream_lock); - for (GstPad *srcpad : state->srcpads_) + for (GstPad *srcpad : state->srcpads_) { + gst_libcamera_pad_set_latency(srcpad, GST_CLOCK_TIME_NONE); gst_libcamera_pad_set_pool(srcpad, nullptr); + } } g_clear_object(&self->allocator);