[{"id":35533,"web_url":"https://patchwork.libcamera.org/comment/35533/","msgid":"<20250820194416.GC10430@pendragon.ideasonboard.com>","date":"2025-08-20T19:44:16","subject":"Re: [PATCH 0/3] Fix fast-booting device enumeration problems","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Wed, Aug 20, 2025 at 02:23:13PM +0100, Daniel Scally wrote:\n> Hello all\n> \n> At present, a CameraManager that is created before the device nodes for subdev\n> entities in a media graph have been created (for example, because some subdev\n> drivers have not yet probed and so the v4l2-async framework hasn't called for\n> them to be built yet) will never match any pipeline handlers for that media\n> device. This is because the MediaDevice populates itself with entities,\n> interfaces, pads and links but ignores the fact that the interfaces for the\n> subdev entities are missing. When DeviceEnumeratorUdev:: populateMediaDevice()\n> is called, the entity's major and minor numbers are checked, seen to be zero (as\n> they come from the missing interfaces) and then the entity is simply ignored\n> rather than treated as dependencies that need to be watched for.\n\nI may be missing something, but aren't the entity and interface\nregistered together on the kernel side ? Even if the device node hasn't\nbeen created by udev yet, the interface should be available, with a\nmajor and minor, in the media graph.\n\n> The MediaDevice\n> is added through DeviceEnumerator::addDevice() to be checked against the\n> pipeline handler's match() functions. When the device nodes for the subdevices\n> are eventually detected as v4l2-async completes they are simply placed onto a\n> list of orphaned nodes and ignored.\n> \n> To remedy the problem this set checks for missing interfaces during\n> MediaDevice::populate() and if it sees some are missing, allows the MediaDevice\n> to be created successfully but leaves the isValid() flag false.\n> DeviceEnumeratorUdev is updated to check isValid() and when it is false, defers\n> the creation of MediaEntity / MediaPad / MediaLink objects. Each time\n> udevNotify() is triggered by a new device MediaDevice::populate() is checked\n> again for each deferred MediaDevice, and once the missing interfaces are found\n> in the topology the other objects are created, dependencies resolved and the\n> MediaDevice added as normal.\n> \n> There may be a better way of receiving notice from the kernel that the media\n> graph is fully complete and ready to be parsed; at the moment we rely on the\n> reported topology version number becoming stable across two iterations of a loop\n> that checks MEDIA_IOC_G_TOPOLOGY, but that's not sufficient to avoid this issue.\n> \n> Thanks\n> Dan\n> \n> Daniel Scally (3):\n> libcamera: media_device: Expand return values for populateEntities()\n> libcamera: device_enumerator_udev: Add initMediaDevice()\n> libcamera: device_enumerator_udev: Defer invalid media devices\n> \n> .../internal/device_enumerator_udev.h | 3 +\n> include/libcamera/internal/media_device.h | 2 +-\n> src/libcamera/device_enumerator.cpp | 2 +-\n> src/libcamera/device_enumerator_udev.cpp | 85 ++++++++++++++-----\n> src/libcamera/media_device.cpp | 27 +++---\n> 5 files changed, 85 insertions(+), 34 deletions(-)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 395B7BEFBE\n\tfor ;\n\tWed, 20 Aug 2025 19:44:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6C8506924B;\n\tWed, 20 Aug 2025 21:44:42 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4DC406142C\n\tfor ;\n\tWed, 20 Aug 2025 21:44:40 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 436DE63B;\n\tWed, 20 Aug 2025 21:43:41 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vobWS8jo\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1755719021;\n\tbh=XQe+Gz2M3If0kjSz00E1PvhUxyuZAFcrrvC4sDchMkk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=vobWS8jouxQ6Rjw5DPSnkkYfmb+sSRJWZ+L+nd/SNJl1TdmroVpspcnVUIgXs+x7b\n\t3SOLhwbKuFJLhriADSX0wPrbHnkKrM39Cy2sco3WLySMSpz2BvXI4GAJgsrRBENWAr\n\tfaQJJ8XH1Y/DOhe3hWNn1Dt4DHFkhRjhG3Y+hCbQ=","Date":"Wed, 20 Aug 2025 22:44:16 +0300","From":"Laurent Pinchart ","To":"Daniel Scally ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH 0/3] Fix fast-booting device enumeration problems","Message-ID":"<20250820194416.GC10430@pendragon.ideasonboard.com>","References":"<20250820132316.1033443-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20250820132316.1033443-1-dan.scally@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35535,"web_url":"https://patchwork.libcamera.org/comment/35535/","msgid":"<511c38c5-5fbf-4f5f-aedb-f3dd5a8c706c@ideasonboard.com>","date":"2025-08-20T21:30:19","subject":"Re: [PATCH 0/3] Fix fast-booting device enumeration problems","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/people/156/","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"content":"Hi Laurent\n\nOn 20/08/2025 20:44, Laurent Pinchart wrote:\n> On Wed, Aug 20, 2025 at 02:23:13PM +0100, Daniel Scally wrote:\n>> Hello all\n>>\n>> At present, a CameraManager that is created before the device nodes for subdev\n>> entities in a media graph have been created (for example, because some subdev\n>> drivers have not yet probed and so the v4l2-async framework hasn't called for\n>> them to be built yet) will never match any pipeline handlers for that media\n>> device. This is because the MediaDevice populates itself with entities,\n>> interfaces, pads and links but ignores the fact that the interfaces for the\n>> subdev entities are missing. When DeviceEnumeratorUdev:: populateMediaDevice()\n>> is called, the entity's major and minor numbers are checked, seen to be zero (as\n>> they come from the missing interfaces) and then the entity is simply ignored\n>> rather than treated as dependencies that need to be watched for.\n> \n> I may be missing something, but aren't the entity and interface\n> registered together on the kernel side ? Even if the device node hasn't\n> been created by udev yet, the interface should be available, with a\n> major and minor, in the media graph.\n\nI don't think so...I think that the subdev's entity gets created by media_device_register_entity() \nfrom __v4l2_device_register_subdev() when that subdev is matched to a waiting notifier in \nv4l2_async_match_notify() but the interfaces not until media_devnode_create() from \nvideo_register_media_controller(), from __video_register_device() which is called for each subdevice \nin __v4l2_device_register_subdev_nodes() - which typically is delayed until a notifier's .complete() \ncallback.\n\nCertainly MediaDevice::findInterface() doesn't find them, so the other option would be that the \ntopology is missing the interface links that connect the entity and the interface but I think that \nthat happens immediately after the above anyway.\n> \n>> The MediaDevice\n>> is added through DeviceEnumerator::addDevice() to be checked against the\n>> pipeline handler's match() functions. When the device nodes for the subdevices\n>> are eventually detected as v4l2-async completes they are simply placed onto a\n>> list of orphaned nodes and ignored.\n>>\n>> To remedy the problem this set checks for missing interfaces during\n>> MediaDevice::populate() and if it sees some are missing, allows the MediaDevice\n>> to be created successfully but leaves the isValid() flag false.\n>> DeviceEnumeratorUdev is updated to check isValid() and when it is false, defers\n>> the creation of MediaEntity / MediaPad / MediaLink objects. Each time\n>> udevNotify() is triggered by a new device MediaDevice::populate() is checked\n>> again for each deferred MediaDevice, and once the missing interfaces are found\n>> in the topology the other objects are created, dependencies resolved and the\n>> MediaDevice added as normal.\n>>\n>> There may be a better way of receiving notice from the kernel that the media\n>> graph is fully complete and ready to be parsed; at the moment we rely on the\n>> reported topology version number becoming stable across two iterations of a loop\n>> that checks MEDIA_IOC_G_TOPOLOGY, but that's not sufficient to avoid this issue.\n>>\n>> Thanks\n>> Dan\n>>\n>> Daniel Scally (3):\n>> libcamera: media_device: Expand return values for populateEntities()\n>> libcamera: device_enumerator_udev: Add initMediaDevice()\n>> libcamera: device_enumerator_udev: Defer invalid media devices\n>>\n>> .../internal/device_enumerator_udev.h | 3 +\n>> include/libcamera/internal/media_device.h | 2 +-\n>> src/libcamera/device_enumerator.cpp | 2 +-\n>> src/libcamera/device_enumerator_udev.cpp | 85 ++++++++++++++-----\n>> src/libcamera/media_device.cpp | 27 +++---\n>> 5 files changed, 85 insertions(+), 34 deletions(-)\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8E51FBEFBE\n\tfor ;\n\tWed, 20 Aug 2025 21:30:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 43DC2692DE;\n\tWed, 20 Aug 2025 23:30:24 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9FB8C6142C\n\tfor ;\n\tWed, 20 Aug 2025 23:30:22 +0200 (CEST)","from [192.168.0.43]\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C30876A6;\n\tWed, 20 Aug 2025 23:29:23 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"quJKlvgC\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1755725363;\n\tbh=6lWm874oqjO2yb67U/KEbWkdASUuOBHxJjHstP0t3jQ=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=quJKlvgCX7yTJcwlTRP0A5tpV1vFnnmSi9RLTTEnN1/+0hy6R02Cs/tAhiB1ION2N\n\tDRXLkrmUcmLp9NnN3qseZkREnUanu7iuHbC9O1qwquNVRbzxBRWQH4dRBurau1U3IK\n\tYBTKqnUwA/JKoNIVLk5eQdh9LCxTFQYluBlaQvhg=","Message-ID":"<511c38c5-5fbf-4f5f-aedb-f3dd5a8c706c@ideasonboard.com>","Date":"Wed, 20 Aug 2025 22:30:19 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH 0/3] Fix fast-booting device enumeration problems","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20250820132316.1033443-1-dan.scally@ideasonboard.com>\n\t<20250820194416.GC10430@pendragon.ideasonboard.com>","Content-Language":"en-US","From":"Dan Scally ","In-Reply-To":"<20250820194416.GC10430@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35537,"web_url":"https://patchwork.libcamera.org/comment/35537/","msgid":"<20250821102727.GA25098@pendragon.ideasonboard.com>","date":"2025-08-21T10:27:27","subject":"Re: [PATCH 0/3] Fix fast-booting device enumeration problems","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Wed, Aug 20, 2025 at 10:30:19PM +0100, Daniel Scally wrote:\n> On 20/08/2025 20:44, Laurent Pinchart wrote:\n> > On Wed, Aug 20, 2025 at 02:23:13PM +0100, Daniel Scally wrote:\n> >> Hello all\n> >>\n> >> At present, a CameraManager that is created before the device nodes for subdev\n> >> entities in a media graph have been created (for example, because some subdev\n> >> drivers have not yet probed and so the v4l2-async framework hasn't called for\n> >> them to be built yet) will never match any pipeline handlers for that media\n> >> device. This is because the MediaDevice populates itself with entities,\n> >> interfaces, pads and links but ignores the fact that the interfaces for the\n> >> subdev entities are missing. When DeviceEnumeratorUdev:: populateMediaDevice()\n> >> is called, the entity's major and minor numbers are checked, seen to be zero (as\n> >> they come from the missing interfaces) and then the entity is simply ignored\n> >> rather than treated as dependencies that need to be watched for.\n> > \n> > I may be missing something, but aren't the entity and interface\n> > registered together on the kernel side ? Even if the device node hasn't\n> > been created by udev yet, the interface should be available, with a\n> > major and minor, in the media graph.\n> \n> I don't think so...I think that the subdev's entity gets created by\n> media_device_register_entity() from __v4l2_device_register_subdev()\n> when that subdev is matched to a waiting notifier in\n> v4l2_async_match_notify() but the interfaces not until\n> media_devnode_create() from video_register_media_controller(), from\n> __video_register_device() which is called for each subdevice in\n> __v4l2_device_register_subdev_nodes() - which typically is delayed\n> until a notifier's .complete() callback.\n\nYou are right, thanks for the explanation.\n\n> Certainly MediaDevice::findInterface() doesn't find them, so the other\n> option would be that the topology is missing the interface links that\n> connect the entity and the interface but I think that that happens\n> immediately after the above anyway.\n\nLet's discuss this in the context of patch 1/3 to avoid splitting the\ndiscussion.\n\n> >> The MediaDevice\n> >> is added through DeviceEnumerator::addDevice() to be checked against the\n> >> pipeline handler's match() functions. When the device nodes for the subdevices\n> >> are eventually detected as v4l2-async completes they are simply placed onto a\n> >> list of orphaned nodes and ignored.\n> >>\n> >> To remedy the problem this set checks for missing interfaces during\n> >> MediaDevice::populate() and if it sees some are missing, allows the MediaDevice\n> >> to be created successfully but leaves the isValid() flag false.\n> >> DeviceEnumeratorUdev is updated to check isValid() and when it is false, defers\n> >> the creation of MediaEntity / MediaPad / MediaLink objects. Each time\n> >> udevNotify() is triggered by a new device MediaDevice::populate() is checked\n> >> again for each deferred MediaDevice, and once the missing interfaces are found\n> >> in the topology the other objects are created, dependencies resolved and the\n> >> MediaDevice added as normal.\n> >>\n> >> There may be a better way of receiving notice from the kernel that the media\n> >> graph is fully complete and ready to be parsed; at the moment we rely on the\n> >> reported topology version number becoming stable across two iterations of a loop\n> >> that checks MEDIA_IOC_G_TOPOLOGY, but that's not sufficient to avoid this issue.\n> >>\n> >> Thanks\n> >> Dan\n> >>\n> >> Daniel Scally (3):\n> >> libcamera: media_device: Expand return values for populateEntities()\n> >> libcamera: device_enumerator_udev: Add initMediaDevice()\n> >> libcamera: device_enumerator_udev: Defer invalid media devices\n> >>\n> >> .../internal/device_enumerator_udev.h | 3 +\n> >> include/libcamera/internal/media_device.h | 2 +-\n> >> src/libcamera/device_enumerator.cpp | 2 +-\n> >> src/libcamera/device_enumerator_udev.cpp | 85 ++++++++++++++-----\n> >> src/libcamera/media_device.cpp | 27 +++---\n> >> 5 files changed, 85 insertions(+), 34 deletions(-)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 45B50BEFBE\n\tfor ;\n\tThu, 21 Aug 2025 10:27:55 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 06EF9692E0;\n\tThu, 21 Aug 2025 12:27:54 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5B757613DC\n\tfor ;\n\tThu, 21 Aug 2025 12:27:51 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A2F9AC78;\n\tThu, 21 Aug 2025 12:26:51 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"CFBgy2rN\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1755772011;\n\tbh=nwKDSOLjBh7gzVBcfOuZcC2yqV7kni4wJHqHZ8VZkFA=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=CFBgy2rNYFB0OATmmhuMkQ+WSm1y05xKGhWh4FT5G1MYZzFcMsKu48YLtAUp1lizX\n\tOSPhJrtiqrk/dIrjB/F2dzP2NHRnucbn8q4y2qSGGQVNkaEP63c/AZs37qU67A+KJU\n\tRjzjRHGzzH01/fXSCNr3vgI6N3qRFBRrdCPArIJI=","Date":"Thu, 21 Aug 2025 13:27:27 +0300","From":"Laurent Pinchart ","To":"Dan Scally ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH 0/3] Fix fast-booting device enumeration problems","Message-ID":"<20250821102727.GA25098@pendragon.ideasonboard.com>","References":"<20250820132316.1033443-1-dan.scally@ideasonboard.com>\n\t<20250820194416.GC10430@pendragon.ideasonboard.com>\n\t<511c38c5-5fbf-4f5f-aedb-f3dd5a8c706c@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<511c38c5-5fbf-4f5f-aedb-f3dd5a8c706c@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]