[{"id":11458,"web_url":"https://patchwork.libcamera.org/comment/11458/","msgid":"<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>","date":"2020-07-21T11:41:16","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn 21/07/2020 12:26, Jacopo Mondi wrote:\n> The CameraHalManager initialization tries to start the\n> libcamera::CameraManager which enumerate the registered cameras.\n> \n> When initialization is called too early during system boot, it might\n> happen that the media graphs are still being registered to user-space\n> preventing any pipeline handler to match and register cameras.\n> \n> If that happens, the CameraHalManager silently accepts that no\n> cameras are available in the system, reporting that information\n> to the camera stack:\n> \n> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> \n\nHrm ... should this be part of the hotplug work that as cameras become\navailable this gets updated somehow? - Even on *non* hotpluggable pipelines?\n\n\n> Fix this by returning an error code if no camera is registered in the\n> system at the time CameraHalManager::init() is called. The camera\n> framework then tries to re-load the HAL module later in time, hopefully\n> after the media device dependencies have been registered:\n> \n> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> ....\n> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> \n> Return -ENODEV as according to camera_common.h specification is the\n> only supported error code. The CrOS HAL adapter does not distinguish\n> between that and other negative values, so it does not really make\n> a difference.\n\nThis feels a bit punchy/racey still. And what happens in the instance\nthat we really don't yet have any cameras? (I.e. a UVC only platform,\nwhich doesn't yet have any cameras connected).\n\nHow many times will we defer? How long does it defer for for instance?\n\n> \n> Signed-off-by: Jacopo Mondi \n> ---\n> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> 1 file changed, 11 insertions(+)\n> \n> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> index 02b6418fb36d..e967d210e547 100644\n> --- a/src/android/camera_hal_manager.cpp\n> +++ b/src/android/camera_hal_manager.cpp\n> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> \t\t++index;\n> \t}\n> \n> +\t/*\n> +\t * If no pipeline has registered cameras, defer initialization to give\n> +\t * time to media devices to register to user-space.\n> +\t */\n> +\tif (index == 0) {\n> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n\nI think perhaps this needs a higher level than Debug to make sure it\nalways prints somewhere.\n\nI fear this will be deflecting the issue, and introduces races (i.e.\nthis would go through if only one camera of two in the system is available).\n\n..\n\nPerhaps it can still go in if it solves an immediate problem, but in\nthat case I think it needs a \\todo or a warning of some kind...\n\n\n> +\t\tdelete cameraManager_;\n> +\t\tcameraManager_ = nullptr;\n> +\t\treturn -ENODEV;\n> +\t}\n> +\n> \treturn 0;\n> }\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6BB02C0109\n\tfor ;\n\tTue, 21 Jul 2020 11:41:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 06EE560832;\n\tTue, 21 Jul 2020 13:41:21 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A1E7E60496\n\tfor ;\n\tTue, 21 Jul 2020 13:41:19 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F83A51A;\n\tTue, 21 Jul 2020 13:41:19 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"sPfIMQTN\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595331679;\n\tbh=DJYgeWBfo/nYKQoxLqQ9FlRpMwDevZS8u7lZtUYenzI=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=sPfIMQTNK/SHvQE5Tbs82T5YHFdeXR47IpDtkIFkUKM4lv7sGB8HpapotQkXWNq70\n\tgLMhEbelAsPifLPXGb8tNnXqDcpo57ree+ixe15QoCIfmLzaGZHitDV8jvG/aBOiaH\n\tdklvTdFdxN5ZOXHLZ+ZyW6i1+VRf4C6OQy0GuLLk=","To":"Jacopo Mondi , libcamera-devel@lists.libcamera.org","References":"<20200721112633.103016-1-jacopo@jmondi.org>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>","Date":"Tue, 21 Jul 2020 12:41:16 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200721112633.103016-1-jacopo@jmondi.org>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11461,"web_url":"https://patchwork.libcamera.org/comment/11461/","msgid":"<20200721121350.llcc7b2phigc647e@uno.localdomain>","date":"2020-07-21T12:13:50","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Kieran,\n\nOn Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> Hi Jacopo,\n>\n> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> > The CameraHalManager initialization tries to start the\n> > libcamera::CameraManager which enumerate the registered cameras.\n> >\n> > When initialization is called too early during system boot, it might\n> > happen that the media graphs are still being registered to user-space\n> > preventing any pipeline handler to match and register cameras.\n> >\n> > If that happens, the CameraHalManager silently accepts that no\n> > cameras are available in the system, reporting that information\n> > to the camera stack:\n> >\n> > cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> > cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> > CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >\n>\n> Hrm ... should this be part of the hotplug work that as cameras become\n> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n>\n\nI don't get what your comment is about here, I'm sorry\n\n>\n> > Fix this by returning an error code if no camera is registered in the\n> > system at the time CameraHalManager::init() is called. The camera\n> > framework then tries to re-load the HAL module later in time, hopefully\n> > after the media device dependencies have been registered:\n> >\n> > 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> > 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> > ....\n> > 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> > 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >\n> > Return -ENODEV as according to camera_common.h specification is the\n> > only supported error code. The CrOS HAL adapter does not distinguish\n> > between that and other negative values, so it does not really make\n> > a difference.\n>\n> This feels a bit punchy/racey still. And what happens in the instance\n> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> which doesn't yet have any cameras connected).\n\nYou keep defferring ? I think the framework handles that\n\n>\n> How many times will we defer? How long does it defer for for instance?\n>\n\nUsually 1 is enough from my testing. I don't think we have to care\nwhat is the timeout in the framework, as afaict android services are\nrestarted by default.\n\n> >\n> > Signed-off-by: Jacopo Mondi \n> > ---\n> > src/android/camera_hal_manager.cpp | 11 +++++++++++\n> > 1 file changed, 11 insertions(+)\n> >\n> > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > index 02b6418fb36d..e967d210e547 100644\n> > --- a/src/android/camera_hal_manager.cpp\n> > +++ b/src/android/camera_hal_manager.cpp\n> > @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> > \t\t++index;\n> > \t}\n> >\n> > +\t/*\n> > +\t * If no pipeline has registered cameras, defer initialization to give\n> > +\t * time to media devices to register to user-space.\n> > +\t */\n> > +\tif (index == 0) {\n> > +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n>\n> I think perhaps this needs a higher level than Debug to make sure it\n> always prints somewhere.\n\nI think that's kind of expected to happen, I don't think this should\nbe an error\n\n>\n> I fear this will be deflecting the issue, and introduces races (i.e.\n> this would go through if only one camera of two in the system is available).\n\nIf the pipeline handler match (ie the device enumerator matches the\ndependencies) than the match() function continues and register\ncameras. I don't think we can found ourselves in a semi-initialized\nstate.\n\n>\n> ..\n>\n> Perhaps it can still go in if it solves an immediate problem, but in\n> that case I think it needs a \\todo or a warning of some kind...\n>\n\nA \\todo item to record this should be fixed how ?\n\nUnless we expect the CameraManager to stall until any of the pipeline\nhandler it tries match, but this seems not a good idea to me.\n\n>\n> > +\t\tdelete cameraManager_;\n> > +\t\tcameraManager_ = nullptr;\n> > +\t\treturn -ENODEV;\n> > +\t}\n> > +\n> > \treturn 0;\n> > }\n> >\n> >\n>\n> --\n> Regards\n> --\n> Kieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 62F66BD792\n\tfor ;\n\tTue, 21 Jul 2020 12:10:15 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D130860832;\n\tTue, 21 Jul 2020 14:10:14 +0200 (CEST)","from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net\n\t[217.70.183.194])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C2F960496\n\tfor ;\n\tTue, 21 Jul 2020 14:10:13 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay2-d.mail.gandi.net (Postfix) with ESMTPSA id A7DC64000B;\n\tTue, 21 Jul 2020 12:10:12 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Tue, 21 Jul 2020 14:13:50 +0200","From":"Jacopo Mondi ","To":"Kieran Bingham ","Message-ID":"<20200721121350.llcc7b2phigc647e@uno.localdomain>","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11465,"web_url":"https://patchwork.libcamera.org/comment/11465/","msgid":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>","date":"2020-07-21T12:47:29","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn 21/07/2020 13:13, Jacopo Mondi wrote:\n> Hi Kieran,\n> \n> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n>> Hi Jacopo,\n>>\n>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n>>> The CameraHalManager initialization tries to start the\n>>> libcamera::CameraManager which enumerate the registered cameras.\n>>>\n>>> When initialization is called too early during system boot, it might\n>>> happen that the media graphs are still being registered to user-space\n>>> preventing any pipeline handler to match and register cameras.\n>>>\n>>> If that happens, the CameraHalManager silently accepts that no\n>>> cameras are available in the system, reporting that information\n>>> to the camera stack:\n>>>\n>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n>>>\n>>\n>> Hrm ... should this be part of the hotplug work that as cameras become\n>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n>>\n> \n> I don't get what your comment is about here, I'm sorry\n\n\nYour patch prevents loading of libcamera completely unless there is at\nleast one camera right ?\n\nWhat happens on say an x86 platform which will only have UVC cameras,\nyet will use libcamera...\n\nWhen will it be acceptable to load the camera service.\n\nShould the service itself continually fail, and respawn until someone\nplugs in a camera?\n\n\nSo taking that further, what I'm saying is, perhaps this should be\nperfectly able to launch with zero cameras, and /when/ cameras are\ncreated they should notify android that a new camera is now available.\n\n\n\n\n>>\n>>> Fix this by returning an error code if no camera is registered in the\n>>> system at the time CameraHalManager::init() is called. The camera\n>>> framework then tries to re-load the HAL module later in time, hopefully\n>>> after the media device dependencies have been registered:\n>>>\n>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>> ....\n>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n>>>\n>>> Return -ENODEV as according to camera_common.h specification is the\n>>> only supported error code. The CrOS HAL adapter does not distinguish\n>>> between that and other negative values, so it does not really make\n>>> a difference.\n>>\n>> This feels a bit punchy/racey still. And what happens in the instance\n>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n>> which doesn't yet have any cameras connected).\n> \n> You keep defferring ? I think the framework handles that\n\nI would be surprised if it would be expected behaviour to just poll\nlaunching of the camera service until there is a camera there to satisfy\nit ...\n\n\n>> How many times will we defer? How long does it defer for for instance?\n>>\n> \n> Usually 1 is enough from my testing. I don't think we have to care\n> what is the timeout in the framework, as afaict android services are\n> restarted by default.\n\n\nOk, so this is mostly dealing with the race at startup ...\n\n\n\n>>> Signed-off-by: Jacopo Mondi \n>>> ---\n>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n>>> 1 file changed, 11 insertions(+)\n>>>\n>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n>>> index 02b6418fb36d..e967d210e547 100644\n>>> --- a/src/android/camera_hal_manager.cpp\n>>> +++ b/src/android/camera_hal_manager.cpp\n>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n>>> \t\t++index;\n>>> \t}\n>>>\n>>> +\t/*\n>>> +\t * If no pipeline has registered cameras, defer initialization to give\n>>> +\t * time to media devices to register to user-space.\n>>> +\t */\n>>> +\tif (index == 0) {\n>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n>>\n>> I think perhaps this needs a higher level than Debug to make sure it\n>> always prints somewhere.\n> \n> I think that's kind of expected to happen, I don't think this should\n> be an error\n\n\nInfo? Warning?\n\nIf libcamera has prevented loading, I think it would be good to know\nabout it by default...\n\n\n>> I fear this will be deflecting the issue, and introduces races (i.e.\n>> this would go through if only one camera of two in the system is available).\n> \n> If the pipeline handler match (ie the device enumerator matches the\n> dependencies) than the match() function continues and register\n> cameras. I don't think we can found ourselves in a semi-initialized\n> state.\n\n\nOk, so it really is about having at least one camera available?\n\nI think looking at CameraHalManager::init(), as it only creates a\nCameraDevice for cameras available in cameraManager_->cameras(), it\n/could/ be a race, it's just that creating two cameras is fast, so we're\nunlikely to get in here between the point that one has been available\nand another is still loading ...\n\nBut still ... I think this is stuff that will get dealt with by hotplug\nbeing added to the HAL.\n\n\n\n\n>> ..\n>>\n>> Perhaps it can still go in if it solves an immediate problem, but in\n>> that case I think it needs a \\todo or a warning of some kind...\n>>\n> \n> A \\todo item to record this should be fixed how ?\n> \n> Unless we expect the CameraManager to stall until any of the pipeline\n> handler it tries match, but this seems not a good idea to me.\n\nNo, i don't expect it to stall ... I expect it to complete successfully,\nand if there are only 0 cameras, it will only have zero cameras - until\nthey become available at which point they'll be registered through the\nsame mechanisms as the hotplugging ...\n\n\n\n> \n>>\n>>> +\t\tdelete cameraManager_;\n>>> +\t\tcameraManager_ = nullptr;\n>>> +\t\treturn -ENODEV;\n>>> +\t}\n>>> +\n>>> \treturn 0;\n>>> }\n>>>\n>>>\n>>\n>> --\n>> Regards\n>> --\n>> Kieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 18112C0109\n\tfor ;\n\tTue, 21 Jul 2020 12:47:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A064460868;\n\tTue, 21 Jul 2020 14:47:34 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 875BC6053C\n\tfor ;\n\tTue, 21 Jul 2020 14:47:32 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id DBED751A;\n\tTue, 21 Jul 2020 14:47:31 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"r8odZ58y\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595335652;\n\tbh=f4xf6TgSyfciGGgeuzK60zfMkaybMZOVGXb7pgxfq0I=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=r8odZ58y0xXZYDUACJ9cV4Gs/vwY4PizsCD8iK8vaxeEoxNObPiB/0YFBt6TF74EO\n\t7DiqCIJQ2MnfWM+B9I6oI/CGKxzJNkKVLH1z87aD4jpzWutzg5ifoGtlhoIU4Zy1tp\n\t+MCD89ReGqeT2khyU9MooYVhpfqMxhw+KK0//lQ0=","To":"Jacopo Mondi ","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>","Date":"Tue, 21 Jul 2020 13:47:29 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200721121350.llcc7b2phigc647e@uno.localdomain>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11466,"web_url":"https://patchwork.libcamera.org/comment/11466/","msgid":"<20200721130958.yvot2wamrpnbgtge@uno.localdomain>","date":"2020-07-21T13:09:58","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> Hi Jacopo,\n>\n> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> > Hi Kieran,\n> >\n> > On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >> Hi Jacopo,\n> >>\n> >> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>> The CameraHalManager initialization tries to start the\n> >>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>\n> >>> When initialization is called too early during system boot, it might\n> >>> happen that the media graphs are still being registered to user-space\n> >>> preventing any pipeline handler to match and register cameras.\n> >>>\n> >>> If that happens, the CameraHalManager silently accepts that no\n> >>> cameras are available in the system, reporting that information\n> >>> to the camera stack:\n> >>>\n> >>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>\n> >>\n> >> Hrm ... should this be part of the hotplug work that as cameras become\n> >> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>\n> >\n> > I don't get what your comment is about here, I'm sorry\n>\n>\n> Your patch prevents loading of libcamera completely unless there is at\n> least one camera right ?\n>\n> What happens on say an x86 platform which will only have UVC cameras,\n> yet will use libcamera...\n>\n> When will it be acceptable to load the camera service.\n>\n> Should the service itself continually fail, and respawn until someone\n> plugs in a camera?\n>\n\nThat's my understanding\n\n>\n> So taking that further, what I'm saying is, perhaps this should be\n> perfectly able to launch with zero cameras, and /when/ cameras are\n> created they should notify android that a new camera is now available.\n\nDo you know if that's even possible ?\n\nI think Android expects the number of possible cameras to be\nstatically defined, there's a callback to notify a change of status of\na device, but we're dealing here with -creating- cameras based on the\nnumber of cameras detected by android.\n\n>\n>\n>\n>\n> >>\n> >>> Fix this by returning an error code if no camera is registered in the\n> >>> system at the time CameraHalManager::init() is called. The camera\n> >>> framework then tries to re-load the HAL module later in time, hopefully\n> >>> after the media device dependencies have been registered:\n> >>>\n> >>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>> ....\n> >>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>\n> >>> Return -ENODEV as according to camera_common.h specification is the\n> >>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>> between that and other negative values, so it does not really make\n> >>> a difference.\n> >>\n> >> This feels a bit punchy/racey still. And what happens in the instance\n> >> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >> which doesn't yet have any cameras connected).\n> >\n> > You keep defferring ? I think the framework handles that\n>\n> I would be surprised if it would be expected behaviour to just poll\n> launching of the camera service until there is a camera there to satisfy\n> it ...\n>\n>\n> >> How many times will we defer? How long does it defer for for instance?\n> >>\n> >\n> > Usually 1 is enough from my testing. I don't think we have to care\n> > what is the timeout in the framework, as afaict android services are\n> > restarted by default.\n>\n>\n> Ok, so this is mostly dealing with the race at startup ...\n>\n>\n>\n> >>> Signed-off-by: Jacopo Mondi \n> >>> ---\n> >>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>> 1 file changed, 11 insertions(+)\n> >>>\n> >>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>> index 02b6418fb36d..e967d210e547 100644\n> >>> --- a/src/android/camera_hal_manager.cpp\n> >>> +++ b/src/android/camera_hal_manager.cpp\n> >>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>> \t\t++index;\n> >>> \t}\n> >>>\n> >>> +\t/*\n> >>> +\t * If no pipeline has registered cameras, defer initialization to give\n> >>> +\t * time to media devices to register to user-space.\n> >>> +\t */\n> >>> +\tif (index == 0) {\n> >>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>\n> >> I think perhaps this needs a higher level than Debug to make sure it\n> >> always prints somewhere.\n> >\n> > I think that's kind of expected to happen, I don't think this should\n> > be an error\n>\n>\n> Info? Warning?\n>\n> If libcamera has prevented loading, I think it would be good to know\n> about it by default...\n>\n>\n> >> I fear this will be deflecting the issue, and introduces races (i.e.\n> >> this would go through if only one camera of two in the system is available).\n> >\n> > If the pipeline handler match (ie the device enumerator matches the\n> > dependencies) than the match() function continues and register\n> > cameras. I don't think we can found ourselves in a semi-initialized\n> > state.\n>\n>\n> Ok, so it really is about having at least one camera available?\n\nIt's about at least a pipeline being matched.\n\n>\n> I think looking at CameraHalManager::init(), as it only creates a\n> CameraDevice for cameras available in cameraManager_->cameras(), it\n> /could/ be a race, it's just that creating two cameras is fast, so we're\n> unlikely to get in here between the point that one has been available\n> and another is still loading ...\n\nI don't think that's possible.\n\nIf a all the dependencies of a pipeline handler are matched, the\ncamera are registered. What is the code flow that registers one\ncamera, returns to the camera manager, then calls the pipeline handler\nagain to register more ?\n\n>\n> But still ... I think this is stuff that will get dealt with by hotplug\n> being added to the HAL.\n\nI agree, we should support inserting and removing new cameras, but I\nthink at this point, we should really start only if cameras are\nregistered and any pipeline handler is matched.\n\nI'm not sure I got what other alternative approach you are proposing\nto be honest.\n\n>\n>\n>\n>\n> >> ..\n> >>\n> >> Perhaps it can still go in if it solves an immediate problem, but in\n> >> that case I think it needs a \\todo or a warning of some kind...\n> >>\n> >\n> > A \\todo item to record this should be fixed how ?\n> >\n> > Unless we expect the CameraManager to stall until any of the pipeline\n> > handler it tries match, but this seems not a good idea to me.\n>\n> No, i don't expect it to stall ... I expect it to complete successfully,\n> and if there are only 0 cameras, it will only have zero cameras - until\n\nThat means Android/CrOS starts without the camera being active (ie.\nthe camera application icon is not available, in CrOS in example).\n\n> they become available at which point they'll be registered through the\n> same mechanisms as the hotplugging ...\n\nStill I don't get if you are talking about libcamera hotplug or\nandroid hotplug, which, to my understanding, expects anyway a camera\ndevice to be registered but marked as 'not present' (looking at the\ncamera_device_status enumeration)\n\n>\n>\n>\n> >\n> >>\n> >>> +\t\tdelete cameraManager_;\n> >>> +\t\tcameraManager_ = nullptr;\n> >>> +\t\treturn -ENODEV;\n> >>> +\t}\n> >>> +\n> >>> \treturn 0;\n> >>> }\n> >>>\n> >>>\n> >>\n> >> --\n> >> Regards\n> >> --\n> >> Kieran\n>\n> --\n> Regards\n> --\n> Kieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B678AC2E68\n\tfor ;\n\tTue, 21 Jul 2020 13:06:33 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 563AA60868;\n\tTue, 21 Jul 2020 15:06:33 +0200 (CEST)","from relay11.mail.gandi.net (relay11.mail.gandi.net\n\t[217.70.178.231])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C80636053C\n\tfor ;\n\tTue, 21 Jul 2020 15:06:32 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay11.mail.gandi.net (Postfix) with ESMTPSA id DDAC6100006;\n\tTue, 21 Jul 2020 13:06:26 +0000 (UTC)"],"Date":"Tue, 21 Jul 2020 15:09:58 +0200","From":"Jacopo Mondi ","To":"Kieran Bingham ","Message-ID":"<20200721130958.yvot2wamrpnbgtge@uno.localdomain>","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11471,"web_url":"https://patchwork.libcamera.org/comment/11471/","msgid":"","date":"2020-07-21T20:59:10","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 21/07/2020 14:09, Jacopo Mondi wrote:\n> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n>> Hi Jacopo,\n>>\n>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n>>> Hi Kieran,\n>>>\n>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n>>>> Hi Jacopo,\n>>>>\n>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n>>>>> The CameraHalManager initialization tries to start the\n>>>>> libcamera::CameraManager which enumerate the registered cameras.\n>>>>>\n>>>>> When initialization is called too early during system boot, it might\n>>>>> happen that the media graphs are still being registered to user-space\n>>>>> preventing any pipeline handler to match and register cameras.\n>>>>>\n>>>>> If that happens, the CameraHalManager silently accepts that no\n>>>>> cameras are available in the system, reporting that information\n>>>>> to the camera stack:\n>>>>>\n>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n>>>>>\n>>>>\n>>>> Hrm ... should this be part of the hotplug work that as cameras become\n>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n>>>>\n>>>\n>>> I don't get what your comment is about here, I'm sorry\n>>\n>>\n>> Your patch prevents loading of libcamera completely unless there is at\n>> least one camera right ?\n>>\n>> What happens on say an x86 platform which will only have UVC cameras,\n>> yet will use libcamera...\n>>\n>> When will it be acceptable to load the camera service.\n>>\n>> Should the service itself continually fail, and respawn until someone\n>> plugs in a camera?\n>>\n> \n> That's my understanding\n\nThat sounds like that would cause a very bad implementation to me.\n\nWaiting for upstart to decide when to retry launching the camera-service\nwouldn't be an acceptable time to wait IMO.\n\nEither it tries every second, and is flooding the system, or it tries\nevery minute and it takes a minute before you can view your newly\nattached UVC camera ...\n\nI would expect event driven camera attachment, not polled ...\n\n>> So taking that further, what I'm saying is, perhaps this should be\n>> perfectly able to launch with zero cameras, and /when/ cameras are\n>> created they should notify android that a new camera is now available.\n> \n> Do you know if that's even possible ?\n> \n> I think Android expects the number of possible cameras to be\n> statically defined, there's a callback to notify a change of status of\n> a device, but we're dealing here with -creating- cameras based on the\n> number of cameras detected by android.\n\nSo, looking at the USB HAL in cros - it does look like they pre-allocate\nsome cameras or such and only activate them on device connection.\n\nI fear we might have to do the same somehow in the future, as indeed it\nmight not be possible to tell android there is a 'new' camera. Only an\nupdate to an existing one... not sure it will require more investigation.\n\n\n\n>>>>> Fix this by returning an error code if no camera is registered in the\n>>>>> system at the time CameraHalManager::init() is called. The camera\n>>>>> framework then tries to re-load the HAL module later in time, hopefully\n>>>>> after the media device dependencies have been registered:\n>>>>>\n>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>> ....\n>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n>>>>>\n>>>>> Return -ENODEV as according to camera_common.h specification is the\n>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n>>>>> between that and other negative values, so it does not really make\n>>>>> a difference.\n>>>>\n>>>> This feels a bit punchy/racey still. And what happens in the instance\n>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n>>>> which doesn't yet have any cameras connected).\n>>>\n>>> You keep defferring ? I think the framework handles that\n>>\n>> I would be surprised if it would be expected behaviour to just poll\n>> launching of the camera service until there is a camera there to satisfy\n>> it ...\n>>\n>>\n>>>> How many times will we defer? How long does it defer for for instance?\n>>>>\n>>>\n>>> Usually 1 is enough from my testing. I don't think we have to care\n>>> what is the timeout in the framework, as afaict android services are\n>>> restarted by default.\n>>\n>>\n>> Ok, so this is mostly dealing with the race at startup ...\n>>\n>>\n>>\n>>>>> Signed-off-by: Jacopo Mondi \n>>>>> ---\n>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n>>>>> 1 file changed, 11 insertions(+)\n>>>>>\n>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n>>>>> index 02b6418fb36d..e967d210e547 100644\n>>>>> --- a/src/android/camera_hal_manager.cpp\n>>>>> +++ b/src/android/camera_hal_manager.cpp\n>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n>>>>> \t\t++index;\n>>>>> \t}\n>>>>>\n>>>>> +\t/*\n>>>>> +\t * If no pipeline has registered cameras, defer initialization to give\n>>>>> +\t * time to media devices to register to user-space.\n>>>>> +\t */\n>>>>> +\tif (index == 0) {\n>>>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n>>>>\n>>>> I think perhaps this needs a higher level than Debug to make sure it\n>>>> always prints somewhere.\n>>>\n>>> I think that's kind of expected to happen, I don't think this should\n>>> be an error\n>>\n>>\n>> Info? Warning?\n>>\n>> If libcamera has prevented loading, I think it would be good to know\n>> about it by default...\n>>\n>>\n>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n>>>> this would go through if only one camera of two in the system is available).\n>>>\n>>> If the pipeline handler match (ie the device enumerator matches the\n>>> dependencies) than the match() function continues and register\n>>> cameras. I don't think we can found ourselves in a semi-initialized\n>>> state.\n>>\n>>\n>> Ok, so it really is about having at least one camera available?\n> \n> It's about at least a pipeline being matched.\n\nOk, but I don't think it's that difficult to conjur up a scenario where\nthere won't be a device\n\n\n> \n>>\n>> I think looking at CameraHalManager::init(), as it only creates a\n>> CameraDevice for cameras available in cameraManager_->cameras(), it\n>> /could/ be a race, it's just that creating two cameras is fast, so we're\n>> unlikely to get in here between the point that one has been available\n>> and another is still loading ...\n> \n> I don't think that's possible.\n> \n> If a all the dependencies of a pipeline handler are matched, the\n> camera are registered. What is the code flow that registers one\n> camera, returns to the camera manager, then calls the pipeline handler\n> again to register more ?\n\n\nAt least PipelineHandlerUVC will call PipelineHandler::match() once for\neach camera attached.\n\nBut yes, perhaps due to the threading model, it won't be a possible race.\n\nI was thinking of when a pipeline starts and only 'one' of it's media\ndevices is available.\n\n\n\n\n>> But still ... I think this is stuff that will get dealt with by hotplug\n>> being added to the HAL.\n> \n> I agree, we should support inserting and removing new cameras, but I\n> think at this point, we should really start only if cameras are\n> registered and any pipeline handler is matched.\n> \n> I'm not sure I got what other alternative approach you are proposing\n> to be honest.\n\nI'm suggesting that when hotplug support is available, this patch, if\nintegrated, might need to be reverted in effect ... and thus a todo note\nshould be added to say that or such.\n\nI believe we should be able to start the camera service successfully\nwith zero cameras if there are zero cameras at the time the service starts.\n\n\n>>>> ..\n>>>>\n>>>> Perhaps it can still go in if it solves an immediate problem, but in\n>>>> that case I think it needs a \\todo or a warning of some kind...\n>>>>\n>>>\n>>> A \\todo item to record this should be fixed how ?\n>>>\n>>> Unless we expect the CameraManager to stall until any of the pipeline\n>>> handler it tries match, but this seems not a good idea to me.\n>>\n>> No, i don't expect it to stall ... I expect it to complete successfully,\n>> and if there are only 0 cameras, it will only have zero cameras - until\n> \n> That means Android/CrOS starts without the camera being active (ie.\n> the camera application icon is not available, in CrOS in example).\n\n\nHrm, so it disables the app completely?\n\nAnyway, essentially I'm thinking - whatever happens with the existing\nUVC stack is what we would need to be able to mirror.\n\n\n\n>> they become available at which point they'll be registered through the\n>> same mechanisms as the hotplugging ...\n> \n> Still I don't get if you are talking about libcamera hotplug or\n> android hotplug, which, to my understanding, expects anyway a camera\n> device to be registered but marked as 'not present' (looking at the\n> camera_device_status enumeration)\n\n\nI was going to ask how does it register camera's it doesn't know will\nexist, but I think I saw that the UVC HAL just pre-allocates some\ncameras or such, so that explains how that one works.\n\n\n\n\n\n>>>>> +\t\tdelete cameraManager_;\n>>>>> +\t\tcameraManager_ = nullptr;\n>>>>> +\t\treturn -ENODEV;\n>>>>> +\t}\n>>>>> +\n>>>>> \treturn 0;\n>>>>> }","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AD116BDB1B\n\tfor ;\n\tTue, 21 Jul 2020 20:59:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 11CCA60939;\n\tTue, 21 Jul 2020 22:59:25 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 54C3160540\n\tfor ;\n\tTue, 21 Jul 2020 22:59:23 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AD67951A;\n\tTue, 21 Jul 2020 22:59:12 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"o6UkJNHG\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595365152;\n\tbh=9lq8yLPDN35Im3AqYrmWeP8wz9+XuO1n7LKKQUQAXDw=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=o6UkJNHGNTF0s7pYGj+DeA1H0fQ/p54A4IpJcAEaABwt2PZjQtxBVb2pxfYSXKejo\n\t3n+5fKj4PXidxYLlJ4tNKrFWE5AZweM2myv5uiMTNciWD9tQuKVa1t6EbK4tPG7qKB\n\tR3e5bM4VufjY0CrIPlGtIgn9axtgVDoAbgDw07N8=","To":"Jacopo Mondi ","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"","Date":"Tue, 21 Jul 2020 21:59:10 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200721130958.yvot2wamrpnbgtge@uno.localdomain>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11482,"web_url":"https://patchwork.libcamera.org/comment/11482/","msgid":"<20200722111221.sezcchaymsnbhwgf@uno.localdomain>","date":"2020-07-22T11:12:21","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Kieran,\n I think the conversation diverged, as I was clearly overlooking\nthe UVC camera use case and you're missing the intended use case of\nthis patch. I'll try to reply and summarize at the end.\n\nOn Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> > On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> >> Hi Jacopo,\n> >>\n> >> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> >>> Hi Kieran,\n> >>>\n> >>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >>>> Hi Jacopo,\n> >>>>\n> >>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>>>> The CameraHalManager initialization tries to start the\n> >>>>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>>>\n> >>>>> When initialization is called too early during system boot, it might\n> >>>>> happen that the media graphs are still being registered to user-space\n> >>>>> preventing any pipeline handler to match and register cameras.\n> >>>>>\n> >>>>> If that happens, the CameraHalManager silently accepts that no\n> >>>>> cameras are available in the system, reporting that information\n> >>>>> to the camera stack:\n> >>>>>\n> >>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>>>\n> >>>>\n> >>>> Hrm ... should this be part of the hotplug work that as cameras become\n> >>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>>>\n> >>>\n> >>> I don't get what your comment is about here, I'm sorry\n> >>\n> >>\n> >> Your patch prevents loading of libcamera completely unless there is at\n> >> least one camera right ?\n> >>\n> >> What happens on say an x86 platform which will only have UVC cameras,\n> >> yet will use libcamera...\n> >>\n> >> When will it be acceptable to load the camera service.\n> >>\n> >> Should the service itself continually fail, and respawn until someone\n> >> plugs in a camera?\n> >>\n> >\n> > That's my understanding\n>\n> That sounds like that would cause a very bad implementation to me.\n>\n> Waiting for upstart to decide when to retry launching the camera-service\n> wouldn't be an acceptable time to wait IMO.\n>\n> Either it tries every second, and is flooding the system, or it tries\n> every minute and it takes a minute before you can view your newly\n> attached UVC camera ...\n>\n> I would expect event driven camera attachment, not polled ...\n>\n\nFor UVC we will have to register non-active cameras and notify to\nandroid they're available when hotplug is detected\n\nFor non-pluggable cameras, as shall wait until the pipeline handler is\nmatched and has registered cameras\n\n> >> So taking that further, what I'm saying is, perhaps this should be\n> >> perfectly able to launch with zero cameras, and /when/ cameras are\n> >> created they should notify android that a new camera is now available.\n> >\n> > Do you know if that's even possible ?\n> >\n> > I think Android expects the number of possible cameras to be\n> > statically defined, there's a callback to notify a change of status of\n> > a device, but we're dealing here with -creating- cameras based on the\n> > number of cameras detected by android.\n>\n> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> some cameras or such and only activate them on device connection.\n>\n> I fear we might have to do the same somehow in the future, as indeed it\n> might not be possible to tell android there is a 'new' camera. Only an\n> update to an existing one... not sure it will require more investigation.\n>\n\nYes, but that's for UVC. Built-in cameras are different, and Android\nbets on the fact if you have a camera service running then your system\nhas cameras which are expected to be registered.\n\nKeep in mind so far there's been an HAL for UVC and one for built-in\ncameras. We're handling both and that's a new use case afaict\n\n>\n>\n> >>>>> Fix this by returning an error code if no camera is registered in the\n> >>>>> system at the time CameraHalManager::init() is called. The camera\n> >>>>> framework then tries to re-load the HAL module later in time, hopefully\n> >>>>> after the media device dependencies have been registered:\n> >>>>>\n> >>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>> ....\n> >>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>>>\n> >>>>> Return -ENODEV as according to camera_common.h specification is the\n> >>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>>>> between that and other negative values, so it does not really make\n> >>>>> a difference.\n> >>>>\n> >>>> This feels a bit punchy/racey still. And what happens in the instance\n> >>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >>>> which doesn't yet have any cameras connected).\n> >>>\n> >>> You keep defferring ? I think the framework handles that\n> >>\n> >> I would be surprised if it would be expected behaviour to just poll\n> >> launching of the camera service until there is a camera there to satisfy\n> >> it ...\n> >>\n> >>\n> >>>> How many times will we defer? How long does it defer for for instance?\n> >>>>\n> >>>\n> >>> Usually 1 is enough from my testing. I don't think we have to care\n> >>> what is the timeout in the framework, as afaict android services are\n> >>> restarted by default.\n> >>\n> >>\n> >> Ok, so this is mostly dealing with the race at startup ...\n> >>\n> >>\n> >>\n> >>>>> Signed-off-by: Jacopo Mondi \n> >>>>> ---\n> >>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>>>> 1 file changed, 11 insertions(+)\n> >>>>>\n> >>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>> index 02b6418fb36d..e967d210e547 100644\n> >>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>>>> \t\t++index;\n> >>>>> \t}\n> >>>>>\n> >>>>> +\t/*\n> >>>>> +\t * If no pipeline has registered cameras, defer initialization to give\n> >>>>> +\t * time to media devices to register to user-space.\n> >>>>> +\t */\n> >>>>> +\tif (index == 0) {\n> >>>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>>>\n> >>>> I think perhaps this needs a higher level than Debug to make sure it\n> >>>> always prints somewhere.\n> >>>\n> >>> I think that's kind of expected to happen, I don't think this should\n> >>> be an error\n> >>\n> >>\n> >> Info? Warning?\n> >>\n> >> If libcamera has prevented loading, I think it would be good to know\n> >> about it by default...\n> >>\n> >>\n> >>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> >>>> this would go through if only one camera of two in the system is available).\n> >>>\n> >>> If the pipeline handler match (ie the device enumerator matches the\n> >>> dependencies) than the match() function continues and register\n> >>> cameras. I don't think we can found ourselves in a semi-initialized\n> >>> state.\n> >>\n> >>\n> >> Ok, so it really is about having at least one camera available?\n> >\n> > It's about at least a pipeline being matched.\n>\n> Ok, but I don't think it's that difficult to conjur up a scenario where\n> there won't be a device\n\nNot for what Android has been thought to work on. You integrate a\nsystem with cameras, it's fair to defer until those cameras are\nregistered. You have no cameras, either you disable the camera stack\nor you register 0 cameras and then the camera subsystem is silent and\nnot accessible.\n\n>\n>\n> >\n> >>\n> >> I think looking at CameraHalManager::init(), as it only creates a\n> >> CameraDevice for cameras available in cameraManager_->cameras(), it\n> >> /could/ be a race, it's just that creating two cameras is fast, so we're\n> >> unlikely to get in here between the point that one has been available\n> >> and another is still loading ...\n> >\n> > I don't think that's possible.\n> >\n> > If a all the dependencies of a pipeline handler are matched, the\n> > camera are registered. What is the code flow that registers one\n> > camera, returns to the camera manager, then calls the pipeline handler\n> > again to register more ?\n>\n>\n> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> each camera attached.\n>\n> But yes, perhaps due to the threading model, it won't be a possible race.\n>\n> I was thinking of when a pipeline starts and only 'one' of it's media\n> devices is available.\n\nYes, for UVC that's different I agree. A call to\nCameraManager::start() might match one uvc media device but miss a\nsecond one which still have to appear to userspace.\n\nAs said, UVC needs some special handling and pre-register cameras\n\n>\n>\n>\n>\n> >> But still ... I think this is stuff that will get dealt with by hotplug\n> >> being added to the HAL.\n> >\n> > I agree, we should support inserting and removing new cameras, but I\n> > think at this point, we should really start only if cameras are\n> > registered and any pipeline handler is matched.\n> >\n> > I'm not sure I got what other alternative approach you are proposing\n> > to be honest.\n>\n> I'm suggesting that when hotplug support is available, this patch, if\n> integrated, might need to be reverted in effect ... and thus a todo note\n> should be added to say that or such.\n>\n> I believe we should be able to start the camera service successfully\n> with zero cameras if there are zero cameras at the time the service starts.\n\nI don't think so for built-in cameras. We start the service when\ncameras are registered, otherwise if you have to pre-allocate cameras\nyou would need to know how many of them the pipeline handler expects\nto register and that's not known before it has been matched.\n\nUVC, again, is a different story as it supports hotplug.\n\n>\n>\n> >>>> ..\n> >>>>\n> >>>> Perhaps it can still go in if it solves an immediate problem, but in\n> >>>> that case I think it needs a \\todo or a warning of some kind...\n> >>>>\n> >>>\n> >>> A \\todo item to record this should be fixed how ?\n> >>>\n> >>> Unless we expect the CameraManager to stall until any of the pipeline\n> >>> handler it tries match, but this seems not a good idea to me.\n> >>\n> >> No, i don't expect it to stall ... I expect it to complete successfully,\n> >> and if there are only 0 cameras, it will only have zero cameras - until\n> >\n> > That means Android/CrOS starts without the camera being active (ie.\n> > the camera application icon is not available, in CrOS in example).\n>\n>\n> Hrm, so it disables the app completely?\n>\n> Anyway, essentially I'm thinking - whatever happens with the existing\n> UVC stack is what we would need to be able to mirror.\n>\n>\n>\n> >> they become available at which point they'll be registered through the\n> >> same mechanisms as the hotplugging ...\n> >\n> > Still I don't get if you are talking about libcamera hotplug or\n> > android hotplug, which, to my understanding, expects anyway a camera\n> > device to be registered but marked as 'not present' (looking at the\n> > camera_device_status enumeration)\n>\n>\n> I was going to ask how does it register camera's it doesn't know will\n> exist, but I think I saw that the UVC HAL just pre-allocates some\n> cameras or such, so that explains how that one works.\n>\n\nI see three cases:\n1) No uvc support\n Defer until a pipeline handler is matched and has registered\n cameras\n\n2) UVC only\n Need to pre-register cameras and activate on hot-plug. Knowing when\n this has to happen is tricky, as the HAL needs to know it should\n not wait for built-in cameras and can proceed to register\n non-active USB cameras\n\n3) built-in + UVC\n Simmilarly, knowing we have to wait for built-in to appear, we\n defer until cameras are available, then pre-register UVC cameras.\n\nThe hard part I think it is to define how to instruct the HAL it has\nto wait for built-in to appear or not before registering UVC\nnon-active cameras.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F312EC2E68\n\tfor ;\n\tWed, 22 Jul 2020 11:08:45 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CC43160948;\n\tWed, 22 Jul 2020 13:08:45 +0200 (CEST)","from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net\n\t[217.70.183.196])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 38E766039F\n\tfor ;\n\tWed, 22 Jul 2020 13:08:44 +0200 (CEST)","from uno.localdomain (2-224-242-101.ip172.fastwebnet.it\n\t[2.224.242.101]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 22D90E000C;\n\tWed, 22 Jul 2020 11:08:42 +0000 (UTC)"],"X-Originating-IP":"2.224.242.101","Date":"Wed, 22 Jul 2020 13:12:21 +0200","From":"Jacopo Mondi ","To":"Kieran Bingham ","Message-ID":"<20200722111221.sezcchaymsnbhwgf@uno.localdomain>","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11484,"web_url":"https://patchwork.libcamera.org/comment/11484/","msgid":"","date":"2020-07-22T11:24:56","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn 22/07/2020 12:12, Jacopo Mondi wrote:\n> Hi Kieran,\n> I think the conversation diverged, as I was clearly overlooking\n> the UVC camera use case and you're missing the intended use case of\n> this patch. I'll try to reply and summarize at the end.\n\nMy interpretation is that this patch fixes the issue you are\nexperiencing, (which is a valid thing to do, and this is likely a\nsuitable solution for the time being)\n\nI was indeed trying to highlight that it would cause breakage for the\nUVC style use cases, and I believe system degradation for systems with\nno cameras attached, and which is why I thought a \\todo is required.\n\n> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n>>>> Hi Jacopo,\n>>>>\n>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n>>>>> Hi Kieran,\n>>>>>\n>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n>>>>>> Hi Jacopo,\n>>>>>>\n>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n>>>>>>> The CameraHalManager initialization tries to start the\n>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n>>>>>>>\n>>>>>>> When initialization is called too early during system boot, it might\n>>>>>>> happen that the media graphs are still being registered to user-space\n>>>>>>> preventing any pipeline handler to match and register cameras.\n>>>>>>>\n>>>>>>> If that happens, the CameraHalManager silently accepts that no\n>>>>>>> cameras are available in the system, reporting that information\n>>>>>>> to the camera stack:\n>>>>>>>\n>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n>>>>>>>\n>>>>>>\n>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n>>>>>>\n>>>>>\n>>>>> I don't get what your comment is about here, I'm sorry\n>>>>\n>>>>\n>>>> Your patch prevents loading of libcamera completely unless there is at\n>>>> least one camera right ?\n>>>>\n>>>> What happens on say an x86 platform which will only have UVC cameras,\n>>>> yet will use libcamera...\n>>>>\n>>>> When will it be acceptable to load the camera service.\n>>>>\n>>>> Should the service itself continually fail, and respawn until someone\n>>>> plugs in a camera?\n>>>>\n>>>\n>>> That's my understanding\n>>\n>> That sounds like that would cause a very bad implementation to me.\n>>\n>> Waiting for upstart to decide when to retry launching the camera-service\n>> wouldn't be an acceptable time to wait IMO.\n>>\n>> Either it tries every second, and is flooding the system, or it tries\n>> every minute and it takes a minute before you can view your newly\n>> attached UVC camera ...\n>>\n>> I would expect event driven camera attachment, not polled ...\n>>\n> \n> For UVC we will have to register non-active cameras and notify to\n> android they're available when hotplug is detected\n\n\nYes, this is what I think I see that the platform2/camera/hal/usb*\nimplementation does in CrOS.\n\n\n> For non-pluggable cameras, as shall wait until the pipeline handler is\n> matched and has registered cameras\n\nThe issue for us is that *we support UVC* ... so we're now a UVC capable\nstack.\n\nI believe that means we will in some form or another have to also do the\nsame (some sort of pre-allocation if that's what is needed to make\nandroid happy). I dislike the idea that we would then have a 'maximum\nsupported cameras' but maybe it is set arbitrarily high so it doesn't\nget hit without an extreme use case.\n\n\nNow ... given that we (if we support UVC) must support dynamically\nadding cameras when they are available - I anticipate that the hotplug\nwork will do that.\n\nWhen we do that, We could also use it for -non hotplug cameras, and\nsimply allow cameras to be registered correctly using the same methods\nas they are discovered/notified by udev/hotplug mechanisms.\n\nBecause even if they are fixed CSI2 busses, we're still going to get\nudev events when they appear as a media-device right ?\n\n\n\n>>>> So taking that further, what I'm saying is, perhaps this should be\n>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n>>>> created they should notify android that a new camera is now available.\n>>>\n>>> Do you know if that's even possible ?\n>>>\n>>> I think Android expects the number of possible cameras to be\n>>> statically defined, there's a callback to notify a change of status of\n>>> a device, but we're dealing here with -creating- cameras based on the\n>>> number of cameras detected by android.\n>>\n>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n>> some cameras or such and only activate them on device connection.\n>>\n>> I fear we might have to do the same somehow in the future, as indeed it\n>> might not be possible to tell android there is a 'new' camera. Only an\n>> update to an existing one... not sure it will require more investigation.\n>>\n> \n> Yes, but that's for UVC. Built-in cameras are different, and Android\n> bets on the fact if you have a camera service running then your system\n> has cameras which are expected to be registered.\n> \n> Keep in mind so far there's been an HAL for UVC and one for built-in\n> cameras. We're handling both and that's a new use case afaict\n\nPrecisely - and that's what I've been trying to discuss on this patch\n;-) I'm sorry my thoughts didn't come across clearly.\n\n\n\n\n>>>>>>> Fix this by returning an error code if no camera is registered in the\n>>>>>>> system at the time CameraHalManager::init() is called. The camera\n>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n>>>>>>> after the media device dependencies have been registered:\n>>>>>>>\n>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>>>> ....\n>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n>>>>>>>\n>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n>>>>>>> between that and other negative values, so it does not really make\n>>>>>>> a difference.\n>>>>>>\n>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n>>>>>> which doesn't yet have any cameras connected).\n>>>>>\n>>>>> You keep defferring ? I think the framework handles that\n>>>>\n>>>> I would be surprised if it would be expected behaviour to just poll\n>>>> launching of the camera service until there is a camera there to satisfy\n>>>> it ...\n>>>>\n>>>>\n>>>>>> How many times will we defer? How long does it defer for for instance?\n>>>>>>\n>>>>>\n>>>>> Usually 1 is enough from my testing. I don't think we have to care\n>>>>> what is the timeout in the framework, as afaict android services are\n>>>>> restarted by default.\n>>>>\n>>>>\n>>>> Ok, so this is mostly dealing with the race at startup ...\n>>>>\n>>>>\n>>>>\n>>>>>>> Signed-off-by: Jacopo Mondi \n>>>>>>> ---\n>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n>>>>>>> 1 file changed, 11 insertions(+)\n>>>>>>>\n>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n>>>>>>> index 02b6418fb36d..e967d210e547 100644\n>>>>>>> --- a/src/android/camera_hal_manager.cpp\n>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n>>>>>>> \t\t++index;\n>>>>>>> \t}\n>>>>>>>\n>>>>>>> +\t/*\n>>>>>>> +\t * If no pipeline has registered cameras, defer initialization to give\n>>>>>>> +\t * time to media devices to register to user-space.\n>>>>>>> +\t */\n>>>>>>> +\tif (index == 0) {\n>>>>>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n>>>>>>\n>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n>>>>>> always prints somewhere.\n>>>>>\n>>>>> I think that's kind of expected to happen, I don't think this should\n>>>>> be an error\n>>>>\n>>>>\n>>>> Info? Warning?\n>>>>\n>>>> If libcamera has prevented loading, I think it would be good to know\n>>>> about it by default...\n>>>>\n>>>>\n>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n>>>>>> this would go through if only one camera of two in the system is available).\n>>>>>\n>>>>> If the pipeline handler match (ie the device enumerator matches the\n>>>>> dependencies) than the match() function continues and register\n>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n>>>>> state.\n>>>>\n>>>>\n>>>> Ok, so it really is about having at least one camera available?\n>>>\n>>> It's about at least a pipeline being matched.\n>>\n>> Ok, but I don't think it's that difficult to conjur up a scenario where\n>> there won't be a device\n> \n> Not for what Android has been thought to work on. You integrate a\n> system with cameras, it's fair to defer until those cameras are\n> registered. You have no cameras, either you disable the camera stack\n> or you register 0 cameras and then the camera subsystem is silent and\n> not accessible.\n\nAndroid is wrong - It should always support UVC hotplug ;-) hehehe.\n\nI've been annoyed I haven't been able to connect a UVC camera to my\nphone before ;-)\n\n\n\n>>>> I think looking at CameraHalManager::init(), as it only creates a\n>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n>>>> unlikely to get in here between the point that one has been available\n>>>> and another is still loading ...\n>>>\n>>> I don't think that's possible.\n>>>\n>>> If a all the dependencies of a pipeline handler are matched, the\n>>> camera are registered. What is the code flow that registers one\n>>> camera, returns to the camera manager, then calls the pipeline handler\n>>> again to register more ?\n>>\n>>\n>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n>> each camera attached.\n>>\n>> But yes, perhaps due to the threading model, it won't be a possible race.\n>>\n>> I was thinking of when a pipeline starts and only 'one' of it's media\n>> devices is available.\n> \n> Yes, for UVC that's different I agree. A call to\n> CameraManager::start() might match one uvc media device but miss a\n> second one which still have to appear to userspace.\n> \n> As said, UVC needs some special handling and pre-register cameras\n\nYes, and we will therefore /have/ to do that.\n\n\n\n>>>> But still ... I think this is stuff that will get dealt with by hotplug\n>>>> being added to the HAL.\n>>>\n>>> I agree, we should support inserting and removing new cameras, but I\n>>> think at this point, we should really start only if cameras are\n>>> registered and any pipeline handler is matched.\n>>>\n>>> I'm not sure I got what other alternative approach you are proposing\n>>> to be honest.\n>>\n>> I'm suggesting that when hotplug support is available, this patch, if\n>> integrated, might need to be reverted in effect ... and thus a todo note\n>> should be added to say that or such.\n>>\n>> I believe we should be able to start the camera service successfully\n>> with zero cameras if there are zero cameras at the time the service starts.\n> \n> I don't think so for built-in cameras. We start the service when\n> cameras are registered, otherwise if you have to pre-allocate cameras\n> you would need to know how many of them the pipeline handler expects\n> to register and that's not known before it has been matched.\n\n\nYes, not knowing in advance is a pain point.\n\n\n> UVC, again, is a different story as it supports hotplug.\n\nBut equally we don't know how many UVC cameras someone could connect,\nbut we can define a max. The question would then be if that 'max'\nincludes static cameras, as well as dynamic UVC cameras, or if it's 'on\ntop' of the static cameras.\n\nIncluding it in the max, means we can just preallocate the camera\nregistrations for Android sake, and register the (static) cameras in\nthat list as soon as they are available through the udev notifications\nsystem.\n\n\n>>>>>> ..\n>>>>>>\n>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n>>>>>> that case I think it needs a \\todo or a warning of some kind...\n>>>>>>\n>>>>>\n>>>>> A \\todo item to record this should be fixed how ?\n>>>>>\n>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n>>>>> handler it tries match, but this seems not a good idea to me.\n>>>>\n>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n>>>> and if there are only 0 cameras, it will only have zero cameras - until\n>>>\n>>> That means Android/CrOS starts without the camera being active (ie.\n>>> the camera application icon is not available, in CrOS in example).\n>>\n>>\n>> Hrm, so it disables the app completely?\n>>\n>> Anyway, essentially I'm thinking - whatever happens with the existing\n>> UVC stack is what we would need to be able to mirror.\n>>\n>>\n>>\n>>>> they become available at which point they'll be registered through the\n>>>> same mechanisms as the hotplugging ...\n>>>\n>>> Still I don't get if you are talking about libcamera hotplug or\n>>> android hotplug, which, to my understanding, expects anyway a camera\n>>> device to be registered but marked as 'not present' (looking at the\n>>> camera_device_status enumeration)\n>>\n>>\n>> I was going to ask how does it register camera's it doesn't know will\n>> exist, but I think I saw that the UVC HAL just pre-allocates some\n>> cameras or such, so that explains how that one works.\n>>\n> \n> I see three cases:\n> 1) No uvc support\n> Defer until a pipeline handler is matched and has registered\n> cameras\n> \n> 2) UVC only\n> Need to pre-register cameras and activate on hot-plug. Knowing when\n> this has to happen is tricky, as the HAL needs to know it should\n> not wait for built-in cameras and can proceed to register\n> non-active USB cameras\n> \n> 3) built-in + UVC\n> Simmilarly, knowing we have to wait for built-in to appear, we\n> defer until cameras are available, then pre-register UVC cameras.\n> \n> The hard part I think it is to define how to instruct the HAL it has\n> to wait for built-in to appear or not before registering UVC\n> non-active cameras.\n\n\nMy suggestion is (not blocking this patch, but on top) to do so by\ntreating static cameras and hotpluggable cameras in the same way.\n\nIt's just that static cameras will never 'unplug' (unless someone\nunbinds them? but we all know how bad that mess would get with V4L2 anyway)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A03D5C2E68\n\tfor ;\n\tWed, 22 Jul 2020 11:25:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 16D5C6093D;\n\tWed, 22 Jul 2020 13:25:02 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F6B86039F\n\tfor ;\n\tWed, 22 Jul 2020 13:25:00 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id F0487AE6;\n\tWed, 22 Jul 2020 13:24:59 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"GALJDqzD\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595417100;\n\tbh=F5fC2OfcbAib6LYPJXrOSMpI8mEB25CnoF1UeX3prE8=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=GALJDqzDqVvkARGrUxlC0pZKoqgnCtIkPk04y/DfGbXvQxzFiIPcDB6jJnW2KAG1p\n\tLRIhlCpbnR9zhpOi7cBdW1wUSPM5c7ur+L/6E9ysiDEUoGSo3sI4n98F00NUFvqA82\n\tTg6u8rLtYBD5Mupm5rGhJHmE9dKiTzBNT70IerQ4=","To":"Jacopo Mondi ","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"","Date":"Wed, 22 Jul 2020 12:24:56 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200722111221.sezcchaymsnbhwgf@uno.localdomain>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11492,"web_url":"https://patchwork.libcamera.org/comment/11492/","msgid":"<20200722134940.GL5833@pendragon.ideasonboard.com>","date":"2020-07-22T13:49:40","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello,\n\n(CC'ing Tomasz)\n\nOn Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> > Hi Kieran,\n> > I think the conversation diverged, as I was clearly overlooking\n> > the UVC camera use case and you're missing the intended use case of\n> > this patch. I'll try to reply and summarize at the end.\n> \n> My interpretation is that this patch fixes the issue you are\n> experiencing, (which is a valid thing to do, and this is likely a\n> suitable solution for the time being)\n>\n> I was indeed trying to highlight that it would cause breakage for the\n> UVC style use cases, and I believe system degradation for systems with\n> no cameras attached, and which is why I thought a \\todo is required.\n> \n> > On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> >> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> >>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> >>>> Hi Jacopo,\n> >>>>\n> >>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> >>>>> Hi Kieran,\n> >>>>>\n> >>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >>>>>> Hi Jacopo,\n> >>>>>>\n> >>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>>>>>> The CameraHalManager initialization tries to start the\n> >>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>>>>>\n> >>>>>>> When initialization is called too early during system boot, it might\n> >>>>>>> happen that the media graphs are still being registered to user-space\n> >>>>>>> preventing any pipeline handler to match and register cameras.\n> >>>>>>>\n> >>>>>>> If that happens, the CameraHalManager silently accepts that no\n> >>>>>>> cameras are available in the system, reporting that information\n> >>>>>>> to the camera stack:\n> >>>>>>>\n> >>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>>>>>\n> >>>>>>\n> >>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> >>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>>>>>\n> >>>>>\n> >>>>> I don't get what your comment is about here, I'm sorry\n> >>>>\n> >>>>\n> >>>> Your patch prevents loading of libcamera completely unless there is at\n> >>>> least one camera right ?\n> >>>>\n> >>>> What happens on say an x86 platform which will only have UVC cameras,\n> >>>> yet will use libcamera...\n> >>>>\n> >>>> When will it be acceptable to load the camera service.\n> >>>>\n> >>>> Should the service itself continually fail, and respawn until someone\n> >>>> plugs in a camera?\n> >>>>\n> >>>\n> >>> That's my understanding\n> >>\n> >> That sounds like that would cause a very bad implementation to me.\n> >>\n> >> Waiting for upstart to decide when to retry launching the camera-service\n> >> wouldn't be an acceptable time to wait IMO.\n> >>\n> >> Either it tries every second, and is flooding the system, or it tries\n> >> every minute and it takes a minute before you can view your newly\n> >> attached UVC camera ...\n> >>\n> >> I would expect event driven camera attachment, not polled ...\n> > \n> > For UVC we will have to register non-active cameras and notify to\n> > android they're available when hotplug is detected\n> \n> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> implementation does in CrOS.\n> \n> > For non-pluggable cameras, as shall wait until the pipeline handler is\n> > matched and has registered cameras\n> \n> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> stack.\n> \n> I believe that means we will in some form or another have to also do the\n> same (some sort of pre-allocation if that's what is needed to make\n> android happy). I dislike the idea that we would then have a 'maximum\n> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> get hit without an extreme use case.\n\nI don't think we can do this, as, as far as I know, Android doesn't\nsupport status changes for internal cameras. Only cameras reported as\nexternal can generate a status change event. Quoting from \ncamera/provider/2.4/ICameraProviderCallback.hal in\nhttps://android.googlesource.com/platform/hardware/interfaces:\n\n * On camera service startup, when ICameraProvider::setCallback is invoked,\n * the camera service must assume that all internal camera devices are in\n * the CAMERA_DEVICE_STATUS_PRESENT state.\n *\n * The provider must call this method to inform the camera service of any\n * initially NOT_PRESENT devices, and of any external camera devices that\n * are already present, as soon as the callbacks are available through\n * setCallback.\n\n> Now ... given that we (if we support UVC) must support dynamically\n> adding cameras when they are available - I anticipate that the hotplug\n> work will do that.\n> \n> When we do that, We could also use it for -non hotplug cameras, and\n> simply allow cameras to be registered correctly using the same methods\n> as they are discovered/notified by udev/hotplug mechanisms.\n> \n> Because even if they are fixed CSI2 busses, we're still going to get\n> udev events when they appear as a media-device right ?\n\nYes, on the libcamera side hotplug should work perfectly fine for\ninternal cameras, but unfortunately not on the Android side.\n\n> >>>> So taking that further, what I'm saying is, perhaps this should be\n> >>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> >>>> created they should notify android that a new camera is now available.\n> >>>\n> >>> Do you know if that's even possible ?\n> >>>\n> >>> I think Android expects the number of possible cameras to be\n> >>> statically defined, there's a callback to notify a change of status of\n> >>> a device, but we're dealing here with -creating- cameras based on the\n> >>> number of cameras detected by android.\n> >>\n> >> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> >> some cameras or such and only activate them on device connection.\n> >>\n> >> I fear we might have to do the same somehow in the future, as indeed it\n> >> might not be possible to tell android there is a 'new' camera. Only an\n> >> update to an existing one... not sure it will require more investigation.\n> > \n> > Yes, but that's for UVC. Built-in cameras are different, and Android\n> > bets on the fact if you have a camera service running then your system\n> > has cameras which are expected to be registered.\n> > \n> > Keep in mind so far there's been an HAL for UVC and one for built-in\n> > cameras. We're handling both and that's a new use case afaict\n> \n> Precisely - and that's what I've been trying to discuss on this patch\n> ;-) I'm sorry my thoughts didn't come across clearly.\n> \n> >>>>>>> Fix this by returning an error code if no camera is registered in the\n> >>>>>>> system at the time CameraHalManager::init() is called. The camera\n> >>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> >>>>>>> after the media device dependencies have been registered:\n> >>>>>>>\n> >>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>> ....\n> >>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>>>>>\n> >>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> >>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>>>>>> between that and other negative values, so it does not really make\n> >>>>>>> a difference.\n> >>>>>>\n> >>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> >>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >>>>>> which doesn't yet have any cameras connected).\n> >>>>>\n> >>>>> You keep defferring ? I think the framework handles that\n> >>>>\n> >>>> I would be surprised if it would be expected behaviour to just poll\n> >>>> launching of the camera service until there is a camera there to satisfy\n> >>>> it ...\n\nThis is actually how the camera HALs in Chrome OS handle this issue :-(\n\n> >>>>>> How many times will we defer? How long does it defer for for instance?\n> >>>>>\n> >>>>> Usually 1 is enough from my testing. I don't think we have to care\n> >>>>> what is the timeout in the framework, as afaict android services are\n> >>>>> restarted by default.\n> >>>>\n> >>>> Ok, so this is mostly dealing with the race at startup ...\n> >>>>\n> >>>>>>> Signed-off-by: Jacopo Mondi \n> >>>>>>> ---\n> >>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>>>>>> 1 file changed, 11 insertions(+)\n> >>>>>>>\n> >>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>> index 02b6418fb36d..e967d210e547 100644\n> >>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>>>>>> \t\t++index;\n> >>>>>>> \t}\n> >>>>>>>\n> >>>>>>> +\t/*\n> >>>>>>> +\t * If no pipeline has registered cameras, defer initialization to give\n> >>>>>>> +\t * time to media devices to register to user-space.\n> >>>>>>> +\t */\n> >>>>>>> +\tif (index == 0) {\n> >>>>>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>>>>>\n> >>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> >>>>>> always prints somewhere.\n> >>>>>\n> >>>>> I think that's kind of expected to happen, I don't think this should\n> >>>>> be an error\n> >>>>\n> >>>> Info? Warning?\n> >>>>\n> >>>> If libcamera has prevented loading, I think it would be good to know\n> >>>> about it by default...\n> >>>>\n> >>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> >>>>>> this would go through if only one camera of two in the system is available).\n> >>>>>\n> >>>>> If the pipeline handler match (ie the device enumerator matches the\n> >>>>> dependencies) than the match() function continues and register\n> >>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> >>>>> state.\n> >>>>\n> >>>> Ok, so it really is about having at least one camera available?\n> >>>\n> >>> It's about at least a pipeline being matched.\n> >>\n> >> Ok, but I don't think it's that difficult to conjur up a scenario where\n> >> there won't be a device\n> > \n> > Not for what Android has been thought to work on. You integrate a\n> > system with cameras, it's fair to defer until those cameras are\n> > registered. You have no cameras, either you disable the camera stack\n> > or you register 0 cameras and then the camera subsystem is silent and\n> > not accessible.\n> \n> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> \n> I've been annoyed I haven't been able to connect a UVC camera to my\n> phone before ;-)\n> \n> >>>> I think looking at CameraHalManager::init(), as it only creates a\n> >>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> >>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> >>>> unlikely to get in here between the point that one has been available\n> >>>> and another is still loading ...\n> >>>\n> >>> I don't think that's possible.\n> >>>\n> >>> If a all the dependencies of a pipeline handler are matched, the\n> >>> camera are registered. What is the code flow that registers one\n> >>> camera, returns to the camera manager, then calls the pipeline handler\n> >>> again to register more ?\n> >>\n> >> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> >> each camera attached.\n> >>\n> >> But yes, perhaps due to the threading model, it won't be a possible race.\n> >>\n> >> I was thinking of when a pipeline starts and only 'one' of it's media\n> >> devices is available.\n> > \n> > Yes, for UVC that's different I agree. A call to\n> > CameraManager::start() might match one uvc media device but miss a\n> > second one which still have to appear to userspace.\n> > \n> > As said, UVC needs some special handling and pre-register cameras\n> \n> Yes, and we will therefore /have/ to do that.\n\nI also don't see any way around this *for UVC*.\n\n> >>>> But still ... I think this is stuff that will get dealt with by hotplug\n> >>>> being added to the HAL.\n> >>>\n> >>> I agree, we should support inserting and removing new cameras, but I\n> >>> think at this point, we should really start only if cameras are\n> >>> registered and any pipeline handler is matched.\n> >>>\n> >>> I'm not sure I got what other alternative approach you are proposing\n> >>> to be honest.\n> >>\n> >> I'm suggesting that when hotplug support is available, this patch, if\n> >> integrated, might need to be reverted in effect ... and thus a todo note\n> >> should be added to say that or such.\n> >>\n> >> I believe we should be able to start the camera service successfully\n> >> with zero cameras if there are zero cameras at the time the service starts.\n> > \n> > I don't think so for built-in cameras. We start the service when\n> > cameras are registered, otherwise if you have to pre-allocate cameras\n> > you would need to know how many of them the pipeline handler expects\n> > to register and that's not known before it has been matched.\n> \n> Yes, not knowing in advance is a pain point.\n> \n> > UVC, again, is a different story as it supports hotplug.\n> \n> But equally we don't know how many UVC cameras someone could connect,\n> but we can define a max. The question would then be if that 'max'\n> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> top' of the static cameras.\n> \n> Including it in the max, means we can just preallocate the camera\n> registrations for Android sake, and register the (static) cameras in\n> that list as soon as they are available through the udev notifications\n> system.\n> \n> >>>>>> ..\n> >>>>>>\n> >>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> >>>>>> that case I think it needs a \\todo or a warning of some kind...\n> >>>>>\n> >>>>> A \\todo item to record this should be fixed how ?\n> >>>>>\n> >>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> >>>>> handler it tries match, but this seems not a good idea to me.\n> >>>>\n> >>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> >>>> and if there are only 0 cameras, it will only have zero cameras - until\n> >>>\n> >>> That means Android/CrOS starts without the camera being active (ie.\n> >>> the camera application icon is not available, in CrOS in example).\n> >>\n> >> Hrm, so it disables the app completely?\n> >>\n> >> Anyway, essentially I'm thinking - whatever happens with the existing\n> >> UVC stack is what we would need to be able to mirror.\n> >>\n> >>>> they become available at which point they'll be registered through the\n> >>>> same mechanisms as the hotplugging ...\n> >>>\n> >>> Still I don't get if you are talking about libcamera hotplug or\n> >>> android hotplug, which, to my understanding, expects anyway a camera\n> >>> device to be registered but marked as 'not present' (looking at the\n> >>> camera_device_status enumeration)\n> >>\n> >>\n> >> I was going to ask how does it register camera's it doesn't know will\n> >> exist, but I think I saw that the UVC HAL just pre-allocates some\n> >> cameras or such, so that explains how that one works.\n> >>\n> > \n> > I see three cases:\n> > 1) No uvc support\n> > Defer until a pipeline handler is matched and has registered\n> > cameras\n> > \n> > 2) UVC only\n> > Need to pre-register cameras and activate on hot-plug. Knowing when\n> > this has to happen is tricky, as the HAL needs to know it should\n> > not wait for built-in cameras and can proceed to register\n> > non-active USB cameras\n> > \n> > 3) built-in + UVC\n> > Simmilarly, knowing we have to wait for built-in to appear, we\n> > defer until cameras are available, then pre-register UVC cameras.\n> > \n> > The hard part I think it is to define how to instruct the HAL it has\n> > to wait for built-in to appear or not before registering UVC\n> > non-active cameras.\n> \n> My suggestion is (not blocking this patch, but on top) to do so by\n> treating static cameras and hotpluggable cameras in the same way.\n> \n> It's just that static cameras will never 'unplug' (unless someone\n> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n\nAs explained above, this is unfortunately not an option. To make this\nmatter more complicated, we need to wait for *all* built-in cameras to\nbe available before initializing, as otherwise the system will be\npermanently stuck with only part of the cameras available.\n\nI think this isn't something we should solve, the system should ensure\nthat device nodes have been created before starting the camera service.\nI've expressed this before in a conversation with Tomasz (not sure if it\nwas on the public mailing list, IRC, or in private), but we didn't reach\nan agreement at the time. One option that was discussed was a platform\nconfiguration file that would list the cameras expected to be present (I\nthink everybody knows how much I like that :-)). Another option that has\nbeen experimented was\nhttps://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 401F5C2E68\n\tfor ;\n\tWed, 22 Jul 2020 13:49:48 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 10C2260983;\n\tWed, 22 Jul 2020 15:49:48 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 034B76053C\n\tfor ;\n\tWed, 22 Jul 2020 15:49:47 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 5DB64AE6;\n\tWed, 22 Jul 2020 15:49:45 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"iB8RCOoE\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595425785;\n\tbh=qr5x0435r9NUKd126bKOjpuPDJLLnv+zY7l3cRbA6X0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=iB8RCOoEF6eQ9AY+Q28d/zQ4NG1hgJ2CN3nXBQYjvWT8hJ4H1JdQqgfaics+ob/5w\n\tDbI/1k+Kvs1fr/v5ThfQuWY33oBli1yUnnhZm37ej0d++VchOAAtvMvnrUBtVfPB7u\n\tQInt52zvcUeh86ZnbmzRk3KedOIXgnz0cnRl57DQ=","Date":"Wed, 22 Jul 2020 16:49:40 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"<20200722134940.GL5833@pendragon.ideasonboard.com>","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Tomasz Figa , libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11494,"web_url":"https://patchwork.libcamera.org/comment/11494/","msgid":"<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","date":"2020-07-22T14:19:28","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Laurent, Jacopo,\n\nJacopo, I'm sorry for the noise, it seems my opinions were based on an\ninvalid assumption about what is reasonable :-( and a lack of\nunderstanding of how CrOS implemented working around that assumption.\n\nSo my whole premise has been completely wrong and should be ignored.\n\n\nOn 22/07/2020 14:49, Laurent Pinchart wrote:\n> Hello,\n> \n> (CC'ing Tomasz)\n> \n> On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n>> On 22/07/2020 12:12, Jacopo Mondi wrote:\n>>> Hi Kieran,\n>>> I think the conversation diverged, as I was clearly overlooking\n>>> the UVC camera use case and you're missing the intended use case of\n>>> this patch. I'll try to reply and summarize at the end.\n>>\n>> My interpretation is that this patch fixes the issue you are\n>> experiencing, (which is a valid thing to do, and this is likely a\n>> suitable solution for the time being)\n>>\n>> I was indeed trying to highlight that it would cause breakage for the\n>> UVC style use cases, and I believe system degradation for systems with\n>> no cameras attached, and which is why I thought a \\todo is required.\n>>\n>>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n>>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n>>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n>>>>>> Hi Jacopo,\n>>>>>>\n>>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n>>>>>>> Hi Kieran,\n>>>>>>>\n>>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n>>>>>>>> Hi Jacopo,\n>>>>>>>>\n>>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n>>>>>>>>> The CameraHalManager initialization tries to start the\n>>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n>>>>>>>>>\n>>>>>>>>> When initialization is called too early during system boot, it might\n>>>>>>>>> happen that the media graphs are still being registered to user-space\n>>>>>>>>> preventing any pipeline handler to match and register cameras.\n>>>>>>>>>\n>>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n>>>>>>>>> cameras are available in the system, reporting that information\n>>>>>>>>> to the camera stack:\n>>>>>>>>>\n>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n>>>>>>>>>\n>>>>>>>>\n>>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n>>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n>>>>>>>>\n>>>>>>>\n>>>>>>> I don't get what your comment is about here, I'm sorry\n>>>>>>\n>>>>>>\n>>>>>> Your patch prevents loading of libcamera completely unless there is at\n>>>>>> least one camera right ?\n>>>>>>\n>>>>>> What happens on say an x86 platform which will only have UVC cameras,\n>>>>>> yet will use libcamera...\n>>>>>>\n>>>>>> When will it be acceptable to load the camera service.\n>>>>>>\n>>>>>> Should the service itself continually fail, and respawn until someone\n>>>>>> plugs in a camera?\n>>>>>>\n>>>>>\n>>>>> That's my understanding\n>>>>\n>>>> That sounds like that would cause a very bad implementation to me.\n>>>>\n>>>> Waiting for upstart to decide when to retry launching the camera-service\n>>>> wouldn't be an acceptable time to wait IMO.\n>>>>\n>>>> Either it tries every second, and is flooding the system, or it tries\n>>>> every minute and it takes a minute before you can view your newly\n>>>> attached UVC camera ...\n>>>>\n>>>> I would expect event driven camera attachment, not polled ...\n>>>\n>>> For UVC we will have to register non-active cameras and notify to\n>>> android they're available when hotplug is detected\n>>\n>> Yes, this is what I think I see that the platform2/camera/hal/usb*\n>> implementation does in CrOS.\n>>\n>>> For non-pluggable cameras, as shall wait until the pipeline handler is\n>>> matched and has registered cameras\n>>\n>> The issue for us is that *we support UVC* ... so we're now a UVC capable\n>> stack.\n>>\n>> I believe that means we will in some form or another have to also do the\n>> same (some sort of pre-allocation if that's what is needed to make\n>> android happy). I dislike the idea that we would then have a 'maximum\n>> supported cameras' but maybe it is set arbitrarily high so it doesn't\n>> get hit without an extreme use case.\n> \n> I don't think we can do this, as, as far as I know, Android doesn't\n> support status changes for internal cameras. Only cameras reported as\n> external can generate a status change event. Quoting from \n> camera/provider/2.4/ICameraProviderCallback.hal in\n> https://android.googlesource.com/platform/hardware/interfaces:\n> \n> * On camera service startup, when ICameraProvider::setCallback is invoked,\n> * the camera service must assume that all internal camera devices are in\n> * the CAMERA_DEVICE_STATUS_PRESENT state.\n> *\n> * The provider must call this method to inform the camera service of any\n> * initially NOT_PRESENT devices, and of any external camera devices that\n> * are already present, as soon as the callbacks are available through\n> * setCallback.\n> \n>> Now ... given that we (if we support UVC) must support dynamically\n>> adding cameras when they are available - I anticipate that the hotplug\n>> work will do that.\n>>\n>> When we do that, We could also use it for -non hotplug cameras, and\n>> simply allow cameras to be registered correctly using the same methods\n>> as they are discovered/notified by udev/hotplug mechanisms.\n>>\n>> Because even if they are fixed CSI2 busses, we're still going to get\n>> udev events when they appear as a media-device right ?\n> \n> Yes, on the libcamera side hotplug should work perfectly fine for\n> internal cameras, but unfortunately not on the Android side.\n> \n>>>>>> So taking that further, what I'm saying is, perhaps this should be\n>>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n>>>>>> created they should notify android that a new camera is now available.\n>>>>>\n>>>>> Do you know if that's even possible ?\n>>>>>\n>>>>> I think Android expects the number of possible cameras to be\n>>>>> statically defined, there's a callback to notify a change of status of\n>>>>> a device, but we're dealing here with -creating- cameras based on the\n>>>>> number of cameras detected by android.\n>>>>\n>>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n>>>> some cameras or such and only activate them on device connection.\n>>>>\n>>>> I fear we might have to do the same somehow in the future, as indeed it\n>>>> might not be possible to tell android there is a 'new' camera. Only an\n>>>> update to an existing one... not sure it will require more investigation.\n>>>\n>>> Yes, but that's for UVC. Built-in cameras are different, and Android\n>>> bets on the fact if you have a camera service running then your system\n>>> has cameras which are expected to be registered.\n>>>\n>>> Keep in mind so far there's been an HAL for UVC and one for built-in\n>>> cameras. We're handling both and that's a new use case afaict\n>>\n>> Precisely - and that's what I've been trying to discuss on this patch\n>> ;-) I'm sorry my thoughts didn't come across clearly.\n>>\n>>>>>>>>> Fix this by returning an error code if no camera is registered in the\n>>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n>>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n>>>>>>>>> after the media device dependencies have been registered:\n>>>>>>>>>\n>>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>>>>>> ....\n>>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n>>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n>>>>>>>>>\n>>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n>>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n>>>>>>>>> between that and other negative values, so it does not really make\n>>>>>>>>> a difference.\n>>>>>>>>\n>>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n>>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n>>>>>>>> which doesn't yet have any cameras connected).\n>>>>>>>\n>>>>>>> You keep defferring ? I think the framework handles that\n>>>>>>\n>>>>>> I would be surprised if it would be expected behaviour to just poll\n>>>>>> launching of the camera service until there is a camera there to satisfy\n>>>>>> it ...\n> \n> This is actually how the camera HALs in Chrome OS handle this issue :-(\n\n\nOh dear :-( Well I guess that invalidates all my assumptions\n\n/me throws toys out of the pram.\n\n\n>>>>>>>> How many times will we defer? How long does it defer for for instance?\n>>>>>>>\n>>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n>>>>>>> what is the timeout in the framework, as afaict android services are\n>>>>>>> restarted by default.\n>>>>>>\n>>>>>> Ok, so this is mostly dealing with the race at startup ...\n>>>>>>\n>>>>>>>>> Signed-off-by: Jacopo Mondi \n>>>>>>>>> ---\n>>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n>>>>>>>>> 1 file changed, 11 insertions(+)\n>>>>>>>>>\n>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n>>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n>>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n>>>>>>>>> \t\t++index;\n>>>>>>>>> \t}\n>>>>>>>>>\n>>>>>>>>> +\t/*\n>>>>>>>>> +\t * If no pipeline has registered cameras, defer initialization to give\n>>>>>>>>> +\t * time to media devices to register to user-space.\n>>>>>>>>> +\t */\n>>>>>>>>> +\tif (index == 0) {\n>>>>>>>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n>>>>>>>>\n>>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n>>>>>>>> always prints somewhere.\n>>>>>>>\n>>>>>>> I think that's kind of expected to happen, I don't think this should\n>>>>>>> be an error\n>>>>>>\n>>>>>> Info? Warning?\n>>>>>>\n>>>>>> If libcamera has prevented loading, I think it would be good to know\n>>>>>> about it by default...\n>>>>>>\n>>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n>>>>>>>> this would go through if only one camera of two in the system is available).\n>>>>>>>\n>>>>>>> If the pipeline handler match (ie the device enumerator matches the\n>>>>>>> dependencies) than the match() function continues and register\n>>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n>>>>>>> state.\n>>>>>>\n>>>>>> Ok, so it really is about having at least one camera available?\n>>>>>\n>>>>> It's about at least a pipeline being matched.\n>>>>\n>>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n>>>> there won't be a device\n>>>\n>>> Not for what Android has been thought to work on. You integrate a\n>>> system with cameras, it's fair to defer until those cameras are\n>>> registered. You have no cameras, either you disable the camera stack\n>>> or you register 0 cameras and then the camera subsystem is silent and\n>>> not accessible.\n>>\n>> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n>>\n>> I've been annoyed I haven't been able to connect a UVC camera to my\n>> phone before ;-)\n>>\n>>>>>> I think looking at CameraHalManager::init(), as it only creates a\n>>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n>>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n>>>>>> unlikely to get in here between the point that one has been available\n>>>>>> and another is still loading ...\n>>>>>\n>>>>> I don't think that's possible.\n>>>>>\n>>>>> If a all the dependencies of a pipeline handler are matched, the\n>>>>> camera are registered. What is the code flow that registers one\n>>>>> camera, returns to the camera manager, then calls the pipeline handler\n>>>>> again to register more ?\n>>>>\n>>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n>>>> each camera attached.\n>>>>\n>>>> But yes, perhaps due to the threading model, it won't be a possible race.\n>>>>\n>>>> I was thinking of when a pipeline starts and only 'one' of it's media\n>>>> devices is available.\n>>>\n>>> Yes, for UVC that's different I agree. A call to\n>>> CameraManager::start() might match one uvc media device but miss a\n>>> second one which still have to appear to userspace.\n>>>\n>>> As said, UVC needs some special handling and pre-register cameras\n>>\n>> Yes, and we will therefore /have/ to do that.\n> \n> I also don't see any way around this *for UVC*.\n> \n>>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n>>>>>> being added to the HAL.\n>>>>>\n>>>>> I agree, we should support inserting and removing new cameras, but I\n>>>>> think at this point, we should really start only if cameras are\n>>>>> registered and any pipeline handler is matched.\n>>>>>\n>>>>> I'm not sure I got what other alternative approach you are proposing\n>>>>> to be honest.\n>>>>\n>>>> I'm suggesting that when hotplug support is available, this patch, if\n>>>> integrated, might need to be reverted in effect ... and thus a todo note\n>>>> should be added to say that or such.\n>>>>\n>>>> I believe we should be able to start the camera service successfully\n>>>> with zero cameras if there are zero cameras at the time the service starts.\n>>>\n>>> I don't think so for built-in cameras. We start the service when\n>>> cameras are registered, otherwise if you have to pre-allocate cameras\n>>> you would need to know how many of them the pipeline handler expects\n>>> to register and that's not known before it has been matched.\n>>\n>> Yes, not knowing in advance is a pain point.\n>>\n>>> UVC, again, is a different story as it supports hotplug.\n>>\n>> But equally we don't know how many UVC cameras someone could connect,\n>> but we can define a max. The question would then be if that 'max'\n>> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n>> top' of the static cameras.\n>>\n>> Including it in the max, means we can just preallocate the camera\n>> registrations for Android sake, and register the (static) cameras in\n>> that list as soon as they are available through the udev notifications\n>> system.\n>>\n>>>>>>>> ..\n>>>>>>>>\n>>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n>>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n>>>>>>>\n>>>>>>> A \\todo item to record this should be fixed how ?\n>>>>>>>\n>>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n>>>>>>> handler it tries match, but this seems not a good idea to me.\n>>>>>>\n>>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n>>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n>>>>>\n>>>>> That means Android/CrOS starts without the camera being active (ie.\n>>>>> the camera application icon is not available, in CrOS in example).\n>>>>\n>>>> Hrm, so it disables the app completely?\n>>>>\n>>>> Anyway, essentially I'm thinking - whatever happens with the existing\n>>>> UVC stack is what we would need to be able to mirror.\n>>>>\n>>>>>> they become available at which point they'll be registered through the\n>>>>>> same mechanisms as the hotplugging ...\n>>>>>\n>>>>> Still I don't get if you are talking about libcamera hotplug or\n>>>>> android hotplug, which, to my understanding, expects anyway a camera\n>>>>> device to be registered but marked as 'not present' (looking at the\n>>>>> camera_device_status enumeration)\n>>>>\n>>>>\n>>>> I was going to ask how does it register camera's it doesn't know will\n>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n>>>> cameras or such, so that explains how that one works.\n>>>>\n>>>\n>>> I see three cases:\n>>> 1) No uvc support\n>>> Defer until a pipeline handler is matched and has registered\n>>> cameras\n>>>\n>>> 2) UVC only\n>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n>>> this has to happen is tricky, as the HAL needs to know it should\n>>> not wait for built-in cameras and can proceed to register\n>>> non-active USB cameras\n>>>\n>>> 3) built-in + UVC\n>>> Simmilarly, knowing we have to wait for built-in to appear, we\n>>> defer until cameras are available, then pre-register UVC cameras.\n>>>\n>>> The hard part I think it is to define how to instruct the HAL it has\n>>> to wait for built-in to appear or not before registering UVC\n>>> non-active cameras.\n>>\n>> My suggestion is (not blocking this patch, but on top) to do so by\n>> treating static cameras and hotpluggable cameras in the same way.\n>>\n>> It's just that static cameras will never 'unplug' (unless someone\n>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> \n> As explained above, this is unfortunately not an option. To make this\n> matter more complicated, we need to wait for *all* built-in cameras to\n> be available before initializing, as otherwise the system will be\n> permanently stuck with only part of the cameras available.\n> \n> I think this isn't something we should solve, the system should ensure\n> that device nodes have been created before starting the camera service.\n\nYes, given what you've clarified, then I agree - the service should be\ndependent upon the devices being brought up. I think systemd can do\nthat, but CrOS uses upstart, so I don't know what that might support.\n\nBut that's out of scope (and control) for us I think ...\n\n\n> I've expressed this before in a conversation with Tomasz (not sure if it\n> was on the public mailing list, IRC, or in private), but we didn't reach\n> an agreement at the time. One option that was discussed was a platform\n> configuration file that would list the cameras expected to be present (I\n> think everybody knows how much I like that :-)). Another option that has\n> been experimented was\n> https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D17D3BDB1B\n\tfor ;\n\tWed, 22 Jul 2020 14:19:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 721AA6093F;\n\tWed, 22 Jul 2020 16:19:34 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CCD476053C\n\tfor ;\n\tWed, 22 Jul 2020 16:19:32 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0315F329;\n\tWed, 22 Jul 2020 16:19:31 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"FSrUxVbV\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595427572;\n\tbh=pEdE8/WqYW3EAdIkKBqYbQvhjAH+u6r+Tyi3/sruJB4=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=FSrUxVbVaH7pGZc9SqN3vUHPza0+UoSqeINbtQr3Pu1hyelzkhBwCe5d2lV6QdiLp\n\tmS8w3Niit4HqY4S2iYrOhBGeY4CXZ32uUQKIp5TszJkOFrdmxgut/Pj6IkDcFBL5JA\n\tanOINsSB4S8dU6o+9QZfajrmHtqFLDKRH3rTUAnM=","To":"Laurent Pinchart ","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","Date":"Wed, 22 Jul 2020 15:19:28 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200722134940.GL5833@pendragon.ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"Tomasz Figa , libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11497,"web_url":"https://patchwork.libcamera.org/comment/11497/","msgid":"<20200722143407.hp3b2airaxd6me3l@uno.localdomain>","date":"2020-07-22T14:34:07","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Kieran,\n\nOn Wed, Jul 22, 2020 at 03:19:28PM +0100, Kieran Bingham wrote:\n> Hi Laurent, Jacopo,\n>\n> Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> invalid assumption about what is reasonable :-( and a lack of\n> understanding of how CrOS implemented working around that assumption.\n>\n> So my whole premise has been completely wrong and should be ignored.\n\nNo worries, I think that helped clarifying what Android expects for\nexternal cameras and will help when we will support UVC hotplug, so it\nwas indeed worth it.\n\nThanks\n j","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1DCF6BDB1B\n\tfor ;\n\tWed, 22 Jul 2020 14:30:32 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E473C6093F;\n\tWed, 22 Jul 2020 16:30:31 +0200 (CEST)","from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net\n\t[217.70.183.201])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A863E6053C\n\tfor ;\n\tWed, 22 Jul 2020 16:30:30 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay8-d.mail.gandi.net (Postfix) with ESMTPSA id B74061BF203;\n\tWed, 22 Jul 2020 14:30:29 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Wed, 22 Jul 2020 16:34:07 +0200","From":"Jacopo Mondi ","To":"Kieran Bingham ","Message-ID":"<20200722143407.hp3b2airaxd6me3l@uno.localdomain>","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Tomasz Figa , libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11498,"web_url":"https://patchwork.libcamera.org/comment/11498/","msgid":"","date":"2020-07-22T14:31:42","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":9,"url":"https://patchwork.libcamera.org/api/people/9/","name":"Tomasz Figa","email":"tfiga@chromium.org"},"content":"On Wed, Jul 22, 2020 at 4:19 PM Kieran Bingham\n wrote:\n>\n> Hi Laurent, Jacopo,\n>\n> Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> invalid assumption about what is reasonable :-( and a lack of\n> understanding of how CrOS implemented working around that assumption.\n>\n> So my whole premise has been completely wrong and should be ignored.\n>\n>\n> On 22/07/2020 14:49, Laurent Pinchart wrote:\n> > Hello,\n> >\n> > (CC'ing Tomasz)\n> >\n> > On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> >> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> >>> Hi Kieran,\n> >>> I think the conversation diverged, as I was clearly overlooking\n> >>> the UVC camera use case and you're missing the intended use case of\n> >>> this patch. I'll try to reply and summarize at the end.\n> >>\n> >> My interpretation is that this patch fixes the issue you are\n> >> experiencing, (which is a valid thing to do, and this is likely a\n> >> suitable solution for the time being)\n> >>\n> >> I was indeed trying to highlight that it would cause breakage for the\n> >> UVC style use cases, and I believe system degradation for systems with\n> >> no cameras attached, and which is why I thought a \\todo is required.\n> >>\n> >>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> >>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> >>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> >>>>>> Hi Jacopo,\n> >>>>>>\n> >>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> >>>>>>> Hi Kieran,\n> >>>>>>>\n> >>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >>>>>>>> Hi Jacopo,\n> >>>>>>>>\n> >>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>>>>>>>> The CameraHalManager initialization tries to start the\n> >>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>>>>>>>\n> >>>>>>>>> When initialization is called too early during system boot, it might\n> >>>>>>>>> happen that the media graphs are still being registered to user-space\n> >>>>>>>>> preventing any pipeline handler to match and register cameras.\n> >>>>>>>>>\n> >>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n> >>>>>>>>> cameras are available in the system, reporting that information\n> >>>>>>>>> to the camera stack:\n> >>>>>>>>>\n> >>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>>>>>>>\n> >>>>>>>>\n> >>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> >>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>>>>>>>\n> >>>>>>>\n> >>>>>>> I don't get what your comment is about here, I'm sorry\n> >>>>>>\n> >>>>>>\n> >>>>>> Your patch prevents loading of libcamera completely unless there is at\n> >>>>>> least one camera right ?\n> >>>>>>\n> >>>>>> What happens on say an x86 platform which will only have UVC cameras,\n> >>>>>> yet will use libcamera...\n> >>>>>>\n> >>>>>> When will it be acceptable to load the camera service.\n> >>>>>>\n> >>>>>> Should the service itself continually fail, and respawn until someone\n> >>>>>> plugs in a camera?\n> >>>>>>\n> >>>>>\n> >>>>> That's my understanding\n> >>>>\n> >>>> That sounds like that would cause a very bad implementation to me.\n> >>>>\n> >>>> Waiting for upstart to decide when to retry launching the camera-service\n> >>>> wouldn't be an acceptable time to wait IMO.\n> >>>>\n> >>>> Either it tries every second, and is flooding the system, or it tries\n> >>>> every minute and it takes a minute before you can view your newly\n> >>>> attached UVC camera ...\n> >>>>\n> >>>> I would expect event driven camera attachment, not polled ...\n> >>>\n> >>> For UVC we will have to register non-active cameras and notify to\n> >>> android they're available when hotplug is detected\n> >>\n> >> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> >> implementation does in CrOS.\n> >>\n> >>> For non-pluggable cameras, as shall wait until the pipeline handler is\n> >>> matched and has registered cameras\n> >>\n> >> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> >> stack.\n> >>\n> >> I believe that means we will in some form or another have to also do the\n> >> same (some sort of pre-allocation if that's what is needed to make\n> >> android happy). I dislike the idea that we would then have a 'maximum\n> >> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> >> get hit without an extreme use case.\n> >\n> > I don't think we can do this, as, as far as I know, Android doesn't\n> > support status changes for internal cameras. Only cameras reported as\n> > external can generate a status change event. Quoting from\n> > camera/provider/2.4/ICameraProviderCallback.hal in\n> > https://android.googlesource.com/platform/hardware/interfaces:\n> >\n> > * On camera service startup, when ICameraProvider::setCallback is invoked,\n> > * the camera service must assume that all internal camera devices are in\n> > * the CAMERA_DEVICE_STATUS_PRESENT state.\n> > *\n> > * The provider must call this method to inform the camera service of any\n> > * initially NOT_PRESENT devices, and of any external camera devices that\n> > * are already present, as soon as the callbacks are available through\n> > * setCallback.\n> >\n> >> Now ... given that we (if we support UVC) must support dynamically\n> >> adding cameras when they are available - I anticipate that the hotplug\n> >> work will do that.\n> >>\n> >> When we do that, We could also use it for -non hotplug cameras, and\n> >> simply allow cameras to be registered correctly using the same methods\n> >> as they are discovered/notified by udev/hotplug mechanisms.\n> >>\n> >> Because even if they are fixed CSI2 busses, we're still going to get\n> >> udev events when they appear as a media-device right ?\n> >\n> > Yes, on the libcamera side hotplug should work perfectly fine for\n> > internal cameras, but unfortunately not on the Android side.\n> >\n> >>>>>> So taking that further, what I'm saying is, perhaps this should be\n> >>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> >>>>>> created they should notify android that a new camera is now available.\n> >>>>>\n> >>>>> Do you know if that's even possible ?\n> >>>>>\n> >>>>> I think Android expects the number of possible cameras to be\n> >>>>> statically defined, there's a callback to notify a change of status of\n> >>>>> a device, but we're dealing here with -creating- cameras based on the\n> >>>>> number of cameras detected by android.\n> >>>>\n> >>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> >>>> some cameras or such and only activate them on device connection.\n> >>>>\n> >>>> I fear we might have to do the same somehow in the future, as indeed it\n> >>>> might not be possible to tell android there is a 'new' camera. Only an\n> >>>> update to an existing one... not sure it will require more investigation.\n> >>>\n> >>> Yes, but that's for UVC. Built-in cameras are different, and Android\n> >>> bets on the fact if you have a camera service running then your system\n> >>> has cameras which are expected to be registered.\n> >>>\n> >>> Keep in mind so far there's been an HAL for UVC and one for built-in\n> >>> cameras. We're handling both and that's a new use case afaict\n> >>\n> >> Precisely - and that's what I've been trying to discuss on this patch\n> >> ;-) I'm sorry my thoughts didn't come across clearly.\n> >>\n> >>>>>>>>> Fix this by returning an error code if no camera is registered in the\n> >>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n> >>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> >>>>>>>>> after the media device dependencies have been registered:\n> >>>>>>>>>\n> >>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>> ....\n> >>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>>>>>>>\n> >>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> >>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>>>>>>>> between that and other negative values, so it does not really make\n> >>>>>>>>> a difference.\n> >>>>>>>>\n> >>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> >>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >>>>>>>> which doesn't yet have any cameras connected).\n> >>>>>>>\n> >>>>>>> You keep defferring ? I think the framework handles that\n> >>>>>>\n> >>>>>> I would be surprised if it would be expected behaviour to just poll\n> >>>>>> launching of the camera service until there is a camera there to satisfy\n> >>>>>> it ...\n> >\n> > This is actually how the camera HALs in Chrome OS handle this issue :-(\n>\n>\n> Oh dear :-( Well I guess that invalidates all my assumptions\n>\n> /me throws toys out of the pram.\n>\n>\n> >>>>>>>> How many times will we defer? How long does it defer for for instance?\n> >>>>>>>\n> >>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n> >>>>>>> what is the timeout in the framework, as afaict android services are\n> >>>>>>> restarted by default.\n> >>>>>>\n> >>>>>> Ok, so this is mostly dealing with the race at startup ...\n> >>>>>>\n> >>>>>>>>> Signed-off-by: Jacopo Mondi \n> >>>>>>>>> ---\n> >>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>>>>>>>> 1 file changed, 11 insertions(+)\n> >>>>>>>>>\n> >>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n> >>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>>>>>>>> ++index;\n> >>>>>>>>> }\n> >>>>>>>>>\n> >>>>>>>>> + /*\n> >>>>>>>>> + * If no pipeline has registered cameras, defer initialization to give\n> >>>>>>>>> + * time to media devices to register to user-space.\n> >>>>>>>>> + */\n> >>>>>>>>> + if (index == 0) {\n> >>>>>>>>> + LOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>>>>>>>\n> >>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> >>>>>>>> always prints somewhere.\n> >>>>>>>\n> >>>>>>> I think that's kind of expected to happen, I don't think this should\n> >>>>>>> be an error\n> >>>>>>\n> >>>>>> Info? Warning?\n> >>>>>>\n> >>>>>> If libcamera has prevented loading, I think it would be good to know\n> >>>>>> about it by default...\n> >>>>>>\n> >>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> >>>>>>>> this would go through if only one camera of two in the system is available).\n> >>>>>>>\n> >>>>>>> If the pipeline handler match (ie the device enumerator matches the\n> >>>>>>> dependencies) than the match() function continues and register\n> >>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> >>>>>>> state.\n> >>>>>>\n> >>>>>> Ok, so it really is about having at least one camera available?\n> >>>>>\n> >>>>> It's about at least a pipeline being matched.\n> >>>>\n> >>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n> >>>> there won't be a device\n> >>>\n> >>> Not for what Android has been thought to work on. You integrate a\n> >>> system with cameras, it's fair to defer until those cameras are\n> >>> registered. You have no cameras, either you disable the camera stack\n> >>> or you register 0 cameras and then the camera subsystem is silent and\n> >>> not accessible.\n> >>\n> >> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> >>\n> >> I've been annoyed I haven't been able to connect a UVC camera to my\n> >> phone before ;-)\n> >>\n> >>>>>> I think looking at CameraHalManager::init(), as it only creates a\n> >>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> >>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> >>>>>> unlikely to get in here between the point that one has been available\n> >>>>>> and another is still loading ...\n> >>>>>\n> >>>>> I don't think that's possible.\n> >>>>>\n> >>>>> If a all the dependencies of a pipeline handler are matched, the\n> >>>>> camera are registered. What is the code flow that registers one\n> >>>>> camera, returns to the camera manager, then calls the pipeline handler\n> >>>>> again to register more ?\n> >>>>\n> >>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> >>>> each camera attached.\n> >>>>\n> >>>> But yes, perhaps due to the threading model, it won't be a possible race.\n> >>>>\n> >>>> I was thinking of when a pipeline starts and only 'one' of it's media\n> >>>> devices is available.\n> >>>\n> >>> Yes, for UVC that's different I agree. A call to\n> >>> CameraManager::start() might match one uvc media device but miss a\n> >>> second one which still have to appear to userspace.\n> >>>\n> >>> As said, UVC needs some special handling and pre-register cameras\n> >>\n> >> Yes, and we will therefore /have/ to do that.\n> >\n> > I also don't see any way around this *for UVC*.\n> >\n> >>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n> >>>>>> being added to the HAL.\n> >>>>>\n> >>>>> I agree, we should support inserting and removing new cameras, but I\n> >>>>> think at this point, we should really start only if cameras are\n> >>>>> registered and any pipeline handler is matched.\n> >>>>>\n> >>>>> I'm not sure I got what other alternative approach you are proposing\n> >>>>> to be honest.\n> >>>>\n> >>>> I'm suggesting that when hotplug support is available, this patch, if\n> >>>> integrated, might need to be reverted in effect ... and thus a todo note\n> >>>> should be added to say that or such.\n> >>>>\n> >>>> I believe we should be able to start the camera service successfully\n> >>>> with zero cameras if there are zero cameras at the time the service starts.\n> >>>\n> >>> I don't think so for built-in cameras. We start the service when\n> >>> cameras are registered, otherwise if you have to pre-allocate cameras\n> >>> you would need to know how many of them the pipeline handler expects\n> >>> to register and that's not known before it has been matched.\n> >>\n> >> Yes, not knowing in advance is a pain point.\n> >>\n> >>> UVC, again, is a different story as it supports hotplug.\n> >>\n> >> But equally we don't know how many UVC cameras someone could connect,\n> >> but we can define a max. The question would then be if that 'max'\n> >> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> >> top' of the static cameras.\n> >>\n> >> Including it in the max, means we can just preallocate the camera\n> >> registrations for Android sake, and register the (static) cameras in\n> >> that list as soon as they are available through the udev notifications\n> >> system.\n> >>\n> >>>>>>>> ..\n> >>>>>>>>\n> >>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> >>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n> >>>>>>>\n> >>>>>>> A \\todo item to record this should be fixed how ?\n> >>>>>>>\n> >>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> >>>>>>> handler it tries match, but this seems not a good idea to me.\n> >>>>>>\n> >>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> >>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n> >>>>>\n> >>>>> That means Android/CrOS starts without the camera being active (ie.\n> >>>>> the camera application icon is not available, in CrOS in example).\n> >>>>\n> >>>> Hrm, so it disables the app completely?\n> >>>>\n> >>>> Anyway, essentially I'm thinking - whatever happens with the existing\n> >>>> UVC stack is what we would need to be able to mirror.\n> >>>>\n> >>>>>> they become available at which point they'll be registered through the\n> >>>>>> same mechanisms as the hotplugging ...\n> >>>>>\n> >>>>> Still I don't get if you are talking about libcamera hotplug or\n> >>>>> android hotplug, which, to my understanding, expects anyway a camera\n> >>>>> device to be registered but marked as 'not present' (looking at the\n> >>>>> camera_device_status enumeration)\n> >>>>\n> >>>>\n> >>>> I was going to ask how does it register camera's it doesn't know will\n> >>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> >>>> cameras or such, so that explains how that one works.\n> >>>>\n> >>>\n> >>> I see three cases:\n> >>> 1) No uvc support\n> >>> Defer until a pipeline handler is matched and has registered\n> >>> cameras\n> >>>\n> >>> 2) UVC only\n> >>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> >>> this has to happen is tricky, as the HAL needs to know it should\n> >>> not wait for built-in cameras and can proceed to register\n> >>> non-active USB cameras\n> >>>\n> >>> 3) built-in + UVC\n> >>> Simmilarly, knowing we have to wait for built-in to appear, we\n> >>> defer until cameras are available, then pre-register UVC cameras.\n> >>>\n> >>> The hard part I think it is to define how to instruct the HAL it has\n> >>> to wait for built-in to appear or not before registering UVC\n> >>> non-active cameras.\n> >>\n> >> My suggestion is (not blocking this patch, but on top) to do so by\n> >> treating static cameras and hotpluggable cameras in the same way.\n> >>\n> >> It's just that static cameras will never 'unplug' (unless someone\n> >> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> >\n> > As explained above, this is unfortunately not an option. To make this\n> > matter more complicated, we need to wait for *all* built-in cameras to\n> > be available before initializing, as otherwise the system will be\n> > permanently stuck with only part of the cameras available.\n> >\n> > I think this isn't something we should solve, the system should ensure\n> > that device nodes have been created before starting the camera service.\n>\n> Yes, given what you've clarified, then I agree - the service should be\n> dependent upon the devices being brought up. I think systemd can do\n> that, but CrOS uses upstart, so I don't know what that might support.\n\nAs ugly as it sounds, we have an udev rule which restarts the camera\nservice when devices matching the internal cameras show up. I believe\nwe match them by the \"video4linux\" class and particular hardware\nsubsystems, e.g. PCI or I2C.\n\n>\n> But that's out of scope (and control) for us I think ...\n>\n>\n> > I've expressed this before in a conversation with Tomasz (not sure if it\n> > was on the public mailing list, IRC, or in private), but we didn't reach\n> > an agreement at the time. One option that was discussed was a platform\n> > configuration file that would list the cameras expected to be present (I\n> > think everybody knows how much I like that :-)). Another option that has\n> > been experimented was\n> > https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.\n>\n>\n> --\n> Regards\n> --\n> Kieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 68F98BDB1B\n\tfor ;\n\tWed, 22 Jul 2020 14:31:59 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 35ED760983;\n\tWed, 22 Jul 2020 16:31:59 +0200 (CEST)","from mail-ej1-x643.google.com (mail-ej1-x643.google.com\n\t[IPv6:2a00:1450:4864:20::643])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D42A56053C\n\tfor ;\n\tWed, 22 Jul 2020 16:31:57 +0200 (CEST)","by mail-ej1-x643.google.com with SMTP id w6so2440929ejq.6\n\tfor ;\n\tWed, 22 Jul 2020 07:31:57 -0700 (PDT)","from mail-wm1-f47.google.com (mail-wm1-f47.google.com.\n\t[209.85.128.47]) by smtp.gmail.com with ESMTPSA id\n\tb98sm42082edf.24.2020.07.22.07.31.55\n\tfor \n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tWed, 22 Jul 2020 07:31:55 -0700 (PDT)","by mail-wm1-f47.google.com with SMTP id g10so4506735wmc.1\n\tfor ;\n\tWed, 22 Jul 2020 07:31:55 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"aHHpQJMS\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=Qxb2jzipeJvQBFMd9anxRbApACZTqZx3AaHR7/gqmVM=;\n\tb=aHHpQJMSnh3uKB5VBogWMf/K0EJDKQt2ZbYHXb09ltKOnyI5ZV2Yya+ERjJExqAqgb\n\tNvFcz39LqMLk79zSk2aq02HeFvmgT0iU7Oz4MzXu7n2KokF9DmothY5ltpvDgkc0Ddak\n\t9wknds9fAHNduZbQAucqNbHrDjmCl/xcxDUug=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=Qxb2jzipeJvQBFMd9anxRbApACZTqZx3AaHR7/gqmVM=;\n\tb=EaOq5+jmcPCmhBx/CJUj1C8U/vd6L9FvK+WDTumug/LtdS3SBrYq81D+ZowV8L8lFi\n\tnb2xCQn+FrH3bxYmeotayqfkuV8o4j68fo5/X2JYHZKKWwHFcFQGpwF+hdvnESBYqglC\n\tNTo8qbiov/MgUexuea9Hak2W7jPHEbmjhXrsAuv1GLNMSFQP3t+PGo0sxMhovLlro7pL\n\tfLsxjAZD9QudZnBLF3371a0ntQPi9fh2h20MLoS9Zz2xBBHMroNHHFJb6rWgQUaG13FQ\n\tGkcPU5GpzsXSIkKJmFqmRpztPmoKHcPac5gJ5nkw9BypM8wPuaqBq6pGEQWGz0YZv9c2\n\tAmlw==","X-Gm-Message-State":"AOAM532wLoSM5vTQXlMPuk75hvXcuF5j0BQAKqUGyHwfeJw67w/S4dne\n\tr6CrNPI7yUbF7XEM5Q2RoRi8/xFKhwttSw==","X-Google-Smtp-Source":"ABdhPJwuXCMCx82H5tIGJLHAN4vVxzvHyCyhyB08k7bwxEv5Gvx6mA+6jPUXCD1SqN1WqmeJQ6ZolQ==","X-Received":["by 2002:a17:906:b2c8:: with SMTP id\n\tcf8mr30806905ejb.132.1595428316308; \n\tWed, 22 Jul 2020 07:31:56 -0700 (PDT)","by 2002:a1c:e382:: with SMTP id\n\ta124mr4554507wmh.11.1595428314633; \n\tWed, 22 Jul 2020 07:31:54 -0700 (PDT)"],"MIME-Version":"1.0","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","In-Reply-To":"<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","From":"Tomasz Figa ","Date":"Wed, 22 Jul 2020 16:31:42 +0200","X-Gmail-Original-Message-ID":"","Message-ID":"","To":"Kieran Bingham ","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11499,"web_url":"https://patchwork.libcamera.org/comment/11499/","msgid":"<20346438-c21a-13c1-0fd9-5721d775c272@ideasonboard.com>","date":"2020-07-22T14:41:15","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Tomasz,\n\nOn 22/07/2020 15:31, Tomasz Figa wrote:\n> On Wed, Jul 22, 2020 at 4:19 PM Kieran Bingham\n> wrote:\n>>\n>> Hi Laurent, Jacopo,\n>>\n>> Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n>> invalid assumption about what is reasonable :-( and a lack of\n>> understanding of how CrOS implemented working around that assumption.\n>>\n>> So my whole premise has been completely wrong and should be ignored.\n>>\n>>\n>> On 22/07/2020 14:49, Laurent Pinchart wrote:\n>>> Hello,\n>>>\n>>> (CC'ing Tomasz)\n>>>\n>>> On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n>>>> On 22/07/2020 12:12, Jacopo Mondi wrote:\n>>>>> Hi Kieran,\n>>>>> I think the conversation diverged, as I was clearly overlooking\n>>>>> the UVC camera use case and you're missing the intended use case of\n>>>>> this patch. I'll try to reply and summarize at the end.\n>>>>\n>>>> My interpretation is that this patch fixes the issue you are\n>>>> experiencing, (which is a valid thing to do, and this is likely a\n>>>> suitable solution for the time being)\n>>>>\n>>>> I was indeed trying to highlight that it would cause breakage for the\n>>>> UVC style use cases, and I believe system degradation for systems with\n>>>> no cameras attached, and which is why I thought a \\todo is required.\n>>>>\n>>>>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n>>>>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n>>>>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n>>>>>>>> Hi Jacopo,\n>>>>>>>>\n>>>>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n>>>>>>>>> Hi Kieran,\n>>>>>>>>>\n>>>>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n>>>>>>>>>> Hi Jacopo,\n>>>>>>>>>>\n>>>>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n>>>>>>>>>>> The CameraHalManager initialization tries to start the\n>>>>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n>>>>>>>>>>>\n>>>>>>>>>>> When initialization is called too early during system boot, it might\n>>>>>>>>>>> happen that the media graphs are still being registered to user-space\n>>>>>>>>>>> preventing any pipeline handler to match and register cameras.\n>>>>>>>>>>>\n>>>>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n>>>>>>>>>>> cameras are available in the system, reporting that information\n>>>>>>>>>>> to the camera stack:\n>>>>>>>>>>>\n>>>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n>>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n>>>>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n>>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> I don't get what your comment is about here, I'm sorry\n>>>>>>>>\n>>>>>>>>\n>>>>>>>> Your patch prevents loading of libcamera completely unless there is at\n>>>>>>>> least one camera right ?\n>>>>>>>>\n>>>>>>>> What happens on say an x86 platform which will only have UVC cameras,\n>>>>>>>> yet will use libcamera...\n>>>>>>>>\n>>>>>>>> When will it be acceptable to load the camera service.\n>>>>>>>>\n>>>>>>>> Should the service itself continually fail, and respawn until someone\n>>>>>>>> plugs in a camera?\n>>>>>>>>\n>>>>>>>\n>>>>>>> That's my understanding\n>>>>>>\n>>>>>> That sounds like that would cause a very bad implementation to me.\n>>>>>>\n>>>>>> Waiting for upstart to decide when to retry launching the camera-service\n>>>>>> wouldn't be an acceptable time to wait IMO.\n>>>>>>\n>>>>>> Either it tries every second, and is flooding the system, or it tries\n>>>>>> every minute and it takes a minute before you can view your newly\n>>>>>> attached UVC camera ...\n>>>>>>\n>>>>>> I would expect event driven camera attachment, not polled ...\n>>>>>\n>>>>> For UVC we will have to register non-active cameras and notify to\n>>>>> android they're available when hotplug is detected\n>>>>\n>>>> Yes, this is what I think I see that the platform2/camera/hal/usb*\n>>>> implementation does in CrOS.\n>>>>\n>>>>> For non-pluggable cameras, as shall wait until the pipeline handler is\n>>>>> matched and has registered cameras\n>>>>\n>>>> The issue for us is that *we support UVC* ... so we're now a UVC capable\n>>>> stack.\n>>>>\n>>>> I believe that means we will in some form or another have to also do the\n>>>> same (some sort of pre-allocation if that's what is needed to make\n>>>> android happy). I dislike the idea that we would then have a 'maximum\n>>>> supported cameras' but maybe it is set arbitrarily high so it doesn't\n>>>> get hit without an extreme use case.\n>>>\n>>> I don't think we can do this, as, as far as I know, Android doesn't\n>>> support status changes for internal cameras. Only cameras reported as\n>>> external can generate a status change event. Quoting from\n>>> camera/provider/2.4/ICameraProviderCallback.hal in\n>>> https://android.googlesource.com/platform/hardware/interfaces:\n>>>\n>>> * On camera service startup, when ICameraProvider::setCallback is invoked,\n>>> * the camera service must assume that all internal camera devices are in\n>>> * the CAMERA_DEVICE_STATUS_PRESENT state.\n>>> *\n>>> * The provider must call this method to inform the camera service of any\n>>> * initially NOT_PRESENT devices, and of any external camera devices that\n>>> * are already present, as soon as the callbacks are available through\n>>> * setCallback.\n>>>\n>>>> Now ... given that we (if we support UVC) must support dynamically\n>>>> adding cameras when they are available - I anticipate that the hotplug\n>>>> work will do that.\n>>>>\n>>>> When we do that, We could also use it for -non hotplug cameras, and\n>>>> simply allow cameras to be registered correctly using the same methods\n>>>> as they are discovered/notified by udev/hotplug mechanisms.\n>>>>\n>>>> Because even if they are fixed CSI2 busses, we're still going to get\n>>>> udev events when they appear as a media-device right ?\n>>>\n>>> Yes, on the libcamera side hotplug should work perfectly fine for\n>>> internal cameras, but unfortunately not on the Android side.\n>>>\n>>>>>>>> So taking that further, what I'm saying is, perhaps this should be\n>>>>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n>>>>>>>> created they should notify android that a new camera is now available.\n>>>>>>>\n>>>>>>> Do you know if that's even possible ?\n>>>>>>>\n>>>>>>> I think Android expects the number of possible cameras to be\n>>>>>>> statically defined, there's a callback to notify a change of status of\n>>>>>>> a device, but we're dealing here with -creating- cameras based on the\n>>>>>>> number of cameras detected by android.\n>>>>>>\n>>>>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n>>>>>> some cameras or such and only activate them on device connection.\n>>>>>>\n>>>>>> I fear we might have to do the same somehow in the future, as indeed it\n>>>>>> might not be possible to tell android there is a 'new' camera. Only an\n>>>>>> update to an existing one... not sure it will require more investigation.\n>>>>>\n>>>>> Yes, but that's for UVC. Built-in cameras are different, and Android\n>>>>> bets on the fact if you have a camera service running then your system\n>>>>> has cameras which are expected to be registered.\n>>>>>\n>>>>> Keep in mind so far there's been an HAL for UVC and one for built-in\n>>>>> cameras. We're handling both and that's a new use case afaict\n>>>>\n>>>> Precisely - and that's what I've been trying to discuss on this patch\n>>>> ;-) I'm sorry my thoughts didn't come across clearly.\n>>>>\n>>>>>>>>>>> Fix this by returning an error code if no camera is registered in the\n>>>>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n>>>>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n>>>>>>>>>>> after the media device dependencies have been registered:\n>>>>>>>>>>>\n>>>>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n>>>>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n>>>>>>>>>>> ....\n>>>>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n>>>>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n>>>>>>>>>>>\n>>>>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n>>>>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n>>>>>>>>>>> between that and other negative values, so it does not really make\n>>>>>>>>>>> a difference.\n>>>>>>>>>>\n>>>>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n>>>>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n>>>>>>>>>> which doesn't yet have any cameras connected).\n>>>>>>>>>\n>>>>>>>>> You keep defferring ? I think the framework handles that\n>>>>>>>>\n>>>>>>>> I would be surprised if it would be expected behaviour to just poll\n>>>>>>>> launching of the camera service until there is a camera there to satisfy\n>>>>>>>> it ...\n>>>\n>>> This is actually how the camera HALs in Chrome OS handle this issue :-(\n>>\n>>\n>> Oh dear :-( Well I guess that invalidates all my assumptions\n>>\n>> /me throws toys out of the pram.\n>>\n>>\n>>>>>>>>>> How many times will we defer? How long does it defer for for instance?\n>>>>>>>>>\n>>>>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n>>>>>>>>> what is the timeout in the framework, as afaict android services are\n>>>>>>>>> restarted by default.\n>>>>>>>>\n>>>>>>>> Ok, so this is mostly dealing with the race at startup ...\n>>>>>>>>\n>>>>>>>>>>> Signed-off-by: Jacopo Mondi \n>>>>>>>>>>> ---\n>>>>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n>>>>>>>>>>> 1 file changed, 11 insertions(+)\n>>>>>>>>>>>\n>>>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n>>>>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n>>>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n>>>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n>>>>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n>>>>>>>>>>> ++index;\n>>>>>>>>>>> }\n>>>>>>>>>>>\n>>>>>>>>>>> + /*\n>>>>>>>>>>> + * If no pipeline has registered cameras, defer initialization to give\n>>>>>>>>>>> + * time to media devices to register to user-space.\n>>>>>>>>>>> + */\n>>>>>>>>>>> + if (index == 0) {\n>>>>>>>>>>> + LOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n>>>>>>>>>>\n>>>>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n>>>>>>>>>> always prints somewhere.\n>>>>>>>>>\n>>>>>>>>> I think that's kind of expected to happen, I don't think this should\n>>>>>>>>> be an error\n>>>>>>>>\n>>>>>>>> Info? Warning?\n>>>>>>>>\n>>>>>>>> If libcamera has prevented loading, I think it would be good to know\n>>>>>>>> about it by default...\n>>>>>>>>\n>>>>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n>>>>>>>>>> this would go through if only one camera of two in the system is available).\n>>>>>>>>>\n>>>>>>>>> If the pipeline handler match (ie the device enumerator matches the\n>>>>>>>>> dependencies) than the match() function continues and register\n>>>>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n>>>>>>>>> state.\n>>>>>>>>\n>>>>>>>> Ok, so it really is about having at least one camera available?\n>>>>>>>\n>>>>>>> It's about at least a pipeline being matched.\n>>>>>>\n>>>>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n>>>>>> there won't be a device\n>>>>>\n>>>>> Not for what Android has been thought to work on. You integrate a\n>>>>> system with cameras, it's fair to defer until those cameras are\n>>>>> registered. You have no cameras, either you disable the camera stack\n>>>>> or you register 0 cameras and then the camera subsystem is silent and\n>>>>> not accessible.\n>>>>\n>>>> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n>>>>\n>>>> I've been annoyed I haven't been able to connect a UVC camera to my\n>>>> phone before ;-)\n>>>>\n>>>>>>>> I think looking at CameraHalManager::init(), as it only creates a\n>>>>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n>>>>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n>>>>>>>> unlikely to get in here between the point that one has been available\n>>>>>>>> and another is still loading ...\n>>>>>>>\n>>>>>>> I don't think that's possible.\n>>>>>>>\n>>>>>>> If a all the dependencies of a pipeline handler are matched, the\n>>>>>>> camera are registered. What is the code flow that registers one\n>>>>>>> camera, returns to the camera manager, then calls the pipeline handler\n>>>>>>> again to register more ?\n>>>>>>\n>>>>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n>>>>>> each camera attached.\n>>>>>>\n>>>>>> But yes, perhaps due to the threading model, it won't be a possible race.\n>>>>>>\n>>>>>> I was thinking of when a pipeline starts and only 'one' of it's media\n>>>>>> devices is available.\n>>>>>\n>>>>> Yes, for UVC that's different I agree. A call to\n>>>>> CameraManager::start() might match one uvc media device but miss a\n>>>>> second one which still have to appear to userspace.\n>>>>>\n>>>>> As said, UVC needs some special handling and pre-register cameras\n>>>>\n>>>> Yes, and we will therefore /have/ to do that.\n>>>\n>>> I also don't see any way around this *for UVC*.\n>>>\n>>>>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n>>>>>>>> being added to the HAL.\n>>>>>>>\n>>>>>>> I agree, we should support inserting and removing new cameras, but I\n>>>>>>> think at this point, we should really start only if cameras are\n>>>>>>> registered and any pipeline handler is matched.\n>>>>>>>\n>>>>>>> I'm not sure I got what other alternative approach you are proposing\n>>>>>>> to be honest.\n>>>>>>\n>>>>>> I'm suggesting that when hotplug support is available, this patch, if\n>>>>>> integrated, might need to be reverted in effect ... and thus a todo note\n>>>>>> should be added to say that or such.\n>>>>>>\n>>>>>> I believe we should be able to start the camera service successfully\n>>>>>> with zero cameras if there are zero cameras at the time the service starts.\n>>>>>\n>>>>> I don't think so for built-in cameras. We start the service when\n>>>>> cameras are registered, otherwise if you have to pre-allocate cameras\n>>>>> you would need to know how many of them the pipeline handler expects\n>>>>> to register and that's not known before it has been matched.\n>>>>\n>>>> Yes, not knowing in advance is a pain point.\n>>>>\n>>>>> UVC, again, is a different story as it supports hotplug.\n>>>>\n>>>> But equally we don't know how many UVC cameras someone could connect,\n>>>> but we can define a max. The question would then be if that 'max'\n>>>> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n>>>> top' of the static cameras.\n>>>>\n>>>> Including it in the max, means we can just preallocate the camera\n>>>> registrations for Android sake, and register the (static) cameras in\n>>>> that list as soon as they are available through the udev notifications\n>>>> system.\n>>>>\n>>>>>>>>>> ..\n>>>>>>>>>>\n>>>>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n>>>>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n>>>>>>>>>\n>>>>>>>>> A \\todo item to record this should be fixed how ?\n>>>>>>>>>\n>>>>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n>>>>>>>>> handler it tries match, but this seems not a good idea to me.\n>>>>>>>>\n>>>>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n>>>>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n>>>>>>>\n>>>>>>> That means Android/CrOS starts without the camera being active (ie.\n>>>>>>> the camera application icon is not available, in CrOS in example).\n>>>>>>\n>>>>>> Hrm, so it disables the app completely?\n>>>>>>\n>>>>>> Anyway, essentially I'm thinking - whatever happens with the existing\n>>>>>> UVC stack is what we would need to be able to mirror.\n>>>>>>\n>>>>>>>> they become available at which point they'll be registered through the\n>>>>>>>> same mechanisms as the hotplugging ...\n>>>>>>>\n>>>>>>> Still I don't get if you are talking about libcamera hotplug or\n>>>>>>> android hotplug, which, to my understanding, expects anyway a camera\n>>>>>>> device to be registered but marked as 'not present' (looking at the\n>>>>>>> camera_device_status enumeration)\n>>>>>>\n>>>>>>\n>>>>>> I was going to ask how does it register camera's it doesn't know will\n>>>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n>>>>>> cameras or such, so that explains how that one works.\n>>>>>>\n>>>>>\n>>>>> I see three cases:\n>>>>> 1) No uvc support\n>>>>> Defer until a pipeline handler is matched and has registered\n>>>>> cameras\n>>>>>\n>>>>> 2) UVC only\n>>>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n>>>>> this has to happen is tricky, as the HAL needs to know it should\n>>>>> not wait for built-in cameras and can proceed to register\n>>>>> non-active USB cameras\n>>>>>\n>>>>> 3) built-in + UVC\n>>>>> Simmilarly, knowing we have to wait for built-in to appear, we\n>>>>> defer until cameras are available, then pre-register UVC cameras.\n>>>>>\n>>>>> The hard part I think it is to define how to instruct the HAL it has\n>>>>> to wait for built-in to appear or not before registering UVC\n>>>>> non-active cameras.\n>>>>\n>>>> My suggestion is (not blocking this patch, but on top) to do so by\n>>>> treating static cameras and hotpluggable cameras in the same way.\n>>>>\n>>>> It's just that static cameras will never 'unplug' (unless someone\n>>>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n>>>\n>>> As explained above, this is unfortunately not an option. To make this\n>>> matter more complicated, we need to wait for *all* built-in cameras to\n>>> be available before initializing, as otherwise the system will be\n>>> permanently stuck with only part of the cameras available.\n>>>\n>>> I think this isn't something we should solve, the system should ensure\n>>> that device nodes have been created before starting the camera service.\n>>\n>> Yes, given what you've clarified, then I agree - the service should be\n>> dependent upon the devices being brought up. I think systemd can do\n>> that, but CrOS uses upstart, so I don't know what that might support.\n> \n> As ugly as it sounds, we have an udev rule which restarts the camera\n> service when devices matching the internal cameras show up. I believe\n> we match them by the \"video4linux\" class and particular hardware\n> subsystems, e.g. PCI or I2C.\n\nThat's actually slightly better than what I had imagined was happening ;-)\n\nI had got the impression there was some arbitrary delay between retries\n... so at least having some event driving the retry is better than a\ntime based interval.\n\n\n>> But that's out of scope (and control) for us I think ...\n>>\n>>\n>>> I've expressed this before in a conversation with Tomasz (not sure if it\n>>> was on the public mailing list, IRC, or in private), but we didn't reach\n>>> an agreement at the time. One option that was discussed was a platform\n>>> configuration file that would list the cameras expected to be present (I\n>>> think everybody knows how much I like that :-)). Another option that has\n>>> been experimented was\n>>> https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.\n>>\n>>\n>> --\n>> Regards\n>> --\n>> Kieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E48D6C2E68\n\tfor ;\n\tWed, 22 Jul 2020 14:41:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6E9206093F;\n\tWed, 22 Jul 2020 16:41:21 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 599166053C\n\tfor ;\n\tWed, 22 Jul 2020 16:41:19 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 84FF5329;\n\tWed, 22 Jul 2020 16:41:18 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"jscUr4gs\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595428878;\n\tbh=J9Cp8A+flxXoZV+pSC4aXVgGQCyc3qLRQxWgaM8HY+o=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=jscUr4gsId32lESTSt0M0h5WgmsrWUNvkdc/EhHAbn2RL9EVjpC+xMETYoNKerH+P\n\tvyWADa44yrEY8+ofKlIJ+DH4VgFoM/DYgNQbozwqj33PTSO+g4oB1br2jURJK7H42P\n\t7DwfhZOY03nfiXtiMLyi23ywe5+RClYC41TZQZLE=","To":"Tomasz Figa ","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<20346438-c21a-13c1-0fd9-5721d775c272@ideasonboard.com>","Date":"Wed, 22 Jul 2020 15:41:15 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11515,"web_url":"https://patchwork.libcamera.org/comment/11515/","msgid":"<20200722175321.GP29813@pendragon.ideasonboard.com>","date":"2020-07-22T17:53:21","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Wed, Jul 22, 2020 at 03:19:28PM +0100, Kieran Bingham wrote:\n> Hi Laurent, Jacopo,\n> \n> Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> invalid assumption about what is reasonable :-( and a lack of\n> understanding of how CrOS implemented working around that assumption.\n> \n> So my whole premise has been completely wrong and should be ignored.\n\nDon't be too harsh on yourself. The devil is in the details here, it's\neasy to overlook some parts, and I'm sure I'm missing info too.\n\n> On 22/07/2020 14:49, Laurent Pinchart wrote:\n> > On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> >> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> >>> Hi Kieran,\n> >>> I think the conversation diverged, as I was clearly overlooking\n> >>> the UVC camera use case and you're missing the intended use case of\n> >>> this patch. I'll try to reply and summarize at the end.\n> >>\n> >> My interpretation is that this patch fixes the issue you are\n> >> experiencing, (which is a valid thing to do, and this is likely a\n> >> suitable solution for the time being)\n> >>\n> >> I was indeed trying to highlight that it would cause breakage for the\n> >> UVC style use cases, and I believe system degradation for systems with\n> >> no cameras attached, and which is why I thought a \\todo is required.\n> >>\n> >>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> >>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> >>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> >>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> >>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>>>>>>>> The CameraHalManager initialization tries to start the\n> >>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>>>>>>>\n> >>>>>>>>> When initialization is called too early during system boot, it might\n> >>>>>>>>> happen that the media graphs are still being registered to user-space\n> >>>>>>>>> preventing any pipeline handler to match and register cameras.\n> >>>>>>>>>\n> >>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n> >>>>>>>>> cameras are available in the system, reporting that information\n> >>>>>>>>> to the camera stack:\n> >>>>>>>>>\n> >>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>>>>>>>\n> >>>>>>>>\n> >>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> >>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>>>>>>>\n> >>>>>>>\n> >>>>>>> I don't get what your comment is about here, I'm sorry\n> >>>>>>\n> >>>>>>\n> >>>>>> Your patch prevents loading of libcamera completely unless there is at\n> >>>>>> least one camera right ?\n> >>>>>>\n> >>>>>> What happens on say an x86 platform which will only have UVC cameras,\n> >>>>>> yet will use libcamera...\n> >>>>>>\n> >>>>>> When will it be acceptable to load the camera service.\n> >>>>>>\n> >>>>>> Should the service itself continually fail, and respawn until someone\n> >>>>>> plugs in a camera?\n> >>>>>\n> >>>>> That's my understanding\n> >>>>\n> >>>> That sounds like that would cause a very bad implementation to me.\n> >>>>\n> >>>> Waiting for upstart to decide when to retry launching the camera-service\n> >>>> wouldn't be an acceptable time to wait IMO.\n> >>>>\n> >>>> Either it tries every second, and is flooding the system, or it tries\n> >>>> every minute and it takes a minute before you can view your newly\n> >>>> attached UVC camera ...\n> >>>>\n> >>>> I would expect event driven camera attachment, not polled ...\n> >>>\n> >>> For UVC we will have to register non-active cameras and notify to\n> >>> android they're available when hotplug is detected\n> >>\n> >> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> >> implementation does in CrOS.\n> >>\n> >>> For non-pluggable cameras, as shall wait until the pipeline handler is\n> >>> matched and has registered cameras\n> >>\n> >> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> >> stack.\n> >>\n> >> I believe that means we will in some form or another have to also do the\n> >> same (some sort of pre-allocation if that's what is needed to make\n> >> android happy). I dislike the idea that we would then have a 'maximum\n> >> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> >> get hit without an extreme use case.\n> > \n> > I don't think we can do this, as, as far as I know, Android doesn't\n> > support status changes for internal cameras. Only cameras reported as\n> > external can generate a status change event. Quoting from \n> > camera/provider/2.4/ICameraProviderCallback.hal in\n> > https://android.googlesource.com/platform/hardware/interfaces:\n> > \n> > * On camera service startup, when ICameraProvider::setCallback is invoked,\n> > * the camera service must assume that all internal camera devices are in\n> > * the CAMERA_DEVICE_STATUS_PRESENT state.\n> > *\n> > * The provider must call this method to inform the camera service of any\n> > * initially NOT_PRESENT devices, and of any external camera devices that\n> > * are already present, as soon as the callbacks are available through\n> > * setCallback.\n> > \n> >> Now ... given that we (if we support UVC) must support dynamically\n> >> adding cameras when they are available - I anticipate that the hotplug\n> >> work will do that.\n> >>\n> >> When we do that, We could also use it for -non hotplug cameras, and\n> >> simply allow cameras to be registered correctly using the same methods\n> >> as they are discovered/notified by udev/hotplug mechanisms.\n> >>\n> >> Because even if they are fixed CSI2 busses, we're still going to get\n> >> udev events when they appear as a media-device right ?\n> > \n> > Yes, on the libcamera side hotplug should work perfectly fine for\n> > internal cameras, but unfortunately not on the Android side.\n> > \n> >>>>>> So taking that further, what I'm saying is, perhaps this should be\n> >>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> >>>>>> created they should notify android that a new camera is now available.\n> >>>>>\n> >>>>> Do you know if that's even possible ?\n> >>>>>\n> >>>>> I think Android expects the number of possible cameras to be\n> >>>>> statically defined, there's a callback to notify a change of status of\n> >>>>> a device, but we're dealing here with -creating- cameras based on the\n> >>>>> number of cameras detected by android.\n> >>>>\n> >>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> >>>> some cameras or such and only activate them on device connection.\n> >>>>\n> >>>> I fear we might have to do the same somehow in the future, as indeed it\n> >>>> might not be possible to tell android there is a 'new' camera. Only an\n> >>>> update to an existing one... not sure it will require more investigation.\n> >>>\n> >>> Yes, but that's for UVC. Built-in cameras are different, and Android\n> >>> bets on the fact if you have a camera service running then your system\n> >>> has cameras which are expected to be registered.\n> >>>\n> >>> Keep in mind so far there's been an HAL for UVC and one for built-in\n> >>> cameras. We're handling both and that's a new use case afaict\n> >>\n> >> Precisely - and that's what I've been trying to discuss on this patch\n> >> ;-) I'm sorry my thoughts didn't come across clearly.\n> >>\n> >>>>>>>>> Fix this by returning an error code if no camera is registered in the\n> >>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n> >>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> >>>>>>>>> after the media device dependencies have been registered:\n> >>>>>>>>>\n> >>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>> ....\n> >>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>>>>>>>\n> >>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> >>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>>>>>>>> between that and other negative values, so it does not really make\n> >>>>>>>>> a difference.\n> >>>>>>>>\n> >>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> >>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >>>>>>>> which doesn't yet have any cameras connected).\n> >>>>>>>\n> >>>>>>> You keep defferring ? I think the framework handles that\n> >>>>>>\n> >>>>>> I would be surprised if it would be expected behaviour to just poll\n> >>>>>> launching of the camera service until there is a camera there to satisfy\n> >>>>>> it ...\n> > \n> > This is actually how the camera HALs in Chrome OS handle this issue :-(\n\nTo be more accurate, it was done this way last time I checked, which\nwasn't very recent. Tomasz has just commented they restart the service\nwhen new devices are found, so maybe there's hope there. I'll reply to\nhis e-mail separately.\n\n> Oh dear :-( Well I guess that invalidates all my assumptions\n> \n> /me throws toys out of the pram.\n\nIs this the step before rolling on the floor throwing a fit ? :-)\n\n> >>>>>>>> How many times will we defer? How long does it defer for for instance?\n> >>>>>>>\n> >>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n> >>>>>>> what is the timeout in the framework, as afaict android services are\n> >>>>>>> restarted by default.\n> >>>>>>\n> >>>>>> Ok, so this is mostly dealing with the race at startup ...\n> >>>>>>\n> >>>>>>>>> Signed-off-by: Jacopo Mondi \n> >>>>>>>>> ---\n> >>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>>>>>>>> 1 file changed, 11 insertions(+)\n> >>>>>>>>>\n> >>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n> >>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>>>>>>>> \t\t++index;\n> >>>>>>>>> \t}\n> >>>>>>>>>\n> >>>>>>>>> +\t/*\n> >>>>>>>>> +\t * If no pipeline has registered cameras, defer initialization to give\n> >>>>>>>>> +\t * time to media devices to register to user-space.\n> >>>>>>>>> +\t */\n> >>>>>>>>> +\tif (index == 0) {\n> >>>>>>>>> +\t\tLOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>>>>>>>\n> >>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> >>>>>>>> always prints somewhere.\n> >>>>>>>\n> >>>>>>> I think that's kind of expected to happen, I don't think this should\n> >>>>>>> be an error\n> >>>>>>\n> >>>>>> Info? Warning?\n> >>>>>>\n> >>>>>> If libcamera has prevented loading, I think it would be good to know\n> >>>>>> about it by default...\n> >>>>>>\n> >>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> >>>>>>>> this would go through if only one camera of two in the system is available).\n> >>>>>>>\n> >>>>>>> If the pipeline handler match (ie the device enumerator matches the\n> >>>>>>> dependencies) than the match() function continues and register\n> >>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> >>>>>>> state.\n> >>>>>>\n> >>>>>> Ok, so it really is about having at least one camera available?\n> >>>>>\n> >>>>> It's about at least a pipeline being matched.\n> >>>>\n> >>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n> >>>> there won't be a device\n> >>>\n> >>> Not for what Android has been thought to work on. You integrate a\n> >>> system with cameras, it's fair to defer until those cameras are\n> >>> registered. You have no cameras, either you disable the camera stack\n> >>> or you register 0 cameras and then the camera subsystem is silent and\n> >>> not accessible.\n> >>\n> >> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> >>\n> >> I've been annoyed I haven't been able to connect a UVC camera to my\n> >> phone before ;-)\n> >>\n> >>>>>> I think looking at CameraHalManager::init(), as it only creates a\n> >>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> >>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> >>>>>> unlikely to get in here between the point that one has been available\n> >>>>>> and another is still loading ...\n> >>>>>\n> >>>>> I don't think that's possible.\n> >>>>>\n> >>>>> If a all the dependencies of a pipeline handler are matched, the\n> >>>>> camera are registered. What is the code flow that registers one\n> >>>>> camera, returns to the camera manager, then calls the pipeline handler\n> >>>>> again to register more ?\n> >>>>\n> >>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> >>>> each camera attached.\n> >>>>\n> >>>> But yes, perhaps due to the threading model, it won't be a possible race.\n> >>>>\n> >>>> I was thinking of when a pipeline starts and only 'one' of it's media\n> >>>> devices is available.\n> >>>\n> >>> Yes, for UVC that's different I agree. A call to\n> >>> CameraManager::start() might match one uvc media device but miss a\n> >>> second one which still have to appear to userspace.\n> >>>\n> >>> As said, UVC needs some special handling and pre-register cameras\n> >>\n> >> Yes, and we will therefore /have/ to do that.\n> > \n> > I also don't see any way around this *for UVC*.\n> > \n> >>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n> >>>>>> being added to the HAL.\n> >>>>>\n> >>>>> I agree, we should support inserting and removing new cameras, but I\n> >>>>> think at this point, we should really start only if cameras are\n> >>>>> registered and any pipeline handler is matched.\n> >>>>>\n> >>>>> I'm not sure I got what other alternative approach you are proposing\n> >>>>> to be honest.\n> >>>>\n> >>>> I'm suggesting that when hotplug support is available, this patch, if\n> >>>> integrated, might need to be reverted in effect ... and thus a todo note\n> >>>> should be added to say that or such.\n> >>>>\n> >>>> I believe we should be able to start the camera service successfully\n> >>>> with zero cameras if there are zero cameras at the time the service starts.\n> >>>\n> >>> I don't think so for built-in cameras. We start the service when\n> >>> cameras are registered, otherwise if you have to pre-allocate cameras\n> >>> you would need to know how many of them the pipeline handler expects\n> >>> to register and that's not known before it has been matched.\n> >>\n> >> Yes, not knowing in advance is a pain point.\n> >>\n> >>> UVC, again, is a different story as it supports hotplug.\n> >>\n> >> But equally we don't know how many UVC cameras someone could connect,\n> >> but we can define a max. The question would then be if that 'max'\n> >> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> >> top' of the static cameras.\n> >>\n> >> Including it in the max, means we can just preallocate the camera\n> >> registrations for Android sake, and register the (static) cameras in\n> >> that list as soon as they are available through the udev notifications\n> >> system.\n> >>\n> >>>>>>>> ..\n> >>>>>>>>\n> >>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> >>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n> >>>>>>>\n> >>>>>>> A \\todo item to record this should be fixed how ?\n> >>>>>>>\n> >>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> >>>>>>> handler it tries match, but this seems not a good idea to me.\n> >>>>>>\n> >>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> >>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n> >>>>>\n> >>>>> That means Android/CrOS starts without the camera being active (ie.\n> >>>>> the camera application icon is not available, in CrOS in example).\n> >>>>\n> >>>> Hrm, so it disables the app completely?\n> >>>>\n> >>>> Anyway, essentially I'm thinking - whatever happens with the existing\n> >>>> UVC stack is what we would need to be able to mirror.\n> >>>>\n> >>>>>> they become available at which point they'll be registered through the\n> >>>>>> same mechanisms as the hotplugging ...\n> >>>>>\n> >>>>> Still I don't get if you are talking about libcamera hotplug or\n> >>>>> android hotplug, which, to my understanding, expects anyway a camera\n> >>>>> device to be registered but marked as 'not present' (looking at the\n> >>>>> camera_device_status enumeration)\n> >>>>\n> >>>> I was going to ask how does it register camera's it doesn't know will\n> >>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> >>>> cameras or such, so that explains how that one works.\n> >>>\n> >>> I see three cases:\n> >>> 1) No uvc support\n> >>> Defer until a pipeline handler is matched and has registered\n> >>> cameras\n> >>>\n> >>> 2) UVC only\n> >>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> >>> this has to happen is tricky, as the HAL needs to know it should\n> >>> not wait for built-in cameras and can proceed to register\n> >>> non-active USB cameras\n> >>>\n> >>> 3) built-in + UVC\n> >>> Simmilarly, knowing we have to wait for built-in to appear, we\n> >>> defer until cameras are available, then pre-register UVC cameras.\n> >>>\n> >>> The hard part I think it is to define how to instruct the HAL it has\n> >>> to wait for built-in to appear or not before registering UVC\n> >>> non-active cameras.\n> >>\n> >> My suggestion is (not blocking this patch, but on top) to do so by\n> >> treating static cameras and hotpluggable cameras in the same way.\n> >>\n> >> It's just that static cameras will never 'unplug' (unless someone\n> >> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> > \n> > As explained above, this is unfortunately not an option. To make this\n> > matter more complicated, we need to wait for *all* built-in cameras to\n> > be available before initializing, as otherwise the system will be\n> > permanently stuck with only part of the cameras available.\n> > \n> > I think this isn't something we should solve, the system should ensure\n> > that device nodes have been created before starting the camera service.\n> \n> Yes, given what you've clarified, then I agree - the service should be\n> dependent upon the devices being brought up. I think systemd can do\n> that, but CrOS uses upstart, so I don't know what that might support.\n> \n> But that's out of scope (and control) for us I think ...\n> \n> > I've expressed this before in a conversation with Tomasz (not sure if it\n> > was on the public mailing list, IRC, or in private), but we didn't reach\n> > an agreement at the time. One option that was discussed was a platform\n> > configuration file that would list the cameras expected to be present (I\n> > think everybody knows how much I like that :-)). Another option that has\n> > been experimented was\n> > https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 48FA9BDB1B\n\tfor ;\n\tWed, 22 Jul 2020 17:53:29 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C28D860C34;\n\tWed, 22 Jul 2020 19:53:28 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 15D9160540\n\tfor ;\n\tWed, 22 Jul 2020 19:53:27 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 68C86329;\n\tWed, 22 Jul 2020 19:53:26 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"DpdzSn2B\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595440406;\n\tbh=VgihDpXeQj3W0tATC5GAg5sKtu2lpE2F8vOCssm4LeU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=DpdzSn2BGxMQl519NQMsJgD3xbezk061ld8jQcyKIYPBBNXKGIf7NrMpzMUzGcOB4\n\tniM6EIhQcynow0c11WrmyBGxccFofryVW54h7J2V0vCBvDUPrDfm9OXU/SoDbc0FVO\n\t8vKMcVK+qRRbb77FBKkXDE5NHKME8P1hR31Kg9nE=","Date":"Wed, 22 Jul 2020 20:53:21 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"<20200722175321.GP29813@pendragon.ideasonboard.com>","References":"<20200721112633.103016-1-jacopo@jmondi.org>\n\t<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Tomasz Figa , libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11516,"web_url":"https://patchwork.libcamera.org/comment/11516/","msgid":"<20200722175630.GQ29813@pendragon.ideasonboard.com>","date":"2020-07-22T17:56:30","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Tomasz,\n\nOn Wed, Jul 22, 2020 at 04:31:42PM +0200, Tomasz Figa wrote:\n> On Wed, Jul 22, 2020 at 4:19 PM Kieran Bingham wrote:\n> >\n> > Hi Laurent, Jacopo,\n> >\n> > Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> > invalid assumption about what is reasonable :-( and a lack of\n> > understanding of how CrOS implemented working around that assumption.\n> >\n> > So my whole premise has been completely wrong and should be ignored.\n> >\n> > On 22/07/2020 14:49, Laurent Pinchart wrote:\n> >> Hello,\n> >>\n> >> (CC'ing Tomasz)\n> >>\n> >> On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> >>> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> >>>> Hi Kieran,\n> >>>> I think the conversation diverged, as I was clearly overlooking\n> >>>> the UVC camera use case and you're missing the intended use case of\n> >>>> this patch. I'll try to reply and summarize at the end.\n> >>>\n> >>> My interpretation is that this patch fixes the issue you are\n> >>> experiencing, (which is a valid thing to do, and this is likely a\n> >>> suitable solution for the time being)\n> >>>\n> >>> I was indeed trying to highlight that it would cause breakage for the\n> >>> UVC style use cases, and I believe system degradation for systems with\n> >>> no cameras attached, and which is why I thought a \\todo is required.\n> >>>\n> >>>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> >>>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> >>>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> >>>>>>> Hi Jacopo,\n> >>>>>>>\n> >>>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> >>>>>>>> Hi Kieran,\n> >>>>>>>>\n> >>>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >>>>>>>>> Hi Jacopo,\n> >>>>>>>>>\n> >>>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>>>>>>>>> The CameraHalManager initialization tries to start the\n> >>>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>>>>>>>>\n> >>>>>>>>>> When initialization is called too early during system boot, it might\n> >>>>>>>>>> happen that the media graphs are still being registered to user-space\n> >>>>>>>>>> preventing any pipeline handler to match and register cameras.\n> >>>>>>>>>>\n> >>>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n> >>>>>>>>>> cameras are available in the system, reporting that information\n> >>>>>>>>>> to the camera stack:\n> >>>>>>>>>>\n> >>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>>>>>>>>\n> >>>>>>>>>\n> >>>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> >>>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>>>>>>>>\n> >>>>>>>>\n> >>>>>>>> I don't get what your comment is about here, I'm sorry\n> >>>>>>>\n> >>>>>>>\n> >>>>>>> Your patch prevents loading of libcamera completely unless there is at\n> >>>>>>> least one camera right ?\n> >>>>>>>\n> >>>>>>> What happens on say an x86 platform which will only have UVC cameras,\n> >>>>>>> yet will use libcamera...\n> >>>>>>>\n> >>>>>>> When will it be acceptable to load the camera service.\n> >>>>>>>\n> >>>>>>> Should the service itself continually fail, and respawn until someone\n> >>>>>>> plugs in a camera?\n> >>>>>>>\n> >>>>>>\n> >>>>>> That's my understanding\n> >>>>>\n> >>>>> That sounds like that would cause a very bad implementation to me.\n> >>>>>\n> >>>>> Waiting for upstart to decide when to retry launching the camera-service\n> >>>>> wouldn't be an acceptable time to wait IMO.\n> >>>>>\n> >>>>> Either it tries every second, and is flooding the system, or it tries\n> >>>>> every minute and it takes a minute before you can view your newly\n> >>>>> attached UVC camera ...\n> >>>>>\n> >>>>> I would expect event driven camera attachment, not polled ...\n> >>>>\n> >>>> For UVC we will have to register non-active cameras and notify to\n> >>>> android they're available when hotplug is detected\n> >>>\n> >>> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> >>> implementation does in CrOS.\n> >>>\n> >>>> For non-pluggable cameras, as shall wait until the pipeline handler is\n> >>>> matched and has registered cameras\n> >>>\n> >>> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> >>> stack.\n> >>>\n> >>> I believe that means we will in some form or another have to also do the\n> >>> same (some sort of pre-allocation if that's what is needed to make\n> >>> android happy). I dislike the idea that we would then have a 'maximum\n> >>> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> >>> get hit without an extreme use case.\n> >>\n> >> I don't think we can do this, as, as far as I know, Android doesn't\n> >> support status changes for internal cameras. Only cameras reported as\n> >> external can generate a status change event. Quoting from\n> >> camera/provider/2.4/ICameraProviderCallback.hal in\n> >> https://android.googlesource.com/platform/hardware/interfaces:\n> >>\n> >> * On camera service startup, when ICameraProvider::setCallback is invoked,\n> >> * the camera service must assume that all internal camera devices are in\n> >> * the CAMERA_DEVICE_STATUS_PRESENT state.\n> >> *\n> >> * The provider must call this method to inform the camera service of any\n> >> * initially NOT_PRESENT devices, and of any external camera devices that\n> >> * are already present, as soon as the callbacks are available through\n> >> * setCallback.\n> >>\n> >>> Now ... given that we (if we support UVC) must support dynamically\n> >>> adding cameras when they are available - I anticipate that the hotplug\n> >>> work will do that.\n> >>>\n> >>> When we do that, We could also use it for -non hotplug cameras, and\n> >>> simply allow cameras to be registered correctly using the same methods\n> >>> as they are discovered/notified by udev/hotplug mechanisms.\n> >>>\n> >>> Because even if they are fixed CSI2 busses, we're still going to get\n> >>> udev events when they appear as a media-device right ?\n> >>\n> >> Yes, on the libcamera side hotplug should work perfectly fine for\n> >> internal cameras, but unfortunately not on the Android side.\n> >>\n> >>>>>>> So taking that further, what I'm saying is, perhaps this should be\n> >>>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> >>>>>>> created they should notify android that a new camera is now available.\n> >>>>>>\n> >>>>>> Do you know if that's even possible ?\n> >>>>>>\n> >>>>>> I think Android expects the number of possible cameras to be\n> >>>>>> statically defined, there's a callback to notify a change of status of\n> >>>>>> a device, but we're dealing here with -creating- cameras based on the\n> >>>>>> number of cameras detected by android.\n> >>>>>\n> >>>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> >>>>> some cameras or such and only activate them on device connection.\n> >>>>>\n> >>>>> I fear we might have to do the same somehow in the future, as indeed it\n> >>>>> might not be possible to tell android there is a 'new' camera. Only an\n> >>>>> update to an existing one... not sure it will require more investigation.\n> >>>>\n> >>>> Yes, but that's for UVC. Built-in cameras are different, and Android\n> >>>> bets on the fact if you have a camera service running then your system\n> >>>> has cameras which are expected to be registered.\n> >>>>\n> >>>> Keep in mind so far there's been an HAL for UVC and one for built-in\n> >>>> cameras. We're handling both and that's a new use case afaict\n> >>>\n> >>> Precisely - and that's what I've been trying to discuss on this patch\n> >>> ;-) I'm sorry my thoughts didn't come across clearly.\n> >>>\n> >>>>>>>>>> Fix this by returning an error code if no camera is registered in the\n> >>>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n> >>>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> >>>>>>>>>> after the media device dependencies have been registered:\n> >>>>>>>>>>\n> >>>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>>> ....\n> >>>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>>>>>>>>\n> >>>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> >>>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>>>>>>>>> between that and other negative values, so it does not really make\n> >>>>>>>>>> a difference.\n> >>>>>>>>>\n> >>>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> >>>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >>>>>>>>> which doesn't yet have any cameras connected).\n> >>>>>>>>\n> >>>>>>>> You keep defferring ? I think the framework handles that\n> >>>>>>>\n> >>>>>>> I would be surprised if it would be expected behaviour to just poll\n> >>>>>>> launching of the camera service until there is a camera there to satisfy\n> >>>>>>> it ...\n> >>\n> >> This is actually how the camera HALs in Chrome OS handle this issue :-(\n> >\n> >\n> > Oh dear :-( Well I guess that invalidates all my assumptions\n> >\n> > /me throws toys out of the pram.\n> >\n> >\n> >>>>>>>>> How many times will we defer? How long does it defer for for instance?\n> >>>>>>>>\n> >>>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n> >>>>>>>> what is the timeout in the framework, as afaict android services are\n> >>>>>>>> restarted by default.\n> >>>>>>>\n> >>>>>>> Ok, so this is mostly dealing with the race at startup ...\n> >>>>>>>\n> >>>>>>>>>> Signed-off-by: Jacopo Mondi \n> >>>>>>>>>> ---\n> >>>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>>>>>>>>> 1 file changed, 11 insertions(+)\n> >>>>>>>>>>\n> >>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n> >>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>>>>>>>>> ++index;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> + /*\n> >>>>>>>>>> + * If no pipeline has registered cameras, defer initialization to give\n> >>>>>>>>>> + * time to media devices to register to user-space.\n> >>>>>>>>>> + */\n> >>>>>>>>>> + if (index == 0) {\n> >>>>>>>>>> + LOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>>>>>>>>\n> >>>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> >>>>>>>>> always prints somewhere.\n> >>>>>>>>\n> >>>>>>>> I think that's kind of expected to happen, I don't think this should\n> >>>>>>>> be an error\n> >>>>>>>\n> >>>>>>> Info? Warning?\n> >>>>>>>\n> >>>>>>> If libcamera has prevented loading, I think it would be good to know\n> >>>>>>> about it by default...\n> >>>>>>>\n> >>>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> >>>>>>>>> this would go through if only one camera of two in the system is available).\n> >>>>>>>>\n> >>>>>>>> If the pipeline handler match (ie the device enumerator matches the\n> >>>>>>>> dependencies) than the match() function continues and register\n> >>>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> >>>>>>>> state.\n> >>>>>>>\n> >>>>>>> Ok, so it really is about having at least one camera available?\n> >>>>>>\n> >>>>>> It's about at least a pipeline being matched.\n> >>>>>\n> >>>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n> >>>>> there won't be a device\n> >>>>\n> >>>> Not for what Android has been thought to work on. You integrate a\n> >>>> system with cameras, it's fair to defer until those cameras are\n> >>>> registered. You have no cameras, either you disable the camera stack\n> >>>> or you register 0 cameras and then the camera subsystem is silent and\n> >>>> not accessible.\n> >>>\n> >>> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> >>>\n> >>> I've been annoyed I haven't been able to connect a UVC camera to my\n> >>> phone before ;-)\n> >>>\n> >>>>>>> I think looking at CameraHalManager::init(), as it only creates a\n> >>>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> >>>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> >>>>>>> unlikely to get in here between the point that one has been available\n> >>>>>>> and another is still loading ...\n> >>>>>>\n> >>>>>> I don't think that's possible.\n> >>>>>>\n> >>>>>> If a all the dependencies of a pipeline handler are matched, the\n> >>>>>> camera are registered. What is the code flow that registers one\n> >>>>>> camera, returns to the camera manager, then calls the pipeline handler\n> >>>>>> again to register more ?\n> >>>>>\n> >>>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> >>>>> each camera attached.\n> >>>>>\n> >>>>> But yes, perhaps due to the threading model, it won't be a possible race.\n> >>>>>\n> >>>>> I was thinking of when a pipeline starts and only 'one' of it's media\n> >>>>> devices is available.\n> >>>>\n> >>>> Yes, for UVC that's different I agree. A call to\n> >>>> CameraManager::start() might match one uvc media device but miss a\n> >>>> second one which still have to appear to userspace.\n> >>>>\n> >>>> As said, UVC needs some special handling and pre-register cameras\n> >>>\n> >>> Yes, and we will therefore /have/ to do that.\n> >>\n> >> I also don't see any way around this *for UVC*.\n> >>\n> >>>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n> >>>>>>> being added to the HAL.\n> >>>>>>\n> >>>>>> I agree, we should support inserting and removing new cameras, but I\n> >>>>>> think at this point, we should really start only if cameras are\n> >>>>>> registered and any pipeline handler is matched.\n> >>>>>>\n> >>>>>> I'm not sure I got what other alternative approach you are proposing\n> >>>>>> to be honest.\n> >>>>>\n> >>>>> I'm suggesting that when hotplug support is available, this patch, if\n> >>>>> integrated, might need to be reverted in effect ... and thus a todo note\n> >>>>> should be added to say that or such.\n> >>>>>\n> >>>>> I believe we should be able to start the camera service successfully\n> >>>>> with zero cameras if there are zero cameras at the time the service starts.\n> >>>>\n> >>>> I don't think so for built-in cameras. We start the service when\n> >>>> cameras are registered, otherwise if you have to pre-allocate cameras\n> >>>> you would need to know how many of them the pipeline handler expects\n> >>>> to register and that's not known before it has been matched.\n> >>>\n> >>> Yes, not knowing in advance is a pain point.\n> >>>\n> >>>> UVC, again, is a different story as it supports hotplug.\n> >>>\n> >>> But equally we don't know how many UVC cameras someone could connect,\n> >>> but we can define a max. The question would then be if that 'max'\n> >>> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> >>> top' of the static cameras.\n> >>>\n> >>> Including it in the max, means we can just preallocate the camera\n> >>> registrations for Android sake, and register the (static) cameras in\n> >>> that list as soon as they are available through the udev notifications\n> >>> system.\n> >>>\n> >>>>>>>>> ..\n> >>>>>>>>>\n> >>>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> >>>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n> >>>>>>>>\n> >>>>>>>> A \\todo item to record this should be fixed how ?\n> >>>>>>>>\n> >>>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> >>>>>>>> handler it tries match, but this seems not a good idea to me.\n> >>>>>>>\n> >>>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> >>>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n> >>>>>>\n> >>>>>> That means Android/CrOS starts without the camera being active (ie.\n> >>>>>> the camera application icon is not available, in CrOS in example).\n> >>>>>\n> >>>>> Hrm, so it disables the app completely?\n> >>>>>\n> >>>>> Anyway, essentially I'm thinking - whatever happens with the existing\n> >>>>> UVC stack is what we would need to be able to mirror.\n> >>>>>\n> >>>>>>> they become available at which point they'll be registered through the\n> >>>>>>> same mechanisms as the hotplugging ...\n> >>>>>>\n> >>>>>> Still I don't get if you are talking about libcamera hotplug or\n> >>>>>> android hotplug, which, to my understanding, expects anyway a camera\n> >>>>>> device to be registered but marked as 'not present' (looking at the\n> >>>>>> camera_device_status enumeration)\n> >>>>>\n> >>>>>\n> >>>>> I was going to ask how does it register camera's it doesn't know will\n> >>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> >>>>> cameras or such, so that explains how that one works.\n> >>>>>\n> >>>>\n> >>>> I see three cases:\n> >>>> 1) No uvc support\n> >>>> Defer until a pipeline handler is matched and has registered\n> >>>> cameras\n> >>>>\n> >>>> 2) UVC only\n> >>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> >>>> this has to happen is tricky, as the HAL needs to know it should\n> >>>> not wait for built-in cameras and can proceed to register\n> >>>> non-active USB cameras\n> >>>>\n> >>>> 3) built-in + UVC\n> >>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> >>>> defer until cameras are available, then pre-register UVC cameras.\n> >>>>\n> >>>> The hard part I think it is to define how to instruct the HAL it has\n> >>>> to wait for built-in to appear or not before registering UVC\n> >>>> non-active cameras.\n> >>>\n> >>> My suggestion is (not blocking this patch, but on top) to do so by\n> >>> treating static cameras and hotpluggable cameras in the same way.\n> >>>\n> >>> It's just that static cameras will never 'unplug' (unless someone\n> >>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> >>\n> >> As explained above, this is unfortunately not an option. To make this\n> >> matter more complicated, we need to wait for *all* built-in cameras to\n> >> be available before initializing, as otherwise the system will be\n> >> permanently stuck with only part of the cameras available.\n> >>\n> >> I think this isn't something we should solve, the system should ensure\n> >> that device nodes have been created before starting the camera service.\n> >\n> > Yes, given what you've clarified, then I agree - the service should be\n> > dependent upon the devices being brought up. I think systemd can do\n> > that, but CrOS uses upstart, so I don't know what that might support.\n> \n> As ugly as it sounds, we have an udev rule which restarts the camera\n> service when devices matching the internal cameras show up. I believe\n> we match them by the \"video4linux\" class and particular hardware\n> subsystems, e.g. PCI or I2C.\n\nHas it always worked like that, or is it fairly recent ? If the service\nis restarted when new devices appear, everything should be fine even if\nthe HAL initializes successfully with zero cameras, shouldn't it ?\n\n> > But that's out of scope (and control) for us I think ...\n> >\n> >> I've expressed this before in a conversation with Tomasz (not sure if it\n> >> was on the public mailing list, IRC, or in private), but we didn't reach\n> >> an agreement at the time. One option that was discussed was a platform\n> >> configuration file that would list the cameras expected to be present (I\n> >> think everybody knows how much I like that :-)). Another option that has\n> >> been experimented was\n> >> https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5771AC2E68\n\tfor ;\n\tWed, 22 Jul 2020 17:56:38 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D27D16099C;\n\tWed, 22 Jul 2020 19:56:37 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 921D460540\n\tfor ;\n\tWed, 22 Jul 2020 19:56:36 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id F04E3329;\n\tWed, 22 Jul 2020 19:56:35 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"dZVE+qr8\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595440596;\n\tbh=U2asxiTkqE+e2OtGlK57/m6yxqkLs4wBM6T+dPXumwg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=dZVE+qr8pRuVYTwN2iy1KzSKylF/eX+C5o+BvbgihskxgkwSOHPXtL5E6YfPF1OyS\n\tZPbzvH/5MM4xqF80iVcLyvqWsA5rYWcas7YBR3spOzBhhHDlJUcG1XCdRC0q9281iy\n\tx7Ktu0wZA1QF+d/zS9LVhFsBRcl4Xpm9bPKWXNw8=","Date":"Wed, 22 Jul 2020 20:56:30 +0300","From":"Laurent Pinchart ","To":"Tomasz Figa ","Message-ID":"<20200722175630.GQ29813@pendragon.ideasonboard.com>","References":"<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11518,"web_url":"https://patchwork.libcamera.org/comment/11518/","msgid":"","date":"2020-07-22T18:10:03","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":9,"url":"https://patchwork.libcamera.org/api/people/9/","name":"Tomasz Figa","email":"tfiga@chromium.org"},"content":"On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart\n wrote:\n>\n> Hi Tomasz,\n>\n> On Wed, Jul 22, 2020 at 04:31:42PM +0200, Tomasz Figa wrote:\n> > On Wed, Jul 22, 2020 at 4:19 PM Kieran Bingham wrote:\n> > >\n> > > Hi Laurent, Jacopo,\n> > >\n> > > Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> > > invalid assumption about what is reasonable :-( and a lack of\n> > > understanding of how CrOS implemented working around that assumption.\n> > >\n> > > So my whole premise has been completely wrong and should be ignored.\n> > >\n> > > On 22/07/2020 14:49, Laurent Pinchart wrote:\n> > >> Hello,\n> > >>\n> > >> (CC'ing Tomasz)\n> > >>\n> > >> On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> > >>> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> > >>>> Hi Kieran,\n> > >>>> I think the conversation diverged, as I was clearly overlooking\n> > >>>> the UVC camera use case and you're missing the intended use case of\n> > >>>> this patch. I'll try to reply and summarize at the end.\n> > >>>\n> > >>> My interpretation is that this patch fixes the issue you are\n> > >>> experiencing, (which is a valid thing to do, and this is likely a\n> > >>> suitable solution for the time being)\n> > >>>\n> > >>> I was indeed trying to highlight that it would cause breakage for the\n> > >>> UVC style use cases, and I believe system degradation for systems with\n> > >>> no cameras attached, and which is why I thought a \\todo is required.\n> > >>>\n> > >>>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> > >>>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> > >>>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> > >>>>>>> Hi Jacopo,\n> > >>>>>>>\n> > >>>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> > >>>>>>>> Hi Kieran,\n> > >>>>>>>>\n> > >>>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> > >>>>>>>>> Hi Jacopo,\n> > >>>>>>>>>\n> > >>>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> > >>>>>>>>>> The CameraHalManager initialization tries to start the\n> > >>>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> > >>>>>>>>>>\n> > >>>>>>>>>> When initialization is called too early during system boot, it might\n> > >>>>>>>>>> happen that the media graphs are still being registered to user-space\n> > >>>>>>>>>> preventing any pipeline handler to match and register cameras.\n> > >>>>>>>>>>\n> > >>>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n> > >>>>>>>>>> cameras are available in the system, reporting that information\n> > >>>>>>>>>> to the camera stack:\n> > >>>>>>>>>>\n> > >>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> > >>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> > >>>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> > >>>>>>>>>>\n> > >>>>>>>>>\n> > >>>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> > >>>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> > >>>>>>>>>\n> > >>>>>>>>\n> > >>>>>>>> I don't get what your comment is about here, I'm sorry\n> > >>>>>>>\n> > >>>>>>>\n> > >>>>>>> Your patch prevents loading of libcamera completely unless there is at\n> > >>>>>>> least one camera right ?\n> > >>>>>>>\n> > >>>>>>> What happens on say an x86 platform which will only have UVC cameras,\n> > >>>>>>> yet will use libcamera...\n> > >>>>>>>\n> > >>>>>>> When will it be acceptable to load the camera service.\n> > >>>>>>>\n> > >>>>>>> Should the service itself continually fail, and respawn until someone\n> > >>>>>>> plugs in a camera?\n> > >>>>>>>\n> > >>>>>>\n> > >>>>>> That's my understanding\n> > >>>>>\n> > >>>>> That sounds like that would cause a very bad implementation to me.\n> > >>>>>\n> > >>>>> Waiting for upstart to decide when to retry launching the camera-service\n> > >>>>> wouldn't be an acceptable time to wait IMO.\n> > >>>>>\n> > >>>>> Either it tries every second, and is flooding the system, or it tries\n> > >>>>> every minute and it takes a minute before you can view your newly\n> > >>>>> attached UVC camera ...\n> > >>>>>\n> > >>>>> I would expect event driven camera attachment, not polled ...\n> > >>>>\n> > >>>> For UVC we will have to register non-active cameras and notify to\n> > >>>> android they're available when hotplug is detected\n> > >>>\n> > >>> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> > >>> implementation does in CrOS.\n> > >>>\n> > >>>> For non-pluggable cameras, as shall wait until the pipeline handler is\n> > >>>> matched and has registered cameras\n> > >>>\n> > >>> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> > >>> stack.\n> > >>>\n> > >>> I believe that means we will in some form or another have to also do the\n> > >>> same (some sort of pre-allocation if that's what is needed to make\n> > >>> android happy). I dislike the idea that we would then have a 'maximum\n> > >>> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> > >>> get hit without an extreme use case.\n> > >>\n> > >> I don't think we can do this, as, as far as I know, Android doesn't\n> > >> support status changes for internal cameras. Only cameras reported as\n> > >> external can generate a status change event. Quoting from\n> > >> camera/provider/2.4/ICameraProviderCallback.hal in\n> > >> https://android.googlesource.com/platform/hardware/interfaces:\n> > >>\n> > >> * On camera service startup, when ICameraProvider::setCallback is invoked,\n> > >> * the camera service must assume that all internal camera devices are in\n> > >> * the CAMERA_DEVICE_STATUS_PRESENT state.\n> > >> *\n> > >> * The provider must call this method to inform the camera service of any\n> > >> * initially NOT_PRESENT devices, and of any external camera devices that\n> > >> * are already present, as soon as the callbacks are available through\n> > >> * setCallback.\n> > >>\n> > >>> Now ... given that we (if we support UVC) must support dynamically\n> > >>> adding cameras when they are available - I anticipate that the hotplug\n> > >>> work will do that.\n> > >>>\n> > >>> When we do that, We could also use it for -non hotplug cameras, and\n> > >>> simply allow cameras to be registered correctly using the same methods\n> > >>> as they are discovered/notified by udev/hotplug mechanisms.\n> > >>>\n> > >>> Because even if they are fixed CSI2 busses, we're still going to get\n> > >>> udev events when they appear as a media-device right ?\n> > >>\n> > >> Yes, on the libcamera side hotplug should work perfectly fine for\n> > >> internal cameras, but unfortunately not on the Android side.\n> > >>\n> > >>>>>>> So taking that further, what I'm saying is, perhaps this should be\n> > >>>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> > >>>>>>> created they should notify android that a new camera is now available.\n> > >>>>>>\n> > >>>>>> Do you know if that's even possible ?\n> > >>>>>>\n> > >>>>>> I think Android expects the number of possible cameras to be\n> > >>>>>> statically defined, there's a callback to notify a change of status of\n> > >>>>>> a device, but we're dealing here with -creating- cameras based on the\n> > >>>>>> number of cameras detected by android.\n> > >>>>>\n> > >>>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> > >>>>> some cameras or such and only activate them on device connection.\n> > >>>>>\n> > >>>>> I fear we might have to do the same somehow in the future, as indeed it\n> > >>>>> might not be possible to tell android there is a 'new' camera. Only an\n> > >>>>> update to an existing one... not sure it will require more investigation.\n> > >>>>\n> > >>>> Yes, but that's for UVC. Built-in cameras are different, and Android\n> > >>>> bets on the fact if you have a camera service running then your system\n> > >>>> has cameras which are expected to be registered.\n> > >>>>\n> > >>>> Keep in mind so far there's been an HAL for UVC and one for built-in\n> > >>>> cameras. We're handling both and that's a new use case afaict\n> > >>>\n> > >>> Precisely - and that's what I've been trying to discuss on this patch\n> > >>> ;-) I'm sorry my thoughts didn't come across clearly.\n> > >>>\n> > >>>>>>>>>> Fix this by returning an error code if no camera is registered in the\n> > >>>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n> > >>>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> > >>>>>>>>>> after the media device dependencies have been registered:\n> > >>>>>>>>>>\n> > >>>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> > >>>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> > >>>>>>>>>> ....\n> > >>>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> > >>>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> > >>>>>>>>>>\n> > >>>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> > >>>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> > >>>>>>>>>> between that and other negative values, so it does not really make\n> > >>>>>>>>>> a difference.\n> > >>>>>>>>>\n> > >>>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> > >>>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> > >>>>>>>>> which doesn't yet have any cameras connected).\n> > >>>>>>>>\n> > >>>>>>>> You keep defferring ? I think the framework handles that\n> > >>>>>>>\n> > >>>>>>> I would be surprised if it would be expected behaviour to just poll\n> > >>>>>>> launching of the camera service until there is a camera there to satisfy\n> > >>>>>>> it ...\n> > >>\n> > >> This is actually how the camera HALs in Chrome OS handle this issue :-(\n> > >\n> > >\n> > > Oh dear :-( Well I guess that invalidates all my assumptions\n> > >\n> > > /me throws toys out of the pram.\n> > >\n> > >\n> > >>>>>>>>> How many times will we defer? How long does it defer for for instance?\n> > >>>>>>>>\n> > >>>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n> > >>>>>>>> what is the timeout in the framework, as afaict android services are\n> > >>>>>>>> restarted by default.\n> > >>>>>>>\n> > >>>>>>> Ok, so this is mostly dealing with the race at startup ...\n> > >>>>>>>\n> > >>>>>>>>>> Signed-off-by: Jacopo Mondi \n> > >>>>>>>>>> ---\n> > >>>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> > >>>>>>>>>> 1 file changed, 11 insertions(+)\n> > >>>>>>>>>>\n> > >>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n> > >>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> > >>>>>>>>>> ++index;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> + /*\n> > >>>>>>>>>> + * If no pipeline has registered cameras, defer initialization to give\n> > >>>>>>>>>> + * time to media devices to register to user-space.\n> > >>>>>>>>>> + */\n> > >>>>>>>>>> + if (index == 0) {\n> > >>>>>>>>>> + LOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> > >>>>>>>>>\n> > >>>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> > >>>>>>>>> always prints somewhere.\n> > >>>>>>>>\n> > >>>>>>>> I think that's kind of expected to happen, I don't think this should\n> > >>>>>>>> be an error\n> > >>>>>>>\n> > >>>>>>> Info? Warning?\n> > >>>>>>>\n> > >>>>>>> If libcamera has prevented loading, I think it would be good to know\n> > >>>>>>> about it by default...\n> > >>>>>>>\n> > >>>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> > >>>>>>>>> this would go through if only one camera of two in the system is available).\n> > >>>>>>>>\n> > >>>>>>>> If the pipeline handler match (ie the device enumerator matches the\n> > >>>>>>>> dependencies) than the match() function continues and register\n> > >>>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> > >>>>>>>> state.\n> > >>>>>>>\n> > >>>>>>> Ok, so it really is about having at least one camera available?\n> > >>>>>>\n> > >>>>>> It's about at least a pipeline being matched.\n> > >>>>>\n> > >>>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n> > >>>>> there won't be a device\n> > >>>>\n> > >>>> Not for what Android has been thought to work on. You integrate a\n> > >>>> system with cameras, it's fair to defer until those cameras are\n> > >>>> registered. You have no cameras, either you disable the camera stack\n> > >>>> or you register 0 cameras and then the camera subsystem is silent and\n> > >>>> not accessible.\n> > >>>\n> > >>> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> > >>>\n> > >>> I've been annoyed I haven't been able to connect a UVC camera to my\n> > >>> phone before ;-)\n> > >>>\n> > >>>>>>> I think looking at CameraHalManager::init(), as it only creates a\n> > >>>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> > >>>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> > >>>>>>> unlikely to get in here between the point that one has been available\n> > >>>>>>> and another is still loading ...\n> > >>>>>>\n> > >>>>>> I don't think that's possible.\n> > >>>>>>\n> > >>>>>> If a all the dependencies of a pipeline handler are matched, the\n> > >>>>>> camera are registered. What is the code flow that registers one\n> > >>>>>> camera, returns to the camera manager, then calls the pipeline handler\n> > >>>>>> again to register more ?\n> > >>>>>\n> > >>>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> > >>>>> each camera attached.\n> > >>>>>\n> > >>>>> But yes, perhaps due to the threading model, it won't be a possible race.\n> > >>>>>\n> > >>>>> I was thinking of when a pipeline starts and only 'one' of it's media\n> > >>>>> devices is available.\n> > >>>>\n> > >>>> Yes, for UVC that's different I agree. A call to\n> > >>>> CameraManager::start() might match one uvc media device but miss a\n> > >>>> second one which still have to appear to userspace.\n> > >>>>\n> > >>>> As said, UVC needs some special handling and pre-register cameras\n> > >>>\n> > >>> Yes, and we will therefore /have/ to do that.\n> > >>\n> > >> I also don't see any way around this *for UVC*.\n> > >>\n> > >>>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n> > >>>>>>> being added to the HAL.\n> > >>>>>>\n> > >>>>>> I agree, we should support inserting and removing new cameras, but I\n> > >>>>>> think at this point, we should really start only if cameras are\n> > >>>>>> registered and any pipeline handler is matched.\n> > >>>>>>\n> > >>>>>> I'm not sure I got what other alternative approach you are proposing\n> > >>>>>> to be honest.\n> > >>>>>\n> > >>>>> I'm suggesting that when hotplug support is available, this patch, if\n> > >>>>> integrated, might need to be reverted in effect ... and thus a todo note\n> > >>>>> should be added to say that or such.\n> > >>>>>\n> > >>>>> I believe we should be able to start the camera service successfully\n> > >>>>> with zero cameras if there are zero cameras at the time the service starts.\n> > >>>>\n> > >>>> I don't think so for built-in cameras. We start the service when\n> > >>>> cameras are registered, otherwise if you have to pre-allocate cameras\n> > >>>> you would need to know how many of them the pipeline handler expects\n> > >>>> to register and that's not known before it has been matched.\n> > >>>\n> > >>> Yes, not knowing in advance is a pain point.\n> > >>>\n> > >>>> UVC, again, is a different story as it supports hotplug.\n> > >>>\n> > >>> But equally we don't know how many UVC cameras someone could connect,\n> > >>> but we can define a max. The question would then be if that 'max'\n> > >>> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> > >>> top' of the static cameras.\n> > >>>\n> > >>> Including it in the max, means we can just preallocate the camera\n> > >>> registrations for Android sake, and register the (static) cameras in\n> > >>> that list as soon as they are available through the udev notifications\n> > >>> system.\n> > >>>\n> > >>>>>>>>> ..\n> > >>>>>>>>>\n> > >>>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> > >>>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n> > >>>>>>>>\n> > >>>>>>>> A \\todo item to record this should be fixed how ?\n> > >>>>>>>>\n> > >>>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> > >>>>>>>> handler it tries match, but this seems not a good idea to me.\n> > >>>>>>>\n> > >>>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> > >>>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n> > >>>>>>\n> > >>>>>> That means Android/CrOS starts without the camera being active (ie.\n> > >>>>>> the camera application icon is not available, in CrOS in example).\n> > >>>>>\n> > >>>>> Hrm, so it disables the app completely?\n> > >>>>>\n> > >>>>> Anyway, essentially I'm thinking - whatever happens with the existing\n> > >>>>> UVC stack is what we would need to be able to mirror.\n> > >>>>>\n> > >>>>>>> they become available at which point they'll be registered through the\n> > >>>>>>> same mechanisms as the hotplugging ...\n> > >>>>>>\n> > >>>>>> Still I don't get if you are talking about libcamera hotplug or\n> > >>>>>> android hotplug, which, to my understanding, expects anyway a camera\n> > >>>>>> device to be registered but marked as 'not present' (looking at the\n> > >>>>>> camera_device_status enumeration)\n> > >>>>>\n> > >>>>>\n> > >>>>> I was going to ask how does it register camera's it doesn't know will\n> > >>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> > >>>>> cameras or such, so that explains how that one works.\n> > >>>>>\n> > >>>>\n> > >>>> I see three cases:\n> > >>>> 1) No uvc support\n> > >>>> Defer until a pipeline handler is matched and has registered\n> > >>>> cameras\n> > >>>>\n> > >>>> 2) UVC only\n> > >>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> > >>>> this has to happen is tricky, as the HAL needs to know it should\n> > >>>> not wait for built-in cameras and can proceed to register\n> > >>>> non-active USB cameras\n> > >>>>\n> > >>>> 3) built-in + UVC\n> > >>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> > >>>> defer until cameras are available, then pre-register UVC cameras.\n> > >>>>\n> > >>>> The hard part I think it is to define how to instruct the HAL it has\n> > >>>> to wait for built-in to appear or not before registering UVC\n> > >>>> non-active cameras.\n> > >>>\n> > >>> My suggestion is (not blocking this patch, but on top) to do so by\n> > >>> treating static cameras and hotpluggable cameras in the same way.\n> > >>>\n> > >>> It's just that static cameras will never 'unplug' (unless someone\n> > >>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> > >>\n> > >> As explained above, this is unfortunately not an option. To make this\n> > >> matter more complicated, we need to wait for *all* built-in cameras to\n> > >> be available before initializing, as otherwise the system will be\n> > >> permanently stuck with only part of the cameras available.\n> > >>\n> > >> I think this isn't something we should solve, the system should ensure\n> > >> that device nodes have been created before starting the camera service.\n> > >\n> > > Yes, given what you've clarified, then I agree - the service should be\n> > > dependent upon the devices being brought up. I think systemd can do\n> > > that, but CrOS uses upstart, so I don't know what that might support.\n> >\n> > As ugly as it sounds, we have an udev rule which restarts the camera\n> > service when devices matching the internal cameras show up. I believe\n> > we match them by the \"video4linux\" class and particular hardware\n> > subsystems, e.g. PCI or I2C.\n>\n> Has it always worked like that, or is it fairly recent ? If the service\n> is restarted when new devices appear, everything should be fine even if\n> the HAL initializes successfully with zero cameras, shouldn't it ?\n>\n\nYes, it's a recent change for a new platform where we ended up with\ndynamic detection of components. Actually it hasn't landed yet. The CL\nfor reference is\nhttps://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n\n> > > But that's out of scope (and control) for us I think ...\n> > >\n> > >> I've expressed this before in a conversation with Tomasz (not sure if it\n> > >> was on the public mailing list, IRC, or in private), but we didn't reach\n> > >> an agreement at the time. One option that was discussed was a platform\n> > >> configuration file that would list the cameras expected to be present (I\n> > >> think everybody knows how much I like that :-)). Another option that has\n> > >> been experimented was\n> > >> https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.\n>\n> --\n> Regards,\n>\n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 064F9BDB1B\n\tfor ;\n\tWed, 22 Jul 2020 18:10:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 812E46099C;\n\tWed, 22 Jul 2020 20:10:20 +0200 (CEST)","from mail-ed1-x541.google.com (mail-ed1-x541.google.com\n\t[IPv6:2a00:1450:4864:20::541])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D36C60540\n\tfor ;\n\tWed, 22 Jul 2020 20:10:19 +0200 (CEST)","by mail-ed1-x541.google.com with SMTP id dg28so2360364edb.3\n\tfor ;\n\tWed, 22 Jul 2020 11:10:19 -0700 (PDT)","from mail-wr1-f43.google.com (mail-wr1-f43.google.com.\n\t[209.85.221.43])\n\tby smtp.gmail.com with ESMTPSA id x4sm277824eju.2.2020.07.22.11.10.16\n\tfor \n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tWed, 22 Jul 2020 11:10:16 -0700 (PDT)","by mail-wr1-f43.google.com with SMTP id a14so2798796wra.5\n\tfor ;\n\tWed, 22 Jul 2020 11:10:16 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"AgMtZVmY\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=PJAcr8JqYDtjfyNpr80L26VputMXq+agNzN1Y5kTETg=;\n\tb=AgMtZVmYXWnD1TO+jBOFu9AsglRFzE5pnZCBvSEuf4yOY9pknGbD5BLAVWf3p/7K73\n\tAtp28bTGFnYCqO8ydeHgnuo45kEyIa82sbNJCXkqD9PsYSaGjBvF/bCXIQMIAwB3mlo+\n\tebNp5zohNYTxVREy2P8W9qss/n5yixigRKvsA=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=PJAcr8JqYDtjfyNpr80L26VputMXq+agNzN1Y5kTETg=;\n\tb=PnyET/AYuYLLV/ot/nasgCO6tAo8CzDvPGkZh9/hWKrrYtnCn9DEDQ6KbAlVEYtYB9\n\tATWTF2tQ0dxZiPUorNkc06guV0M+karOi4nq7j5FYdW1iJ1vrIvqtJ+z6U6d9lq4/Zpf\n\thL8UEwLuZgKczwWaXcxxPhYuw3lZbF0DGsTnPmynU3S7NckvfFQmuz7p+wuON9uhNUts\n\tTF9pdiTk9E/FRcvU2eMn5JJdj4N6cUMKjZtT2L4vTxpEH47oCLqpOhI1Eok8BElmsiTY\n\t/4RDe+3DjoO8u5SGO10bSUhzrfd7idPNTF+A0L/MonrS7EA5wPFCs9azl4Jzp8z47mD7\n\tqibg==","X-Gm-Message-State":"AOAM532Yknllm+NjoM76dJCA5HOf6NFOYhLETxzWOGv3oJHYk8FMhE2k\n\t5OTJT7EjP0UQcecvIHYb3wo4vXplSmvoYA==","X-Google-Smtp-Source":"ABdhPJxoTp/0p9tQmM9cXaFfjVf32YRf4b4XcWwk9/ZDPXnlI/df/r2ybPb4gbZ9BEJt3SYajXBIzQ==","X-Received":["by 2002:aa7:dd10:: with SMTP id i16mr663090edv.227.1595441418146;\n\tWed, 22 Jul 2020 11:10:18 -0700 (PDT)","by 2002:adf:f008:: with SMTP id j8mr613795wro.385.1595441415635; \n\tWed, 22 Jul 2020 11:10:15 -0700 (PDT)"],"MIME-Version":"1.0","References":"<9154b7dc-c896-4e86-99ee-48f25ada72be@ideasonboard.com>\n\t<20200721121350.llcc7b2phigc647e@uno.localdomain>\n\t<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>","In-Reply-To":"<20200722175630.GQ29813@pendragon.ideasonboard.com>","From":"Tomasz Figa ","Date":"Wed, 22 Jul 2020 20:10:03 +0200","X-Gmail-Original-Message-ID":"","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11520,"web_url":"https://patchwork.libcamera.org/comment/11520/","msgid":"<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>","date":"2020-07-23T09:46:54","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Tomasz,\n\nOn Wed, Jul 22, 2020 at 08:10:03PM +0200, Tomasz Figa wrote:\n> On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart\n> wrote:\n> >\n> > Hi Tomasz,\n> >\n\nsnip\n\n> > > >>>>> I was going to ask how does it register camera's it doesn't know will\n> > > >>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> > > >>>>> cameras or such, so that explains how that one works.\n> > > >>>>>\n> > > >>>>\n> > > >>>> I see three cases:\n> > > >>>> 1) No uvc support\n> > > >>>> Defer until a pipeline handler is matched and has registered\n> > > >>>> cameras\n> > > >>>>\n> > > >>>> 2) UVC only\n> > > >>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> > > >>>> this has to happen is tricky, as the HAL needs to know it should\n> > > >>>> not wait for built-in cameras and can proceed to register\n> > > >>>> non-active USB cameras\n> > > >>>>\n> > > >>>> 3) built-in + UVC\n> > > >>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> > > >>>> defer until cameras are available, then pre-register UVC cameras.\n> > > >>>>\n> > > >>>> The hard part I think it is to define how to instruct the HAL it has\n> > > >>>> to wait for built-in to appear or not before registering UVC\n> > > >>>> non-active cameras.\n> > > >>>\n> > > >>> My suggestion is (not blocking this patch, but on top) to do so by\n> > > >>> treating static cameras and hotpluggable cameras in the same way.\n> > > >>>\n> > > >>> It's just that static cameras will never 'unplug' (unless someone\n> > > >>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> > > >>\n> > > >> As explained above, this is unfortunately not an option. To make this\n> > > >> matter more complicated, we need to wait for *all* built-in cameras to\n> > > >> be available before initializing, as otherwise the system will be\n> > > >> permanently stuck with only part of the cameras available.\n> > > >>\n> > > >> I think this isn't something we should solve, the system should ensure\n> > > >> that device nodes have been created before starting the camera service.\n> > > >\n> > > > Yes, given what you've clarified, then I agree - the service should be\n> > > > dependent upon the devices being brought up. I think systemd can do\n> > > > that, but CrOS uses upstart, so I don't know what that might support.\n> > >\n> > > As ugly as it sounds, we have an udev rule which restarts the camera\n> > > service when devices matching the internal cameras show up. I believe\n> > > we match them by the \"video4linux\" class and particular hardware\n> > > subsystems, e.g. PCI or I2C.\n> >\n> > Has it always worked like that, or is it fairly recent ? If the service\n> > is restarted when new devices appear, everything should be fine even if\n> > the HAL initializes successfully with zero cameras, shouldn't it ?\n> >\n>\n> Yes, it's a recent change for a new platform where we ended up with\n> dynamic detection of components. Actually it hasn't landed yet. The CL\n> for reference is\n> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n>\n\nDo you happen to know why I see this issue being triggered only for\nCTS which goes through the android services ? It goes through the\ncros_camera_service if I got the architecture right, but even without\nthis patch, the camera service is always able to identify both\ncameras...\n\nIt is still not clear to me if the android camera service (is this ARC++?)\ninteracts with the cros_camera_service through the same mojo interface\nas stock ChromeOS camera application (and cros_camera_test?) use or\nthere's a different interface for it..\n\nThanks\n j","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 75DD5BD86F\n\tfor ;\n\tThu, 23 Jul 2020 09:43:19 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E9A1861136;\n\tThu, 23 Jul 2020 11:43:18 +0200 (CEST)","from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net\n\t[217.70.183.201])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B39C76039B\n\tfor ;\n\tThu, 23 Jul 2020 11:43:17 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 8F7A91BF214;\n\tThu, 23 Jul 2020 09:43:16 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Thu, 23 Jul 2020 11:46:54 +0200","From":"Jacopo Mondi ","To":"Tomasz Figa ","Message-ID":"<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>","References":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11530,"web_url":"https://patchwork.libcamera.org/comment/11530/","msgid":"<20200724012831.GK21353@pendragon.ideasonboard.com>","date":"2020-07-24T01:28:31","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Tomasz,\n\nOn Wed, Jul 22, 2020 at 08:10:03PM +0200, Tomasz Figa wrote:\n> On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart wrote:\n> > On Wed, Jul 22, 2020 at 04:31:42PM +0200, Tomasz Figa wrote:\n> >> On Wed, Jul 22, 2020 at 4:19 PM Kieran Bingham wrote:\n> >>>\n> >>> Hi Laurent, Jacopo,\n> >>>\n> >>> Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> >>> invalid assumption about what is reasonable :-( and a lack of\n> >>> understanding of how CrOS implemented working around that assumption.\n> >>>\n> >>> So my whole premise has been completely wrong and should be ignored.\n> >>>\n> >>> On 22/07/2020 14:49, Laurent Pinchart wrote:\n> >>>> Hello,\n> >>>>\n> >>>> (CC'ing Tomasz)\n> >>>>\n> >>>> On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> >>>>> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> >>>>>> Hi Kieran,\n> >>>>>> I think the conversation diverged, as I was clearly overlooking\n> >>>>>> the UVC camera use case and you're missing the intended use case of\n> >>>>>> this patch. I'll try to reply and summarize at the end.\n> >>>>>\n> >>>>> My interpretation is that this patch fixes the issue you are\n> >>>>> experiencing, (which is a valid thing to do, and this is likely a\n> >>>>> suitable solution for the time being)\n> >>>>>\n> >>>>> I was indeed trying to highlight that it would cause breakage for the\n> >>>>> UVC style use cases, and I believe system degradation for systems with\n> >>>>> no cameras attached, and which is why I thought a \\todo is required.\n> >>>>>\n> >>>>>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> >>>>>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> >>>>>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> >>>>>>>>> Hi Jacopo,\n> >>>>>>>>>\n> >>>>>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> >>>>>>>>>> Hi Kieran,\n> >>>>>>>>>>\n> >>>>>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> >>>>>>>>>>> Hi Jacopo,\n> >>>>>>>>>>>\n> >>>>>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> >>>>>>>>>>>> The CameraHalManager initialization tries to start the\n> >>>>>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> When initialization is called too early during system boot, it might\n> >>>>>>>>>>>> happen that the media graphs are still being registered to user-space\n> >>>>>>>>>>>> preventing any pipeline handler to match and register cameras.\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n> >>>>>>>>>>>> cameras are available in the system, reporting that information\n> >>>>>>>>>>>> to the camera stack:\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> >>>>>>>>>>>>\n> >>>>>>>>>>>\n> >>>>>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> >>>>>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> >>>>>>>>>>>\n> >>>>>>>>>>\n> >>>>>>>>>> I don't get what your comment is about here, I'm sorry\n> >>>>>>>>>\n> >>>>>>>>>\n> >>>>>>>>> Your patch prevents loading of libcamera completely unless there is at\n> >>>>>>>>> least one camera right ?\n> >>>>>>>>>\n> >>>>>>>>> What happens on say an x86 platform which will only have UVC cameras,\n> >>>>>>>>> yet will use libcamera...\n> >>>>>>>>>\n> >>>>>>>>> When will it be acceptable to load the camera service.\n> >>>>>>>>>\n> >>>>>>>>> Should the service itself continually fail, and respawn until someone\n> >>>>>>>>> plugs in a camera?\n> >>>>>>>>>\n> >>>>>>>>\n> >>>>>>>> That's my understanding\n> >>>>>>>\n> >>>>>>> That sounds like that would cause a very bad implementation to me.\n> >>>>>>>\n> >>>>>>> Waiting for upstart to decide when to retry launching the camera-service\n> >>>>>>> wouldn't be an acceptable time to wait IMO.\n> >>>>>>>\n> >>>>>>> Either it tries every second, and is flooding the system, or it tries\n> >>>>>>> every minute and it takes a minute before you can view your newly\n> >>>>>>> attached UVC camera ...\n> >>>>>>>\n> >>>>>>> I would expect event driven camera attachment, not polled ...\n> >>>>>>\n> >>>>>> For UVC we will have to register non-active cameras and notify to\n> >>>>>> android they're available when hotplug is detected\n> >>>>>\n> >>>>> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> >>>>> implementation does in CrOS.\n> >>>>>\n> >>>>>> For non-pluggable cameras, as shall wait until the pipeline handler is\n> >>>>>> matched and has registered cameras\n> >>>>>\n> >>>>> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> >>>>> stack.\n> >>>>>\n> >>>>> I believe that means we will in some form or another have to also do the\n> >>>>> same (some sort of pre-allocation if that's what is needed to make\n> >>>>> android happy). I dislike the idea that we would then have a 'maximum\n> >>>>> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> >>>>> get hit without an extreme use case.\n> >>>>\n> >>>> I don't think we can do this, as, as far as I know, Android doesn't\n> >>>> support status changes for internal cameras. Only cameras reported as\n> >>>> external can generate a status change event. Quoting from\n> >>>> camera/provider/2.4/ICameraProviderCallback.hal in\n> >>>> https://android.googlesource.com/platform/hardware/interfaces:\n> >>>>\n> >>>> * On camera service startup, when ICameraProvider::setCallback is invoked,\n> >>>> * the camera service must assume that all internal camera devices are in\n> >>>> * the CAMERA_DEVICE_STATUS_PRESENT state.\n> >>>> *\n> >>>> * The provider must call this method to inform the camera service of any\n> >>>> * initially NOT_PRESENT devices, and of any external camera devices that\n> >>>> * are already present, as soon as the callbacks are available through\n> >>>> * setCallback.\n> >>>>\n> >>>>> Now ... given that we (if we support UVC) must support dynamically\n> >>>>> adding cameras when they are available - I anticipate that the hotplug\n> >>>>> work will do that.\n> >>>>>\n> >>>>> When we do that, We could also use it for -non hotplug cameras, and\n> >>>>> simply allow cameras to be registered correctly using the same methods\n> >>>>> as they are discovered/notified by udev/hotplug mechanisms.\n> >>>>>\n> >>>>> Because even if they are fixed CSI2 busses, we're still going to get\n> >>>>> udev events when they appear as a media-device right ?\n> >>>>\n> >>>> Yes, on the libcamera side hotplug should work perfectly fine for\n> >>>> internal cameras, but unfortunately not on the Android side.\n> >>>>\n> >>>>>>>>> So taking that further, what I'm saying is, perhaps this should be\n> >>>>>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> >>>>>>>>> created they should notify android that a new camera is now available.\n> >>>>>>>>\n> >>>>>>>> Do you know if that's even possible ?\n> >>>>>>>>\n> >>>>>>>> I think Android expects the number of possible cameras to be\n> >>>>>>>> statically defined, there's a callback to notify a change of status of\n> >>>>>>>> a device, but we're dealing here with -creating- cameras based on the\n> >>>>>>>> number of cameras detected by android.\n> >>>>>>>\n> >>>>>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> >>>>>>> some cameras or such and only activate them on device connection.\n> >>>>>>>\n> >>>>>>> I fear we might have to do the same somehow in the future, as indeed it\n> >>>>>>> might not be possible to tell android there is a 'new' camera. Only an\n> >>>>>>> update to an existing one... not sure it will require more investigation.\n> >>>>>>\n> >>>>>> Yes, but that's for UVC. Built-in cameras are different, and Android\n> >>>>>> bets on the fact if you have a camera service running then your system\n> >>>>>> has cameras which are expected to be registered.\n> >>>>>>\n> >>>>>> Keep in mind so far there's been an HAL for UVC and one for built-in\n> >>>>>> cameras. We're handling both and that's a new use case afaict\n> >>>>>\n> >>>>> Precisely - and that's what I've been trying to discuss on this patch\n> >>>>> ;-) I'm sorry my thoughts didn't come across clearly.\n> >>>>>\n> >>>>>>>>>>>> Fix this by returning an error code if no camera is registered in the\n> >>>>>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n> >>>>>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> >>>>>>>>>>>> after the media device dependencies have been registered:\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> >>>>>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> >>>>>>>>>>>> ....\n> >>>>>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> >>>>>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> >>>>>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> >>>>>>>>>>>> between that and other negative values, so it does not really make\n> >>>>>>>>>>>> a difference.\n> >>>>>>>>>>>\n> >>>>>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> >>>>>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> >>>>>>>>>>> which doesn't yet have any cameras connected).\n> >>>>>>>>>>\n> >>>>>>>>>> You keep defferring ? I think the framework handles that\n> >>>>>>>>>\n> >>>>>>>>> I would be surprised if it would be expected behaviour to just poll\n> >>>>>>>>> launching of the camera service until there is a camera there to satisfy\n> >>>>>>>>> it ...\n> >>>>\n> >>>> This is actually how the camera HALs in Chrome OS handle this issue :-(\n> >>>\n> >>>\n> >>> Oh dear :-( Well I guess that invalidates all my assumptions\n> >>>\n> >>> /me throws toys out of the pram.\n> >>>\n> >>>\n> >>>>>>>>>>> How many times will we defer? How long does it defer for for instance?\n> >>>>>>>>>>\n> >>>>>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n> >>>>>>>>>> what is the timeout in the framework, as afaict android services are\n> >>>>>>>>>> restarted by default.\n> >>>>>>>>>\n> >>>>>>>>> Ok, so this is mostly dealing with the race at startup ...\n> >>>>>>>>>\n> >>>>>>>>>>>> Signed-off-by: Jacopo Mondi \n> >>>>>>>>>>>> ---\n> >>>>>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> >>>>>>>>>>>> 1 file changed, 11 insertions(+)\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n> >>>>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> >>>>>>>>>>>> ++index;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> + /*\n> >>>>>>>>>>>> + * If no pipeline has registered cameras, defer initialization to give\n> >>>>>>>>>>>> + * time to media devices to register to user-space.\n> >>>>>>>>>>>> + */\n> >>>>>>>>>>>> + if (index == 0) {\n> >>>>>>>>>>>> + LOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> >>>>>>>>>>>\n> >>>>>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> >>>>>>>>>>> always prints somewhere.\n> >>>>>>>>>>\n> >>>>>>>>>> I think that's kind of expected to happen, I don't think this should\n> >>>>>>>>>> be an error\n> >>>>>>>>>\n> >>>>>>>>> Info? Warning?\n> >>>>>>>>>\n> >>>>>>>>> If libcamera has prevented loading, I think it would be good to know\n> >>>>>>>>> about it by default...\n> >>>>>>>>>\n> >>>>>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> >>>>>>>>>>> this would go through if only one camera of two in the system is available).\n> >>>>>>>>>>\n> >>>>>>>>>> If the pipeline handler match (ie the device enumerator matches the\n> >>>>>>>>>> dependencies) than the match() function continues and register\n> >>>>>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> >>>>>>>>>> state.\n> >>>>>>>>>\n> >>>>>>>>> Ok, so it really is about having at least one camera available?\n> >>>>>>>>\n> >>>>>>>> It's about at least a pipeline being matched.\n> >>>>>>>\n> >>>>>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n> >>>>>>> there won't be a device\n> >>>>>>\n> >>>>>> Not for what Android has been thought to work on. You integrate a\n> >>>>>> system with cameras, it's fair to defer until those cameras are\n> >>>>>> registered. You have no cameras, either you disable the camera stack\n> >>>>>> or you register 0 cameras and then the camera subsystem is silent and\n> >>>>>> not accessible.\n> >>>>>\n> >>>>> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> >>>>>\n> >>>>> I've been annoyed I haven't been able to connect a UVC camera to my\n> >>>>> phone before ;-)\n> >>>>>\n> >>>>>>>>> I think looking at CameraHalManager::init(), as it only creates a\n> >>>>>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> >>>>>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> >>>>>>>>> unlikely to get in here between the point that one has been available\n> >>>>>>>>> and another is still loading ...\n> >>>>>>>>\n> >>>>>>>> I don't think that's possible.\n> >>>>>>>>\n> >>>>>>>> If a all the dependencies of a pipeline handler are matched, the\n> >>>>>>>> camera are registered. What is the code flow that registers one\n> >>>>>>>> camera, returns to the camera manager, then calls the pipeline handler\n> >>>>>>>> again to register more ?\n> >>>>>>>\n> >>>>>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> >>>>>>> each camera attached.\n> >>>>>>>\n> >>>>>>> But yes, perhaps due to the threading model, it won't be a possible race.\n> >>>>>>>\n> >>>>>>> I was thinking of when a pipeline starts and only 'one' of it's media\n> >>>>>>> devices is available.\n> >>>>>>\n> >>>>>> Yes, for UVC that's different I agree. A call to\n> >>>>>> CameraManager::start() might match one uvc media device but miss a\n> >>>>>> second one which still have to appear to userspace.\n> >>>>>>\n> >>>>>> As said, UVC needs some special handling and pre-register cameras\n> >>>>>\n> >>>>> Yes, and we will therefore /have/ to do that.\n> >>>>\n> >>>> I also don't see any way around this *for UVC*.\n> >>>>\n> >>>>>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n> >>>>>>>>> being added to the HAL.\n> >>>>>>>>\n> >>>>>>>> I agree, we should support inserting and removing new cameras, but I\n> >>>>>>>> think at this point, we should really start only if cameras are\n> >>>>>>>> registered and any pipeline handler is matched.\n> >>>>>>>>\n> >>>>>>>> I'm not sure I got what other alternative approach you are proposing\n> >>>>>>>> to be honest.\n> >>>>>>>\n> >>>>>>> I'm suggesting that when hotplug support is available, this patch, if\n> >>>>>>> integrated, might need to be reverted in effect ... and thus a todo note\n> >>>>>>> should be added to say that or such.\n> >>>>>>>\n> >>>>>>> I believe we should be able to start the camera service successfully\n> >>>>>>> with zero cameras if there are zero cameras at the time the service starts.\n> >>>>>>\n> >>>>>> I don't think so for built-in cameras. We start the service when\n> >>>>>> cameras are registered, otherwise if you have to pre-allocate cameras\n> >>>>>> you would need to know how many of them the pipeline handler expects\n> >>>>>> to register and that's not known before it has been matched.\n> >>>>>\n> >>>>> Yes, not knowing in advance is a pain point.\n> >>>>>\n> >>>>>> UVC, again, is a different story as it supports hotplug.\n> >>>>>\n> >>>>> But equally we don't know how many UVC cameras someone could connect,\n> >>>>> but we can define a max. The question would then be if that 'max'\n> >>>>> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> >>>>> top' of the static cameras.\n> >>>>>\n> >>>>> Including it in the max, means we can just preallocate the camera\n> >>>>> registrations for Android sake, and register the (static) cameras in\n> >>>>> that list as soon as they are available through the udev notifications\n> >>>>> system.\n> >>>>>\n> >>>>>>>>>>> ..\n> >>>>>>>>>>>\n> >>>>>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> >>>>>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n> >>>>>>>>>>\n> >>>>>>>>>> A \\todo item to record this should be fixed how ?\n> >>>>>>>>>>\n> >>>>>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> >>>>>>>>>> handler it tries match, but this seems not a good idea to me.\n> >>>>>>>>>\n> >>>>>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> >>>>>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n> >>>>>>>>\n> >>>>>>>> That means Android/CrOS starts without the camera being active (ie.\n> >>>>>>>> the camera application icon is not available, in CrOS in example).\n> >>>>>>>\n> >>>>>>> Hrm, so it disables the app completely?\n> >>>>>>>\n> >>>>>>> Anyway, essentially I'm thinking - whatever happens with the existing\n> >>>>>>> UVC stack is what we would need to be able to mirror.\n> >>>>>>>\n> >>>>>>>>> they become available at which point they'll be registered through the\n> >>>>>>>>> same mechanisms as the hotplugging ...\n> >>>>>>>>\n> >>>>>>>> Still I don't get if you are talking about libcamera hotplug or\n> >>>>>>>> android hotplug, which, to my understanding, expects anyway a camera\n> >>>>>>>> device to be registered but marked as 'not present' (looking at the\n> >>>>>>>> camera_device_status enumeration)\n> >>>>>>>\n> >>>>>>>\n> >>>>>>> I was going to ask how does it register camera's it doesn't know will\n> >>>>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> >>>>>>> cameras or such, so that explains how that one works.\n> >>>>>>>\n> >>>>>>\n> >>>>>> I see three cases:\n> >>>>>> 1) No uvc support\n> >>>>>> Defer until a pipeline handler is matched and has registered\n> >>>>>> cameras\n> >>>>>>\n> >>>>>> 2) UVC only\n> >>>>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> >>>>>> this has to happen is tricky, as the HAL needs to know it should\n> >>>>>> not wait for built-in cameras and can proceed to register\n> >>>>>> non-active USB cameras\n> >>>>>>\n> >>>>>> 3) built-in + UVC\n> >>>>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> >>>>>> defer until cameras are available, then pre-register UVC cameras.\n> >>>>>>\n> >>>>>> The hard part I think it is to define how to instruct the HAL it has\n> >>>>>> to wait for built-in to appear or not before registering UVC\n> >>>>>> non-active cameras.\n> >>>>>\n> >>>>> My suggestion is (not blocking this patch, but on top) to do so by\n> >>>>> treating static cameras and hotpluggable cameras in the same way.\n> >>>>>\n> >>>>> It's just that static cameras will never 'unplug' (unless someone\n> >>>>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> >>>>\n> >>>> As explained above, this is unfortunately not an option. To make this\n> >>>> matter more complicated, we need to wait for *all* built-in cameras to\n> >>>> be available before initializing, as otherwise the system will be\n> >>>> permanently stuck with only part of the cameras available.\n> >>>>\n> >>>> I think this isn't something we should solve, the system should ensure\n> >>>> that device nodes have been created before starting the camera service.\n> >>>\n> >>> Yes, given what you've clarified, then I agree - the service should be\n> >>> dependent upon the devices being brought up. I think systemd can do\n> >>> that, but CrOS uses upstart, so I don't know what that might support.\n> >>\n> >> As ugly as it sounds, we have an udev rule which restarts the camera\n> >> service when devices matching the internal cameras show up. I believe\n> >> we match them by the \"video4linux\" class and particular hardware\n> >> subsystems, e.g. PCI or I2C.\n> >\n> > Has it always worked like that, or is it fairly recent ? If the service\n> > is restarted when new devices appear, everything should be fine even if\n> > the HAL initializes successfully with zero cameras, shouldn't it ?\n> \n> Yes, it's a recent change for a new platform where we ended up with\n> dynamic detection of components. Actually it hasn't landed yet. The CL\n> for reference is\n> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n\nDoes this mean that if we integrate that change manually (including\npossible dependencies), then no further action should be needed on the\nHAL side ?\n\n> >>> But that's out of scope (and control) for us I think ...\n> >>>\n> >>>> I've expressed this before in a conversation with Tomasz (not sure if it\n> >>>> was on the public mailing list, IRC, or in private), but we didn't reach\n> >>>> an agreement at the time. One option that was discussed was a platform\n> >>>> configuration file that would list the cameras expected to be present (I\n> >>>> think everybody knows how much I like that :-)). Another option that has\n> >>>> been experimented was\n> >>>> https://lists.libcamera.org/pipermail/libcamera-devel/2019-September/004850.html.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 31E66BD86F\n\tfor ;\n\tFri, 24 Jul 2020 01:28:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A241E6117A;\n\tFri, 24 Jul 2020 03:28:39 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DE6160539\n\tfor ;\n\tFri, 24 Jul 2020 03:28:38 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 99088279;\n\tFri, 24 Jul 2020 03:28:37 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"BV+MEWIa\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595554117;\n\tbh=8oSG0yEVpFx8eB1vrQqIOZx2V31xOv7zKRdft1ffb58=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=BV+MEWIa/DEQrKsql9zMRLUotkI+HAhtc2uueOIymRJP43NF/E1p30bB4yTEPVbKN\n\twHPDByxXRtdRDZBTQL5WIEls2OvjY9eK/Kbb/9+uiab6IisEee4mx8J0prBIADAH/0\n\tGDancq4qJw5ujH2CAQDbt8gYoOPIQPdb/K0/cOBE=","Date":"Fri, 24 Jul 2020 04:28:31 +0300","From":"Laurent Pinchart ","To":"Tomasz Figa ","Message-ID":"<20200724012831.GK21353@pendragon.ideasonboard.com>","References":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11617,"web_url":"https://patchwork.libcamera.org/comment/11617/","msgid":"","date":"2020-07-27T09:30:16","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":9,"url":"https://patchwork.libcamera.org/api/people/9/","name":"Tomasz Figa","email":"tfiga@chromium.org"},"content":"On Thu, Jul 23, 2020 at 11:43 AM Jacopo Mondi wrote:\n>\n> Hi Tomasz,\n>\n> On Wed, Jul 22, 2020 at 08:10:03PM +0200, Tomasz Figa wrote:\n> > On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart\n> > wrote:\n> > >\n> > > Hi Tomasz,\n> > >\n>\n> snip\n>\n> > > > >>>>> I was going to ask how does it register camera's it doesn't know will\n> > > > >>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> > > > >>>>> cameras or such, so that explains how that one works.\n> > > > >>>>>\n> > > > >>>>\n> > > > >>>> I see three cases:\n> > > > >>>> 1) No uvc support\n> > > > >>>> Defer until a pipeline handler is matched and has registered\n> > > > >>>> cameras\n> > > > >>>>\n> > > > >>>> 2) UVC only\n> > > > >>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> > > > >>>> this has to happen is tricky, as the HAL needs to know it should\n> > > > >>>> not wait for built-in cameras and can proceed to register\n> > > > >>>> non-active USB cameras\n> > > > >>>>\n> > > > >>>> 3) built-in + UVC\n> > > > >>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> > > > >>>> defer until cameras are available, then pre-register UVC cameras.\n> > > > >>>>\n> > > > >>>> The hard part I think it is to define how to instruct the HAL it has\n> > > > >>>> to wait for built-in to appear or not before registering UVC\n> > > > >>>> non-active cameras.\n> > > > >>>\n> > > > >>> My suggestion is (not blocking this patch, but on top) to do so by\n> > > > >>> treating static cameras and hotpluggable cameras in the same way.\n> > > > >>>\n> > > > >>> It's just that static cameras will never 'unplug' (unless someone\n> > > > >>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> > > > >>\n> > > > >> As explained above, this is unfortunately not an option. To make this\n> > > > >> matter more complicated, we need to wait for *all* built-in cameras to\n> > > > >> be available before initializing, as otherwise the system will be\n> > > > >> permanently stuck with only part of the cameras available.\n> > > > >>\n> > > > >> I think this isn't something we should solve, the system should ensure\n> > > > >> that device nodes have been created before starting the camera service.\n> > > > >\n> > > > > Yes, given what you've clarified, then I agree - the service should be\n> > > > > dependent upon the devices being brought up. I think systemd can do\n> > > > > that, but CrOS uses upstart, so I don't know what that might support.\n> > > >\n> > > > As ugly as it sounds, we have an udev rule which restarts the camera\n> > > > service when devices matching the internal cameras show up. I believe\n> > > > we match them by the \"video4linux\" class and particular hardware\n> > > > subsystems, e.g. PCI or I2C.\n> > >\n> > > Has it always worked like that, or is it fairly recent ? If the service\n> > > is restarted when new devices appear, everything should be fine even if\n> > > the HAL initializes successfully with zero cameras, shouldn't it ?\n> > >\n> >\n> > Yes, it's a recent change for a new platform where we ended up with\n> > dynamic detection of components. Actually it hasn't landed yet. The CL\n> > for reference is\n> > https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n> >\n>\n> Do you happen to know why I see this issue being triggered only for\n> CTS which goes through the android services ? It goes through the\n> cros_camera_service if I got the architecture right, but even without\n> this patch, the camera service is always able to identify both\n> cameras...\n>\n> It is still not clear to me if the android camera service (is this ARC++?)\n> interacts with the cros_camera_service through the same mojo interface\n> as stock ChromeOS camera application (and cros_camera_test?) use or\n> there's a different interface for it..\n\nYes, Android interacts with the cros_camera service using the same\ninterface. I think it's just that Android camera frameworks cache the\ncamera metadata when they query it the first time and would ignore any\nlater changes. On the contrary, Chrome probably just queries the\nmetadata every time some camera use case is executed.\n\nBest regards,\nTomasz","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 94DE2BD86F\n\tfor ;\n\tMon, 27 Jul 2020 09:30:32 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 66B2561247;\n\tMon, 27 Jul 2020 11:30:32 +0200 (CEST)","from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com\n\t[IPv6:2a00:1450:4864:20::52c])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CB6A6039B\n\tfor ;\n\tMon, 27 Jul 2020 11:30:30 +0200 (CEST)","by mail-ed1-x52c.google.com with SMTP id a8so11613816edy.1\n\tfor ;\n\tMon, 27 Jul 2020 02:30:30 -0700 (PDT)","from mail-wm1-f46.google.com (mail-wm1-f46.google.com.\n\t[209.85.128.46]) by smtp.gmail.com with ESMTPSA id\n\tp21sm3773798eds.11.2020.07.27.02.30.28\n\tfor \n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 27 Jul 2020 02:30:29 -0700 (PDT)","by mail-wm1-f46.google.com with SMTP id x5so12937426wmi.2\n\tfor ;\n\tMon, 27 Jul 2020 02:30:28 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"i3mXwkuA\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=iEELqdZZIGQ8nxa162flZNjTFqa8IHWgeBsDS96Yo6I=;\n\tb=i3mXwkuAvOkkurV8iC+hatkN+kTqg7JUV+JdJMYn8wpYy5hS6m8NVDyCrK4Z6lRXzh\n\tP2ZxkQ+Q8Qt6TBi6l6MNB3VlNxlXKiwmF2iHbaU8PARJ4JnpwP4VJL/l9EVLveWD2WIf\n\tsrCw7GN4/Xo3cGoi2He4SbAcUufWBT1AGYqxQ=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=iEELqdZZIGQ8nxa162flZNjTFqa8IHWgeBsDS96Yo6I=;\n\tb=OfIm7THOxkKwoWmt5JtMyUQ1KkdgOY0cWDd7VErTRlVpw2NVCiRz3fOhyLe9/gojG2\n\tIOUAmi+IJe+/4LU69Y8UlchYtswGGCb8sk7BEmDaLonoz21i1ggVJcfkXGnmYY4rifuE\n\tr/SF78bOpOfREKDBniDKuqYg33YBl7VDrbuA22Ljyq6IfFwGnNjFtNhJ9bYLqWy4bPPV\n\t1UfD+5N8AbUc8cwKE79jg+UyiRoP5m8nwO5KfXIBKrFrMYZqThXJg7Ism/zF6d1VEhBG\n\tSXdxDdjIPNB/8Xb5AGnqWUNn52QQgjEVXrmf0/J7mOsXY8U44CTeKJMpbygn/gmBY6LZ\n\tif9g==","X-Gm-Message-State":"AOAM531QCbIfUdIBvIqu2OZw6iH+lU13h7qSQZdmmtuNoAIrUm0CYNEg\n\tk+xWA9XNkoFb1ZtjgurAhJo2TyM54OE=","X-Google-Smtp-Source":"ABdhPJxD/n6hMtAo8mEmOd+T8KWcEWVw5x6x9ViGIqDDyP4AofPlcr1rkLSV8j6NCXxxTpgwj6LvLw==","X-Received":["by 2002:aa7:d458:: with SMTP id\n\tq24mr20275422edr.25.1595842229655; \n\tMon, 27 Jul 2020 02:30:29 -0700 (PDT)","by 2002:a1c:dc02:: with SMTP id t2mr19434624wmg.55.1595842227887;\n\tMon, 27 Jul 2020 02:30:27 -0700 (PDT)"],"MIME-Version":"1.0","References":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t\n\t<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>","In-Reply-To":"<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>","From":"Tomasz Figa ","Date":"Mon, 27 Jul 2020 11:30:16 +0200","X-Gmail-Original-Message-ID":"","Message-ID":"","To":"Jacopo Mondi ","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11618,"web_url":"https://patchwork.libcamera.org/comment/11618/","msgid":"","date":"2020-07-27T09:35:00","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":9,"url":"https://patchwork.libcamera.org/api/people/9/","name":"Tomasz Figa","email":"tfiga@chromium.org"},"content":"On Fri, Jul 24, 2020 at 3:28 AM Laurent Pinchart\n wrote:\n>\n> Hi Tomasz,\n>\n> On Wed, Jul 22, 2020 at 08:10:03PM +0200, Tomasz Figa wrote:\n> > On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart wrote:\n> > > On Wed, Jul 22, 2020 at 04:31:42PM +0200, Tomasz Figa wrote:\n> > >> On Wed, Jul 22, 2020 at 4:19 PM Kieran Bingham wrote:\n> > >>>\n> > >>> Hi Laurent, Jacopo,\n> > >>>\n> > >>> Jacopo, I'm sorry for the noise, it seems my opinions were based on an\n> > >>> invalid assumption about what is reasonable :-( and a lack of\n> > >>> understanding of how CrOS implemented working around that assumption.\n> > >>>\n> > >>> So my whole premise has been completely wrong and should be ignored.\n> > >>>\n> > >>> On 22/07/2020 14:49, Laurent Pinchart wrote:\n> > >>>> Hello,\n> > >>>>\n> > >>>> (CC'ing Tomasz)\n> > >>>>\n> > >>>> On Wed, Jul 22, 2020 at 12:24:56PM +0100, Kieran Bingham wrote:\n> > >>>>> On 22/07/2020 12:12, Jacopo Mondi wrote:\n> > >>>>>> Hi Kieran,\n> > >>>>>> I think the conversation diverged, as I was clearly overlooking\n> > >>>>>> the UVC camera use case and you're missing the intended use case of\n> > >>>>>> this patch. I'll try to reply and summarize at the end.\n> > >>>>>\n> > >>>>> My interpretation is that this patch fixes the issue you are\n> > >>>>> experiencing, (which is a valid thing to do, and this is likely a\n> > >>>>> suitable solution for the time being)\n> > >>>>>\n> > >>>>> I was indeed trying to highlight that it would cause breakage for the\n> > >>>>> UVC style use cases, and I believe system degradation for systems with\n> > >>>>> no cameras attached, and which is why I thought a \\todo is required.\n> > >>>>>\n> > >>>>>> On Tue, Jul 21, 2020 at 09:59:10PM +0100, Kieran Bingham wrote:\n> > >>>>>>> On 21/07/2020 14:09, Jacopo Mondi wrote:\n> > >>>>>>>> On Tue, Jul 21, 2020 at 01:47:29PM +0100, Kieran Bingham wrote:\n> > >>>>>>>>> Hi Jacopo,\n> > >>>>>>>>>\n> > >>>>>>>>> On 21/07/2020 13:13, Jacopo Mondi wrote:\n> > >>>>>>>>>> Hi Kieran,\n> > >>>>>>>>>>\n> > >>>>>>>>>> On Tue, Jul 21, 2020 at 12:41:16PM +0100, Kieran Bingham wrote:\n> > >>>>>>>>>>> Hi Jacopo,\n> > >>>>>>>>>>>\n> > >>>>>>>>>>> On 21/07/2020 12:26, Jacopo Mondi wrote:\n> > >>>>>>>>>>>> The CameraHalManager initialization tries to start the\n> > >>>>>>>>>>>> libcamera::CameraManager which enumerate the registered cameras.\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> When initialization is called too early during system boot, it might\n> > >>>>>>>>>>>> happen that the media graphs are still being registered to user-space\n> > >>>>>>>>>>>> preventing any pipeline handler to match and register cameras.\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> If that happens, the CameraHalManager silently accepts that no\n> > >>>>>>>>>>>> cameras are available in the system, reporting that information\n> > >>>>>>>>>>>> to the camera stack:\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> > >>>>>>>>>>>> cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> > >>>>>>>>>>>> CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>\n> > >>>>>>>>>>> Hrm ... should this be part of the hotplug work that as cameras become\n> > >>>>>>>>>>> available this gets updated somehow? - Even on *non* hotpluggable pipelines?\n> > >>>>>>>>>>>\n> > >>>>>>>>>>\n> > >>>>>>>>>> I don't get what your comment is about here, I'm sorry\n> > >>>>>>>>>\n> > >>>>>>>>>\n> > >>>>>>>>> Your patch prevents loading of libcamera completely unless there is at\n> > >>>>>>>>> least one camera right ?\n> > >>>>>>>>>\n> > >>>>>>>>> What happens on say an x86 platform which will only have UVC cameras,\n> > >>>>>>>>> yet will use libcamera...\n> > >>>>>>>>>\n> > >>>>>>>>> When will it be acceptable to load the camera service.\n> > >>>>>>>>>\n> > >>>>>>>>> Should the service itself continually fail, and respawn until someone\n> > >>>>>>>>> plugs in a camera?\n> > >>>>>>>>>\n> > >>>>>>>>\n> > >>>>>>>> That's my understanding\n> > >>>>>>>\n> > >>>>>>> That sounds like that would cause a very bad implementation to me.\n> > >>>>>>>\n> > >>>>>>> Waiting for upstart to decide when to retry launching the camera-service\n> > >>>>>>> wouldn't be an acceptable time to wait IMO.\n> > >>>>>>>\n> > >>>>>>> Either it tries every second, and is flooding the system, or it tries\n> > >>>>>>> every minute and it takes a minute before you can view your newly\n> > >>>>>>> attached UVC camera ...\n> > >>>>>>>\n> > >>>>>>> I would expect event driven camera attachment, not polled ...\n> > >>>>>>\n> > >>>>>> For UVC we will have to register non-active cameras and notify to\n> > >>>>>> android they're available when hotplug is detected\n> > >>>>>\n> > >>>>> Yes, this is what I think I see that the platform2/camera/hal/usb*\n> > >>>>> implementation does in CrOS.\n> > >>>>>\n> > >>>>>> For non-pluggable cameras, as shall wait until the pipeline handler is\n> > >>>>>> matched and has registered cameras\n> > >>>>>\n> > >>>>> The issue for us is that *we support UVC* ... so we're now a UVC capable\n> > >>>>> stack.\n> > >>>>>\n> > >>>>> I believe that means we will in some form or another have to also do the\n> > >>>>> same (some sort of pre-allocation if that's what is needed to make\n> > >>>>> android happy). I dislike the idea that we would then have a 'maximum\n> > >>>>> supported cameras' but maybe it is set arbitrarily high so it doesn't\n> > >>>>> get hit without an extreme use case.\n> > >>>>\n> > >>>> I don't think we can do this, as, as far as I know, Android doesn't\n> > >>>> support status changes for internal cameras. Only cameras reported as\n> > >>>> external can generate a status change event. Quoting from\n> > >>>> camera/provider/2.4/ICameraProviderCallback.hal in\n> > >>>> https://android.googlesource.com/platform/hardware/interfaces:\n> > >>>>\n> > >>>> * On camera service startup, when ICameraProvider::setCallback is invoked,\n> > >>>> * the camera service must assume that all internal camera devices are in\n> > >>>> * the CAMERA_DEVICE_STATUS_PRESENT state.\n> > >>>> *\n> > >>>> * The provider must call this method to inform the camera service of any\n> > >>>> * initially NOT_PRESENT devices, and of any external camera devices that\n> > >>>> * are already present, as soon as the callbacks are available through\n> > >>>> * setCallback.\n> > >>>>\n> > >>>>> Now ... given that we (if we support UVC) must support dynamically\n> > >>>>> adding cameras when they are available - I anticipate that the hotplug\n> > >>>>> work will do that.\n> > >>>>>\n> > >>>>> When we do that, We could also use it for -non hotplug cameras, and\n> > >>>>> simply allow cameras to be registered correctly using the same methods\n> > >>>>> as they are discovered/notified by udev/hotplug mechanisms.\n> > >>>>>\n> > >>>>> Because even if they are fixed CSI2 busses, we're still going to get\n> > >>>>> udev events when they appear as a media-device right ?\n> > >>>>\n> > >>>> Yes, on the libcamera side hotplug should work perfectly fine for\n> > >>>> internal cameras, but unfortunately not on the Android side.\n> > >>>>\n> > >>>>>>>>> So taking that further, what I'm saying is, perhaps this should be\n> > >>>>>>>>> perfectly able to launch with zero cameras, and /when/ cameras are\n> > >>>>>>>>> created they should notify android that a new camera is now available.\n> > >>>>>>>>\n> > >>>>>>>> Do you know if that's even possible ?\n> > >>>>>>>>\n> > >>>>>>>> I think Android expects the number of possible cameras to be\n> > >>>>>>>> statically defined, there's a callback to notify a change of status of\n> > >>>>>>>> a device, but we're dealing here with -creating- cameras based on the\n> > >>>>>>>> number of cameras detected by android.\n> > >>>>>>>\n> > >>>>>>> So, looking at the USB HAL in cros - it does look like they pre-allocate\n> > >>>>>>> some cameras or such and only activate them on device connection.\n> > >>>>>>>\n> > >>>>>>> I fear we might have to do the same somehow in the future, as indeed it\n> > >>>>>>> might not be possible to tell android there is a 'new' camera. Only an\n> > >>>>>>> update to an existing one... not sure it will require more investigation.\n> > >>>>>>\n> > >>>>>> Yes, but that's for UVC. Built-in cameras are different, and Android\n> > >>>>>> bets on the fact if you have a camera service running then your system\n> > >>>>>> has cameras which are expected to be registered.\n> > >>>>>>\n> > >>>>>> Keep in mind so far there's been an HAL for UVC and one for built-in\n> > >>>>>> cameras. We're handling both and that's a new use case afaict\n> > >>>>>\n> > >>>>> Precisely - and that's what I've been trying to discuss on this patch\n> > >>>>> ;-) I'm sorry my thoughts didn't come across clearly.\n> > >>>>>\n> > >>>>>>>>>>>> Fix this by returning an error code if no camera is registered in the\n> > >>>>>>>>>>>> system at the time CameraHalManager::init() is called. The camera\n> > >>>>>>>>>>>> framework then tries to re-load the HAL module later in time, hopefully\n> > >>>>>>>>>>>> after the media device dependencies have been registered:\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> 2020-07-21T12:26:37.903456+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 0 built-in camera(s)\n> > >>>>>>>>>>>> 2020-07-21T12:26:37.903521+02:00 INFO cros_camera_service[2054]: (5) StartOnThread(): SuperHAL started with 1 modules and 0 built-in cameras\n> > >>>>>>>>>>>> ....\n> > >>>>>>>>>>>> 2020-07-21T12:30:36.662877+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): Camera module \"libcamera camera HALv3 module\" has 2 built-in camera(s)\n> > >>>>>>>>>>>> 2020-07-21T12:30:36.663196+02:00 INFO cros_camera_service[5908]: (5910) StartOnThread(): SuperHAL started with 1 modules and 2 built-in cameras\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> Return -ENODEV as according to camera_common.h specification is the\n> > >>>>>>>>>>>> only supported error code. The CrOS HAL adapter does not distinguish\n> > >>>>>>>>>>>> between that and other negative values, so it does not really make\n> > >>>>>>>>>>>> a difference.\n> > >>>>>>>>>>>\n> > >>>>>>>>>>> This feels a bit punchy/racey still. And what happens in the instance\n> > >>>>>>>>>>> that we really don't yet have any cameras? (I.e. a UVC only platform,\n> > >>>>>>>>>>> which doesn't yet have any cameras connected).\n> > >>>>>>>>>>\n> > >>>>>>>>>> You keep defferring ? I think the framework handles that\n> > >>>>>>>>>\n> > >>>>>>>>> I would be surprised if it would be expected behaviour to just poll\n> > >>>>>>>>> launching of the camera service until there is a camera there to satisfy\n> > >>>>>>>>> it ...\n> > >>>>\n> > >>>> This is actually how the camera HALs in Chrome OS handle this issue :-(\n> > >>>\n> > >>>\n> > >>> Oh dear :-( Well I guess that invalidates all my assumptions\n> > >>>\n> > >>> /me throws toys out of the pram.\n> > >>>\n> > >>>\n> > >>>>>>>>>>> How many times will we defer? How long does it defer for for instance?\n> > >>>>>>>>>>\n> > >>>>>>>>>> Usually 1 is enough from my testing. I don't think we have to care\n> > >>>>>>>>>> what is the timeout in the framework, as afaict android services are\n> > >>>>>>>>>> restarted by default.\n> > >>>>>>>>>\n> > >>>>>>>>> Ok, so this is mostly dealing with the race at startup ...\n> > >>>>>>>>>\n> > >>>>>>>>>>>> Signed-off-by: Jacopo Mondi \n> > >>>>>>>>>>>> ---\n> > >>>>>>>>>>>> src/android/camera_hal_manager.cpp | 11 +++++++++++\n> > >>>>>>>>>>>> 1 file changed, 11 insertions(+)\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>>>> index 02b6418fb36d..e967d210e547 100644\n> > >>>>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>>>> @@ -73,6 +73,17 @@ int CameraHalManager::init()\n> > >>>>>>>>>>>> ++index;\n> > >>>>>>>>>>>> }\n> > >>>>>>>>>>>>\n> > >>>>>>>>>>>> + /*\n> > >>>>>>>>>>>> + * If no pipeline has registered cameras, defer initialization to give\n> > >>>>>>>>>>>> + * time to media devices to register to user-space.\n> > >>>>>>>>>>>> + */\n> > >>>>>>>>>>>> + if (index == 0) {\n> > >>>>>>>>>>>> + LOG(HAL, Debug) << \"Defer CameraHALManager initialization\";\n> > >>>>>>>>>>>\n> > >>>>>>>>>>> I think perhaps this needs a higher level than Debug to make sure it\n> > >>>>>>>>>>> always prints somewhere.\n> > >>>>>>>>>>\n> > >>>>>>>>>> I think that's kind of expected to happen, I don't think this should\n> > >>>>>>>>>> be an error\n> > >>>>>>>>>\n> > >>>>>>>>> Info? Warning?\n> > >>>>>>>>>\n> > >>>>>>>>> If libcamera has prevented loading, I think it would be good to know\n> > >>>>>>>>> about it by default...\n> > >>>>>>>>>\n> > >>>>>>>>>>> I fear this will be deflecting the issue, and introduces races (i.e.\n> > >>>>>>>>>>> this would go through if only one camera of two in the system is available).\n> > >>>>>>>>>>\n> > >>>>>>>>>> If the pipeline handler match (ie the device enumerator matches the\n> > >>>>>>>>>> dependencies) than the match() function continues and register\n> > >>>>>>>>>> cameras. I don't think we can found ourselves in a semi-initialized\n> > >>>>>>>>>> state.\n> > >>>>>>>>>\n> > >>>>>>>>> Ok, so it really is about having at least one camera available?\n> > >>>>>>>>\n> > >>>>>>>> It's about at least a pipeline being matched.\n> > >>>>>>>\n> > >>>>>>> Ok, but I don't think it's that difficult to conjur up a scenario where\n> > >>>>>>> there won't be a device\n> > >>>>>>\n> > >>>>>> Not for what Android has been thought to work on. You integrate a\n> > >>>>>> system with cameras, it's fair to defer until those cameras are\n> > >>>>>> registered. You have no cameras, either you disable the camera stack\n> > >>>>>> or you register 0 cameras and then the camera subsystem is silent and\n> > >>>>>> not accessible.\n> > >>>>>\n> > >>>>> Android is wrong - It should always support UVC hotplug ;-) hehehe.\n> > >>>>>\n> > >>>>> I've been annoyed I haven't been able to connect a UVC camera to my\n> > >>>>> phone before ;-)\n> > >>>>>\n> > >>>>>>>>> I think looking at CameraHalManager::init(), as it only creates a\n> > >>>>>>>>> CameraDevice for cameras available in cameraManager_->cameras(), it\n> > >>>>>>>>> /could/ be a race, it's just that creating two cameras is fast, so we're\n> > >>>>>>>>> unlikely to get in here between the point that one has been available\n> > >>>>>>>>> and another is still loading ...\n> > >>>>>>>>\n> > >>>>>>>> I don't think that's possible.\n> > >>>>>>>>\n> > >>>>>>>> If a all the dependencies of a pipeline handler are matched, the\n> > >>>>>>>> camera are registered. What is the code flow that registers one\n> > >>>>>>>> camera, returns to the camera manager, then calls the pipeline handler\n> > >>>>>>>> again to register more ?\n> > >>>>>>>\n> > >>>>>>> At least PipelineHandlerUVC will call PipelineHandler::match() once for\n> > >>>>>>> each camera attached.\n> > >>>>>>>\n> > >>>>>>> But yes, perhaps due to the threading model, it won't be a possible race.\n> > >>>>>>>\n> > >>>>>>> I was thinking of when a pipeline starts and only 'one' of it's media\n> > >>>>>>> devices is available.\n> > >>>>>>\n> > >>>>>> Yes, for UVC that's different I agree. A call to\n> > >>>>>> CameraManager::start() might match one uvc media device but miss a\n> > >>>>>> second one which still have to appear to userspace.\n> > >>>>>>\n> > >>>>>> As said, UVC needs some special handling and pre-register cameras\n> > >>>>>\n> > >>>>> Yes, and we will therefore /have/ to do that.\n> > >>>>\n> > >>>> I also don't see any way around this *for UVC*.\n> > >>>>\n> > >>>>>>>>> But still ... I think this is stuff that will get dealt with by hotplug\n> > >>>>>>>>> being added to the HAL.\n> > >>>>>>>>\n> > >>>>>>>> I agree, we should support inserting and removing new cameras, but I\n> > >>>>>>>> think at this point, we should really start only if cameras are\n> > >>>>>>>> registered and any pipeline handler is matched.\n> > >>>>>>>>\n> > >>>>>>>> I'm not sure I got what other alternative approach you are proposing\n> > >>>>>>>> to be honest.\n> > >>>>>>>\n> > >>>>>>> I'm suggesting that when hotplug support is available, this patch, if\n> > >>>>>>> integrated, might need to be reverted in effect ... and thus a todo note\n> > >>>>>>> should be added to say that or such.\n> > >>>>>>>\n> > >>>>>>> I believe we should be able to start the camera service successfully\n> > >>>>>>> with zero cameras if there are zero cameras at the time the service starts.\n> > >>>>>>\n> > >>>>>> I don't think so for built-in cameras. We start the service when\n> > >>>>>> cameras are registered, otherwise if you have to pre-allocate cameras\n> > >>>>>> you would need to know how many of them the pipeline handler expects\n> > >>>>>> to register and that's not known before it has been matched.\n> > >>>>>\n> > >>>>> Yes, not knowing in advance is a pain point.\n> > >>>>>\n> > >>>>>> UVC, again, is a different story as it supports hotplug.\n> > >>>>>\n> > >>>>> But equally we don't know how many UVC cameras someone could connect,\n> > >>>>> but we can define a max. The question would then be if that 'max'\n> > >>>>> includes static cameras, as well as dynamic UVC cameras, or if it's 'on\n> > >>>>> top' of the static cameras.\n> > >>>>>\n> > >>>>> Including it in the max, means we can just preallocate the camera\n> > >>>>> registrations for Android sake, and register the (static) cameras in\n> > >>>>> that list as soon as they are available through the udev notifications\n> > >>>>> system.\n> > >>>>>\n> > >>>>>>>>>>> ..\n> > >>>>>>>>>>>\n> > >>>>>>>>>>> Perhaps it can still go in if it solves an immediate problem, but in\n> > >>>>>>>>>>> that case I think it needs a \\todo or a warning of some kind...\n> > >>>>>>>>>>\n> > >>>>>>>>>> A \\todo item to record this should be fixed how ?\n> > >>>>>>>>>>\n> > >>>>>>>>>> Unless we expect the CameraManager to stall until any of the pipeline\n> > >>>>>>>>>> handler it tries match, but this seems not a good idea to me.\n> > >>>>>>>>>\n> > >>>>>>>>> No, i don't expect it to stall ... I expect it to complete successfully,\n> > >>>>>>>>> and if there are only 0 cameras, it will only have zero cameras - until\n> > >>>>>>>>\n> > >>>>>>>> That means Android/CrOS starts without the camera being active (ie.\n> > >>>>>>>> the camera application icon is not available, in CrOS in example).\n> > >>>>>>>\n> > >>>>>>> Hrm, so it disables the app completely?\n> > >>>>>>>\n> > >>>>>>> Anyway, essentially I'm thinking - whatever happens with the existing\n> > >>>>>>> UVC stack is what we would need to be able to mirror.\n> > >>>>>>>\n> > >>>>>>>>> they become available at which point they'll be registered through the\n> > >>>>>>>>> same mechanisms as the hotplugging ...\n> > >>>>>>>>\n> > >>>>>>>> Still I don't get if you are talking about libcamera hotplug or\n> > >>>>>>>> android hotplug, which, to my understanding, expects anyway a camera\n> > >>>>>>>> device to be registered but marked as 'not present' (looking at the\n> > >>>>>>>> camera_device_status enumeration)\n> > >>>>>>>\n> > >>>>>>>\n> > >>>>>>> I was going to ask how does it register camera's it doesn't know will\n> > >>>>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> > >>>>>>> cameras or such, so that explains how that one works.\n> > >>>>>>>\n> > >>>>>>\n> > >>>>>> I see three cases:\n> > >>>>>> 1) No uvc support\n> > >>>>>> Defer until a pipeline handler is matched and has registered\n> > >>>>>> cameras\n> > >>>>>>\n> > >>>>>> 2) UVC only\n> > >>>>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> > >>>>>> this has to happen is tricky, as the HAL needs to know it should\n> > >>>>>> not wait for built-in cameras and can proceed to register\n> > >>>>>> non-active USB cameras\n> > >>>>>>\n> > >>>>>> 3) built-in + UVC\n> > >>>>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> > >>>>>> defer until cameras are available, then pre-register UVC cameras.\n> > >>>>>>\n> > >>>>>> The hard part I think it is to define how to instruct the HAL it has\n> > >>>>>> to wait for built-in to appear or not before registering UVC\n> > >>>>>> non-active cameras.\n> > >>>>>\n> > >>>>> My suggestion is (not blocking this patch, but on top) to do so by\n> > >>>>> treating static cameras and hotpluggable cameras in the same way.\n> > >>>>>\n> > >>>>> It's just that static cameras will never 'unplug' (unless someone\n> > >>>>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> > >>>>\n> > >>>> As explained above, this is unfortunately not an option. To make this\n> > >>>> matter more complicated, we need to wait for *all* built-in cameras to\n> > >>>> be available before initializing, as otherwise the system will be\n> > >>>> permanently stuck with only part of the cameras available.\n> > >>>>\n> > >>>> I think this isn't something we should solve, the system should ensure\n> > >>>> that device nodes have been created before starting the camera service.\n> > >>>\n> > >>> Yes, given what you've clarified, then I agree - the service should be\n> > >>> dependent upon the devices being brought up. I think systemd can do\n> > >>> that, but CrOS uses upstart, so I don't know what that might support.\n> > >>\n> > >> As ugly as it sounds, we have an udev rule which restarts the camera\n> > >> service when devices matching the internal cameras show up. I believe\n> > >> we match them by the \"video4linux\" class and particular hardware\n> > >> subsystems, e.g. PCI or I2C.\n> > >\n> > > Has it always worked like that, or is it fairly recent ? If the service\n> > > is restarted when new devices appear, everything should be fine even if\n> > > the HAL initializes successfully with zero cameras, shouldn't it ?\n> >\n> > Yes, it's a recent change for a new platform where we ended up with\n> > dynamic detection of components. Actually it hasn't landed yet. The CL\n> > for reference is\n> > https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n>\n> Does this mean that if we integrate that change manually (including\n> possible dependencies), then no further action should be needed on the\n> HAL side ?\n\nYes, I believe so, although I recall some changes have been requested\nin the review, so it might change a bit.\n\nBest regards,\nTomasz","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BF3D3BD878\n\tfor ;\n\tMon, 27 Jul 2020 09:35:16 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2DD30612E1;\n\tMon, 27 Jul 2020 11:35:16 +0200 (CEST)","from mail-ej1-x629.google.com (mail-ej1-x629.google.com\n\t[IPv6:2a00:1450:4864:20::629])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B51136039B\n\tfor ;\n\tMon, 27 Jul 2020 11:35:14 +0200 (CEST)","by mail-ej1-x629.google.com with SMTP id f14so501623ejb.2\n\tfor ;\n\tMon, 27 Jul 2020 02:35:14 -0700 (PDT)","from mail-wm1-f53.google.com (mail-wm1-f53.google.com.\n\t[209.85.128.53]) by smtp.gmail.com with ESMTPSA id\n\tj7sm3662679ejb.64.2020.07.27.02.35.12\n\tfor \n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 27 Jul 2020 02:35:13 -0700 (PDT)","by mail-wm1-f53.google.com with SMTP id g8so2345592wmk.3\n\tfor ;\n\tMon, 27 Jul 2020 02:35:12 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"JCkHYPVR\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=YF3U/L1Ppt4NXWkiQsR7c9wdDmzYF65d5pEOciV9ZQc=;\n\tb=JCkHYPVRBrTOHJCoYaMga3+CS7/1rxmgLHz7yAPCxzVCyACa3oB0zLH0P+vGrWGbyO\n\txN6/RZHq3dSbyq3pPC2SFA4oz/fGYALwvR4yhgA1GpL8tvPAt40hMraNP8WR3YEmW49e\n\tQ8NAxHVAB0cyYwbCOTOr6MGm25GFFqQf48z6Q=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=YF3U/L1Ppt4NXWkiQsR7c9wdDmzYF65d5pEOciV9ZQc=;\n\tb=ngJSWGZPMD760+wWxdQUr6FjE1X0OFK82+q9KNAx3yasVAZjR2vwRelDXja0oCUQv0\n\tOXYAg3vFv00jOemUv0Ld0wD31cUlSuyTjB4fMqGl6X4+R7Vo8w4afkefUxOZSEuEr01Q\n\ti7rUUe0YVzcoLMdh3bZ5EcCYeYyHO3uGNe72lCtBnzRz//1SIQdiN3evMVUCH2SCrjrZ\n\tkgrNVX0/HyQTPOcA0U1qck2oqqTqRQbuSN9lutuNFJ0V84gLyNVZTAPOKKh0s4y1NYpK\n\tQhcxDlJokb0Mmy8u2JvQMQdlXXBmXqc7KCNBXFZdt/ehMDvcQnBRgLaNykx0QYp8iN1y\n\tzHeQ==","X-Gm-Message-State":"AOAM532HTxTHOuAMTyntrJF17vxdVxYobdoKNs1d7m3diwmI/fdP1nu1\n\tSwNWvr+ttwfC5LTLxkTMpJi7rNg/HgA=","X-Google-Smtp-Source":"ABdhPJwBZ5GTqDI/Jx+wWaMqKOavv6dn1zN8J9vKV3EQbgEALK3QnKC58yVjlnK7QzxHjgvyhMRoDg==","X-Received":["by 2002:a17:906:f290:: with SMTP id\n\tgu16mr8552886ejb.502.1595842513580; \n\tMon, 27 Jul 2020 02:35:13 -0700 (PDT)","by 2002:a1c:e382:: with SMTP id\n\ta124mr19806615wmh.11.1595842512085; \n\tMon, 27 Jul 2020 02:35:12 -0700 (PDT)"],"MIME-Version":"1.0","References":"<27d4a4f1-71aa-be54-3bb4-207171b47e42@ideasonboard.com>\n\t<20200721130958.yvot2wamrpnbgtge@uno.localdomain>\n\t\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t\n\t<20200724012831.GK21353@pendragon.ideasonboard.com>","In-Reply-To":"<20200724012831.GK21353@pendragon.ideasonboard.com>","From":"Tomasz Figa ","Date":"Mon, 27 Jul 2020 11:35:00 +0200","X-Gmail-Original-Message-ID":"","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11623,"web_url":"https://patchwork.libcamera.org/comment/11623/","msgid":"<20200727105205.GB5925@pendragon.ideasonboard.com>","date":"2020-07-27T10:52:05","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Tomasz,\n\nOn Mon, Jul 27, 2020 at 11:30:16AM +0200, Tomasz Figa wrote:\n> On Thu, Jul 23, 2020 at 11:43 AM Jacopo Mondi wrote:\n> > On Wed, Jul 22, 2020 at 08:10:03PM +0200, Tomasz Figa wrote:\n> >> On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart wrote:\n> >\n> > snip\n> >\n> >>>>>>>>> I was going to ask how does it register camera's it doesn't know will\n> >>>>>>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> >>>>>>>>> cameras or such, so that explains how that one works.\n> >>>>>>>>>\n> >>>>>>>>\n> >>>>>>>> I see three cases:\n> >>>>>>>> 1) No uvc support\n> >>>>>>>> Defer until a pipeline handler is matched and has registered\n> >>>>>>>> cameras\n> >>>>>>>>\n> >>>>>>>> 2) UVC only\n> >>>>>>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> >>>>>>>> this has to happen is tricky, as the HAL needs to know it should\n> >>>>>>>> not wait for built-in cameras and can proceed to register\n> >>>>>>>> non-active USB cameras\n> >>>>>>>>\n> >>>>>>>> 3) built-in + UVC\n> >>>>>>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> >>>>>>>> defer until cameras are available, then pre-register UVC cameras.\n> >>>>>>>>\n> >>>>>>>> The hard part I think it is to define how to instruct the HAL it has\n> >>>>>>>> to wait for built-in to appear or not before registering UVC\n> >>>>>>>> non-active cameras.\n> >>>>>>>\n> >>>>>>> My suggestion is (not blocking this patch, but on top) to do so by\n> >>>>>>> treating static cameras and hotpluggable cameras in the same way.\n> >>>>>>>\n> >>>>>>> It's just that static cameras will never 'unplug' (unless someone\n> >>>>>>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> >>>>>>\n> >>>>>> As explained above, this is unfortunately not an option. To make this\n> >>>>>> matter more complicated, we need to wait for *all* built-in cameras to\n> >>>>>> be available before initializing, as otherwise the system will be\n> >>>>>> permanently stuck with only part of the cameras available.\n> >>>>>>\n> >>>>>> I think this isn't something we should solve, the system should ensure\n> >>>>>> that device nodes have been created before starting the camera service.\n> >>>>>\n> >>>>> Yes, given what you've clarified, then I agree - the service should be\n> >>>>> dependent upon the devices being brought up. I think systemd can do\n> >>>>> that, but CrOS uses upstart, so I don't know what that might support.\n> >>>>\n> >>>> As ugly as it sounds, we have an udev rule which restarts the camera\n> >>>> service when devices matching the internal cameras show up. I believe\n> >>>> we match them by the \"video4linux\" class and particular hardware\n> >>>> subsystems, e.g. PCI or I2C.\n> >>>\n> >>> Has it always worked like that, or is it fairly recent ? If the service\n> >>> is restarted when new devices appear, everything should be fine even if\n> >>> the HAL initializes successfully with zero cameras, shouldn't it ?\n> >>\n> >> Yes, it's a recent change for a new platform where we ended up with\n> >> dynamic detection of components. Actually it hasn't landed yet. The CL\n> >> for reference is\n> >> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n> >\n> > Do you happen to know why I see this issue being triggered only for\n> > CTS which goes through the android services ? It goes through the\n> > cros_camera_service if I got the architecture right, but even without\n> > this patch, the camera service is always able to identify both\n> > cameras...\n> >\n> > It is still not clear to me if the android camera service (is this ARC++?)\n> > interacts with the cros_camera_service through the same mojo interface\n> > as stock ChromeOS camera application (and cros_camera_test?) use or\n> > there's a different interface for it..\n> \n> Yes, Android interacts with the cros_camera service using the same\n> interface. I think it's just that Android camera frameworks cache the\n> camera metadata when they query it the first time and would ignore any\n> later changes. On the contrary, Chrome probably just queries the\n> metadata every time some camera use case is executed.\n\nIS there a service that could be restarted to reset the Android camera\nframework state instead of rebooting the device ?","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C983FBD86F\n\tfor ;\n\tMon, 27 Jul 2020 10:52:15 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5D5F360536;\n\tMon, 27 Jul 2020 12:52:15 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B20D660536\n\tfor ;\n\tMon, 27 Jul 2020 12:52:13 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 21CFF2A9;\n\tMon, 27 Jul 2020 12:52:13 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"XBtSxBHI\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595847133;\n\tbh=7vcelZAGTC5c1rYEXqV34WGdXKiOFOSx7T8ljauEwYo=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=XBtSxBHI694d8/Cbe9fcjHOWbZshLdwgymsyxKC3m3JN4q5Xd1bfO/NJXAJYSDjqM\n\tcjZE+vPNCbexxa+tJPqYOsAN/JpEE84R0q9E3ACu4zdzVfjwUKZkGRNpJGGBNzMYRq\n\tA/zbqigF0qP/A1Bzncynbm4qPPxlnYTPuP7B0Rqs=","Date":"Mon, 27 Jul 2020 13:52:05 +0300","From":"Laurent Pinchart ","To":"Tomasz Figa ","Message-ID":"<20200727105205.GB5925@pendragon.ideasonboard.com>","References":"\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t\n\t<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11624,"web_url":"https://patchwork.libcamera.org/comment/11624/","msgid":"","date":"2020-07-27T11:00:12","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":9,"url":"https://patchwork.libcamera.org/api/people/9/","name":"Tomasz Figa","email":"tfiga@chromium.org"},"content":"On Mon, Jul 27, 2020 at 12:52 PM Laurent Pinchart\n wrote:\n>\n> Hi Tomasz,\n>\n> On Mon, Jul 27, 2020 at 11:30:16AM +0200, Tomasz Figa wrote:\n> > On Thu, Jul 23, 2020 at 11:43 AM Jacopo Mondi wrote:\n> > > On Wed, Jul 22, 2020 at 08:10:03PM +0200, Tomasz Figa wrote:\n> > >> On Wed, Jul 22, 2020 at 7:56 PM Laurent Pinchart wrote:\n> > >\n> > > snip\n> > >\n> > >>>>>>>>> I was going to ask how does it register camera's it doesn't know will\n> > >>>>>>>>> exist, but I think I saw that the UVC HAL just pre-allocates some\n> > >>>>>>>>> cameras or such, so that explains how that one works.\n> > >>>>>>>>>\n> > >>>>>>>>\n> > >>>>>>>> I see three cases:\n> > >>>>>>>> 1) No uvc support\n> > >>>>>>>> Defer until a pipeline handler is matched and has registered\n> > >>>>>>>> cameras\n> > >>>>>>>>\n> > >>>>>>>> 2) UVC only\n> > >>>>>>>> Need to pre-register cameras and activate on hot-plug. Knowing when\n> > >>>>>>>> this has to happen is tricky, as the HAL needs to know it should\n> > >>>>>>>> not wait for built-in cameras and can proceed to register\n> > >>>>>>>> non-active USB cameras\n> > >>>>>>>>\n> > >>>>>>>> 3) built-in + UVC\n> > >>>>>>>> Simmilarly, knowing we have to wait for built-in to appear, we\n> > >>>>>>>> defer until cameras are available, then pre-register UVC cameras.\n> > >>>>>>>>\n> > >>>>>>>> The hard part I think it is to define how to instruct the HAL it has\n> > >>>>>>>> to wait for built-in to appear or not before registering UVC\n> > >>>>>>>> non-active cameras.\n> > >>>>>>>\n> > >>>>>>> My suggestion is (not blocking this patch, but on top) to do so by\n> > >>>>>>> treating static cameras and hotpluggable cameras in the same way.\n> > >>>>>>>\n> > >>>>>>> It's just that static cameras will never 'unplug' (unless someone\n> > >>>>>>> unbinds them? but we all know how bad that mess would get with V4L2 anyway)\n> > >>>>>>\n> > >>>>>> As explained above, this is unfortunately not an option. To make this\n> > >>>>>> matter more complicated, we need to wait for *all* built-in cameras to\n> > >>>>>> be available before initializing, as otherwise the system will be\n> > >>>>>> permanently stuck with only part of the cameras available.\n> > >>>>>>\n> > >>>>>> I think this isn't something we should solve, the system should ensure\n> > >>>>>> that device nodes have been created before starting the camera service.\n> > >>>>>\n> > >>>>> Yes, given what you've clarified, then I agree - the service should be\n> > >>>>> dependent upon the devices being brought up. I think systemd can do\n> > >>>>> that, but CrOS uses upstart, so I don't know what that might support.\n> > >>>>\n> > >>>> As ugly as it sounds, we have an udev rule which restarts the camera\n> > >>>> service when devices matching the internal cameras show up. I believe\n> > >>>> we match them by the \"video4linux\" class and particular hardware\n> > >>>> subsystems, e.g. PCI or I2C.\n> > >>>\n> > >>> Has it always worked like that, or is it fairly recent ? If the service\n> > >>> is restarted when new devices appear, everything should be fine even if\n> > >>> the HAL initializes successfully with zero cameras, shouldn't it ?\n> > >>\n> > >> Yes, it's a recent change for a new platform where we ended up with\n> > >> dynamic detection of components. Actually it hasn't landed yet. The CL\n> > >> for reference is\n> > >> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n> > >\n> > > Do you happen to know why I see this issue being triggered only for\n> > > CTS which goes through the android services ? It goes through the\n> > > cros_camera_service if I got the architecture right, but even without\n> > > this patch, the camera service is always able to identify both\n> > > cameras...\n> > >\n> > > It is still not clear to me if the android camera service (is this ARC++?)\n> > > interacts with the cros_camera_service through the same mojo interface\n> > > as stock ChromeOS camera application (and cros_camera_test?) use or\n> > > there's a different interface for it..\n> >\n> > Yes, Android interacts with the cros_camera service using the same\n> > interface. I think it's just that Android camera frameworks cache the\n> > camera metadata when they query it the first time and would ignore any\n> > later changes. On the contrary, Chrome probably just queries the\n> > metadata every time some camera use case is executed.\n>\n> IS there a service that could be restarted to reset the Android camera\n> framework state instead of rebooting the device ?\n\nI'm not sure unfortunately. I think logging out and in again in Chrome\nOS should restart the Android container, but not sure if that improves\nanything. Also given that inside the container is just plain Android,\nmaybe there is some generic Android way to reload the camera HAL?","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C1EC4BD878\n\tfor ;\n\tMon, 27 Jul 2020 11:00:27 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 52A5F612F3;\n\tMon, 27 Jul 2020 13:00:27 +0200 (CEST)","from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com\n\t[IPv6:2a00:1450:4864:20::62d])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 776EB60536\n\tfor ;\n\tMon, 27 Jul 2020 13:00:26 +0200 (CEST)","by mail-ej1-x62d.google.com with SMTP id w9so16585109ejc.8\n\tfor ;\n\tMon, 27 Jul 2020 04:00:26 -0700 (PDT)","from mail-wr1-f43.google.com (mail-wr1-f43.google.com.\n\t[209.85.221.43]) by smtp.gmail.com with ESMTPSA id\n\tah1sm6731451ejc.43.2020.07.27.04.00.24\n\tfor \n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 27 Jul 2020 04:00:24 -0700 (PDT)","by mail-wr1-f43.google.com with SMTP id a14so14433394wra.5\n\tfor ;\n\tMon, 27 Jul 2020 04:00:24 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"VUxICQsw\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=Awrs78RiJQwFyhAcMZ5zaISxVjnBhuQA9xVdorDqvMY=;\n\tb=VUxICQswZxqV7euNCYZpgRMybQrKaFSuriws6eDPbs89lXTPkqTlwmWqjYW0knHOYx\n\tcAgJYwn3IEmsJDLQcVX4v9lpA+V9epWcwWGc4cgcQ3fmSxDr0TloYd5Rr/2hHxWC2rnU\n\tn+dyJOFsuiJdG8R4L7TWK2VIndu5oF4z86/00=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=Awrs78RiJQwFyhAcMZ5zaISxVjnBhuQA9xVdorDqvMY=;\n\tb=VobdIV1u+dMPHYN75XfbGNPxm4h0u0fUY7rC3VmG5oVFiBrgcATdo1zFcLLzyig7lz\n\tOhZEQm16agrS0+TUmySUF+EZFE7zbik/7nBLDEti6rZ6f5zW1B4dR5kTyQCg2FVp+LNV\n\tihNTLmWxV68oX9vYNM42emXF/66DWyOGk8tF5WOv8MU+W3K9JfCPNCYR30O5+nABj3xd\n\tJ4jlfWpqY+d+HOSxPMK8pH8np5WRhOrKNZWzcJ4kwKGXt6OQOYSzR4nhBDiBhuqYELmE\n\tndkqgOoRDYDgc+Db5ag8z2ftHnGsU/fSjv7KUPBAF0f6blLQWXpNaZWXGb+HjWJdKL4q\n\tcMsg==","X-Gm-Message-State":"AOAM531aHzM3sDwHmF1QLrj+qLHjs74eceZd3egZ0LuDHwt6FFSuax4O\n\tbc8HgAzYOYzpoiVG/40iahbpavnqMuxKRw==","X-Google-Smtp-Source":"ABdhPJxbubwNHwbjqRVC/Z2+sO4313JjDX5RyD3jMNiZqjAIsC7rFKYH9uB5iEum7f46+VigjeU/hA==","X-Received":["by 2002:a17:906:1813:: with SMTP id\n\tv19mr21352377eje.249.1595847625697; \n\tMon, 27 Jul 2020 04:00:25 -0700 (PDT)","by 2002:a5d:6744:: with SMTP id\n\tl4mr19012378wrw.105.1595847624214; \n\tMon, 27 Jul 2020 04:00:24 -0700 (PDT)"],"MIME-Version":"1.0","References":"\n\t<20200722111221.sezcchaymsnbhwgf@uno.localdomain>\n\t\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t\n\t<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>\n\t\n\t<20200727105205.GB5925@pendragon.ideasonboard.com>","In-Reply-To":"<20200727105205.GB5925@pendragon.ideasonboard.com>","From":"Tomasz Figa ","Date":"Mon, 27 Jul 2020 13:00:12 +0200","X-Gmail-Original-Message-ID":"","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11669,"web_url":"https://patchwork.libcamera.org/comment/11669/","msgid":"<20200728141229.x7zs6uobaxz2cot2@uno.localdomain>","date":"2020-07-28T14:12:29","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hello,\n\nOn Mon, Jul 27, 2020 at 01:00:12PM +0200, Tomasz Figa wrote:\n> On Mon, Jul 27, 2020 at 12:52 PM Laurent Pinchart\n> wrote:\n> > > >>\n> > > >> Yes, it's a recent change for a new platform where we ended up with\n> > > >> dynamic detection of components. Actually it hasn't landed yet. The CL\n> > > >> for reference is\n> > > >> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n\nFWIW I tested\nhttps://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080\napplied to libcamera and not to ipu6, with\nhttps://chromium-review.googlesource.com/c/chromiumos/platform2/+/2239846/2/camera/hal_adapter/init/cros-camera.conf\n\nand I have two cameras registered at ARC++ start up.\n\nI think we can drop this patch and just keep it as a local fix to run\nCTS or Android camera applications, and drop it when the above change\nlands in upstream chromium\n\n\n> > > >\n> > > > Do you happen to know why I see this issue being triggered only for\n> > > > CTS which goes through the android services ? It goes through the\n> > > > cros_camera_service if I got the architecture right, but even without\n> > > > this patch, the camera service is always able to identify both\n> > > > cameras...\n> > > >\n> > > > It is still not clear to me if the android camera service (is this ARC++?)\n> > > > interacts with the cros_camera_service through the same mojo interface\n> > > > as stock ChromeOS camera application (and cros_camera_test?) use or\n> > > > there's a different interface for it..\n> > >\n> > > Yes, Android interacts with the cros_camera service using the same\n> > > interface. I think it's just that Android camera frameworks cache the\n> > > camera metadata when they query it the first time and would ignore any\n> > > later changes. On the contrary, Chrome probably just queries the\n> > > metadata every time some camera use case is executed.\n> >\n> > IS there a service that could be restarted to reset the Android camera\n> > framework state instead of rebooting the device ?\n>\n> I'm not sure unfortunately. I think logging out and in again in Chrome\n> OS should restart the Android container, but not sure if that improves\n> anything. Also given that inside the container is just plain Android,\n> maybe there is some generic Android way to reload the camera HAL?","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 584F8BD878\n\tfor ;\n\tTue, 28 Jul 2020 14:08:52 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BEB09613C6;\n\tTue, 28 Jul 2020 16:08:51 +0200 (CEST)","from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net\n\t[217.70.183.197])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E940260923\n\tfor ;\n\tTue, 28 Jul 2020 16:08:50 +0200 (CEST)","from uno.localdomain (2-224-242-101.ip172.fastwebnet.it\n\t[2.224.242.101]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay5-d.mail.gandi.net (Postfix) with ESMTPSA id D229C1C0015;\n\tTue, 28 Jul 2020 14:08:49 +0000 (UTC)"],"X-Originating-IP":"2.224.242.101","Date":"Tue, 28 Jul 2020 16:12:29 +0200","From":"Jacopo Mondi ","To":"Tomasz Figa ","Message-ID":"<20200728141229.x7zs6uobaxz2cot2@uno.localdomain>","References":"\n\t<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t\n\t<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>\n\t\n\t<20200727105205.GB5925@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11678,"web_url":"https://patchwork.libcamera.org/comment/11678/","msgid":"<20200728192912.GA19622@pendragon.ideasonboard.com>","date":"2020-07-28T19:29:12","subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn Tue, Jul 28, 2020 at 04:12:29PM +0200, Jacopo Mondi wrote:\n> On Mon, Jul 27, 2020 at 01:00:12PM +0200, Tomasz Figa wrote:\n> > On Mon, Jul 27, 2020 at 12:52 PM Laurent Pinchart wrote:\n> > > > >>\n> > > > >> Yes, it's a recent change for a new platform where we ended up with\n> > > > >> dynamic detection of components. Actually it hasn't landed yet. The CL\n> > > > >> for reference is\n> > > > >> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080.\n> \n> FWIW I tested\n> https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/2239080\n> applied to libcamera and not to ipu6, with\n> https://chromium-review.googlesource.com/c/chromiumos/platform2/+/2239846/2/camera/hal_adapter/init/cros-camera.conf\n> \n> and I have two cameras registered at ARC++ start up.\n> \n> I think we can drop this patch and just keep it as a local fix to run\n> CTS or Android camera applications, and drop it when the above change\n> lands in upstream chromium\n\nNice !! Thank you for testing.\n\n> > > > > Do you happen to know why I see this issue being triggered only for\n> > > > > CTS which goes through the android services ? It goes through the\n> > > > > cros_camera_service if I got the architecture right, but even without\n> > > > > this patch, the camera service is always able to identify both\n> > > > > cameras...\n> > > > >\n> > > > > It is still not clear to me if the android camera service (is this ARC++?)\n> > > > > interacts with the cros_camera_service through the same mojo interface\n> > > > > as stock ChromeOS camera application (and cros_camera_test?) use or\n> > > > > there's a different interface for it..\n> > > >\n> > > > Yes, Android interacts with the cros_camera service using the same\n> > > > interface. I think it's just that Android camera frameworks cache the\n> > > > camera metadata when they query it the first time and would ignore any\n> > > > later changes. On the contrary, Chrome probably just queries the\n> > > > metadata every time some camera use case is executed.\n> > >\n> > > IS there a service that could be restarted to reset the Android camera\n> > > framework state instead of rebooting the device ?\n> >\n> > I'm not sure unfortunately. I think logging out and in again in Chrome\n> > OS should restart the Android container, but not sure if that improves\n> > anything. Also given that inside the container is just plain Android,\n> > maybe there is some generic Android way to reload the camera HAL?","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3F199BD878\n\tfor ;\n\tTue, 28 Jul 2020 19:29:23 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C94EB617AF;\n\tTue, 28 Jul 2020 21:29:22 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 197BA60923\n\tfor ;\n\tTue, 28 Jul 2020 21:29:22 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6E30F563;\n\tTue, 28 Jul 2020 21:29:21 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"jnHNswJZ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1595964561;\n\tbh=H2/hRblWQ+moQzRylxdL/id1NQoPg99j2E5bwFfH6j0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=jnHNswJZ2Esm2XA9poAj5ONVxkRKGCmEDx+BGvy2qiUpcFffWRWX6AZ3O42MRuaaO\n\tEhl46MbW6ZXujF4m5xdLzdxB20ZTvaJRQ/dm3057Xajg4rWJdkgMwZD1SMj+03giWs\n\tAKopEguvgXpEvF3gpT2O7dMy9w1sWOYpzena4AAs=","Date":"Tue, 28 Jul 2020 22:29:12 +0300","From":"Laurent Pinchart ","To":"Jacopo Mondi ","Message-ID":"<20200728192912.GA19622@pendragon.ideasonboard.com>","References":"<20200722134940.GL5833@pendragon.ideasonboard.com>\n\t<376a2943-cec3-649b-6c83-5c8539f3c4d5@ideasonboard.com>\n\t\n\t<20200722175630.GQ29813@pendragon.ideasonboard.com>\n\t\n\t<20200723094654.jvmu3drgu6uht2ee@uno.localdomain>\n\t\n\t<20200727105205.GB5925@pendragon.ideasonboard.com>\n\t\n\t<20200728141229.x7zs6uobaxz2cot2@uno.localdomain>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200728141229.x7zs6uobaxz2cot2@uno.localdomain>","Subject":"Re: [libcamera-devel] [PATCH] android: camera_hal_manager: Fail on\n\tno cameras","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]