[{"id":11980,"web_url":"https://patchwork.libcamera.org/comment/11980/","msgid":"<91d8710b-fd54-0ac0-b37c-8288620b6986@ideasonboard.com>","date":"2020-08-11T15:02:44","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Umang,\n\nOn 10/08/2020 13:04, Umang Jain wrote:\n> Extend the support for camera hotplug from libcamera's CameraManager\n> to CameraHalManager. Use camera module callbacks to let the framework\n> know about the hotplug events and change the status of cameras being\n> being hotplugged or unplugged via camera_device_status_change().\n> \n> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> past by the CameraHalManager. If the camera is seen for the first time,\n> a new id is assigned to it. If the camera has been seen before by the\n> manager, it's old id is reused. IDs for internal cameras start with\n> '0' and for external cameras, they start with '1000'. Note, for the\n> current implementation, we assume all UVC cameras are external cameras.\n> \n> CameraDevice is now a shared object and cameras_ vector stores shared\n> pointers to CameraDevice. This is done in order to introduce reference\n> counting for CameraDevice objects - especially to handle hot-unplug\n> events. Both camerasMap_ and cameras_ are protected by a mutex.\n> \n> Signed-off-by: Umang Jain \n> ---\n> src/android/camera_device.cpp | 15 +++\n> src/android/camera_device.h | 8 +-\n> src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> src/android/camera_hal_manager.h | 15 ++-\n> 4 files changed, 166 insertions(+), 25 deletions(-)\n> \n> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> index d918350..a79bb69 100644\n> --- a/src/android/camera_device.cpp\n> +++ b/src/android/camera_device.cpp\n> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> \t\tdelete it.second;\n> }\n> \n> +std::shared_ptr CameraDevice::create(unsigned int id,\n> +\t\t\t\t\t\t const std::shared_ptr &cam)\n> +{\n> +\tstruct Deleter : std::default_delete {\n> +\t\tvoid operator()(CameraDevice *camera)\n> +\t\t{\n> +\t\t\tdelete camera;\n> +\t\t}\n> +\t};\n> +\n> +\tCameraDevice *camera = new CameraDevice(id, cam);\n> +\n> +\treturn std::shared_ptr(camera, Deleter());\n> +}\n> +\n\nSplitting out the making CameraDevice a shared_ptr might have been\nhelpful in it's own patch.\n\nIt's a single unified change,and would make the actual hotplug code\nchanges easier to identify.\n\n\n> /*\n> * Initialize the camera static information.\n> * This method is called before the camera device is opened.\n> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> index 7be9e11..7f9e010 100644\n> --- a/src/android/camera_device.h\n> +++ b/src/android/camera_device.h\n> @@ -47,8 +47,8 @@ struct CameraStream {\n> class CameraDevice : protected libcamera::Loggable\n> {\n> public:\n> -\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> -\t~CameraDevice();\n> +\tstatic std::shared_ptr create(unsigned int id,\n> +\t\t\t\t\t\t const std::shared_ptr &cam);\n> \n> \tint initialize();\n> \n> @@ -57,6 +57,7 @@ public:\n> \n> \tunsigned int id() const { return id_; }\n> \tcamera3_device_t *camera3Device() { return &camera3Device_; }\n> +\tconst libcamera::Camera *getCamera() { return camera_.get(); };\n> \n> \tint facing() const { return facing_; }\n> \tint orientation() const { return orientation_; }\n> @@ -72,6 +73,9 @@ protected:\n> \tstd::string logPrefix() const override;\n> \n> private:\n> +\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> +\t~CameraDevice();\n> +\n> \tstruct Camera3RequestDescriptor {\n> \t\tCamera3RequestDescriptor(unsigned int frameNumber,\n> \t\t\t\t\t unsigned int numBuffers);\n> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> index 3d6d2b4..fdde2c0 100644\n> --- a/src/android/camera_hal_manager.cpp\n> +++ b/src/android/camera_hal_manager.cpp\n> @@ -8,6 +8,7 @@\n> #include \"camera_hal_manager.h\"\n> \n> #include \n> +#include \n> \n> #include \"libcamera/internal/log.h\"\n> \n> @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> CameraHalManager::~CameraHalManager()\n> {\n> \tcameras_.clear();\n> +\tcamerasMap_.clear();\n> \n> \tif (cameraManager_) {\n> \t\tcameraManager_->stop();\n> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> {\n> \tcameraManager_ = new CameraManager();\n> \n> +\t/* Support camera hotplug. */\n> +\tcameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> +\tcameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> +\n> +\tcameraCounter_ = 0;\n> +\texternalCameraCounter_ = 1000;\n> +\n> \tint ret = cameraManager_->start();\n> \tif (ret) {\n> \t\tLOG(HAL, Error) << \"Failed to start camera manager: \"\n> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> \t\treturn ret;\n> \t}\n> \n> -\t/*\n> -\t * For each Camera registered in the system, a CameraDevice\n> -\t * gets created here to wraps a libcamera Camera instance.\n> -\t *\n> -\t * \\todo Support camera hotplug.\n> -\t */\n> -\tunsigned int index = 0;\n> -\tfor (auto &cam : cameraManager_->cameras()) {\n> -\t\tCameraDevice *camera = new CameraDevice(index, cam);\n> -\t\tret = camera->initialize();\n> -\t\tif (ret)\n> -\t\t\tcontinue;\n> -\n> -\t\tcameras_.emplace_back(camera);\n> -\t\t++index;\n> -\t}\n> -\n> \treturn 0;\n> }\n> \n> CameraDevice *CameraHalManager::open(unsigned int id,\n> \t\t\t\t const hw_module_t *hardwareModule)\n> {\n> -\tif (id >= numCameras()) {\n> +\tMutexLocker locker(mutex_);\n> +\n> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> +\t\t\t\t [id](std::shared_ptr &cam) {\n> +\t\t\t\t\treturn cam->id() == id;\n> +\t\t\t\t});\n> +\tif (iter == cameras_.end()) {\n> \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> \t\treturn nullptr;\n> \t}\n> \n> -\tCameraDevice *camera = cameras_[id].get();\n> +\tCameraDevice *camera = iter->get();\n> +\n> \tif (camera->open(hardwareModule))\n> \t\treturn nullptr;\n> \n> @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> \treturn camera;\n> }\n> \n> +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> +{\n> +\tunsigned int id;\n> +\tbool isCameraExternal = false;\n> +\tbool isCameraNew = false;\n> +\n> +\tMutexLocker locker(mutex_);\n> +\n> +\t/* Each camera is assigned a unique integer id when it is seen for the\n\nBlock comments are opened with an empty line:\n\n\t/*\n\t * Each camera ...\n\nI guess I should really get round to doing the checkpatch style checker\nfor comments. I don't want to turn in to Laurent ;-) hehe.\n\n\n> +\t * first time. If the camera has been seen before, the id is reused and\n> +\t * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> +\t *\n> +\t * ID starts from '0' for internal cameras and '1000' for external cameras.\n\nI'd pluralise this at ID\n\n : IDs start from ...\n\n\n> +\t */\n> +\tauto iter = camerasMap_.find(cam->id());\n> +\tif (iter != camerasMap_.end()) {\n> +\t\tid = iter->second;\n> +\t} else {\n> +\t\tisCameraNew = true;\n> +\n> +\t\t/*\n> +\t\t * Now check if this is an external camera and assign\n> +\t\t * its id accordingly.\n> +\t\t */\n> +\t\tconst ControlList &properties = cam->properties();\n> +\t\tif (properties.contains(properties::Location) &&\n> +\t\t properties.get(properties::Location) &\n\nShould that be & or == ?\n\nCan cameras have multiple locations?\n\n> +\t\t properties::CameraLocationExternal) {\n> +\t\t\tisCameraExternal = true;\n> +\t\t\tid = externalCameraCounter_;\n> +\t\t} else {\n> +\t\t\tid = cameraCounter_;\n> +\t\t}\n> +\t}\n> +\n> +\t/*\n> +\t * For each Camera registered in the system, a CameraDevice\n> +\t * gets created here to wraps a libcamera Camera instance.\n> +\t */\n> +\tstd::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> +\tint ret = camera->initialize();\n> +\tif (ret) {\n> +\t\tLOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> +\t\treturn;\n> +\t}\n> +\n> +\tif (isCameraNew) {\n> +\t\tcamerasMap_.emplace(cam->id(), id);\n> +\n> +\t\tif (isCameraExternal)\n> +\t\t\texternalCameraCounter_++;\n> +\t\telse\n> +\t\t\tcameraCounter_++;\n> +\t}\n> +\n> +\tcameras_.emplace_back(std::move(camera));\n> +\n> +\tif (callbacks_)\n> +\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> +\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n\nI'd add a new line here before the debug print to keep it separate.\n\n\n> +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> +}\n> +\n> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> +{\n> +\tMutexLocker locker(mutex_);\n> +\n> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> +\t\t\t\t [cam](std::shared_ptr &camera) {\n> +\t\t\t\t\treturn cam.get() == camera->getCamera();\n> +\t\t\t\t});\n> +\tif (iter == cameras_.end())\n> +\t\treturn;\n> +\n> +\tunsigned int id = (*iter)->id();\n> +\tcallbacks_->camera_device_status_change(callbacks_, id,\n> +\t\t\t\t\t\tCAMERA_DEVICE_STATUS_NOT_PRESENT);\n> +\tcameras_.erase(iter);\n> +\n> +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> +}\n> +\n> unsigned int CameraHalManager::numCameras() const\n> {\n> -\treturn cameraManager_->cameras().size();\n> +\treturn cameraCounter_;\n\nIs this right?\n\nThis won't count external cameras which are connected at the time\nlibrary startup.\n\nBut perhaps it's only meant to count 'internal' cameras anyway...#\n\nI think cameraCounter_ probably needs to be named\n internalCameras_\n\nor such maybe, as 'cameraCounter_' to me counts all cameras including\nexternal ones...\n\n\n\n> }\n> \n> int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> \tif (!info)\n> \t\treturn -EINVAL;\n> \n> -\tif (id >= numCameras()) {\n> +\tMutexLocker locker(mutex_);\n> +\n> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> +\t\t\t\t [id](std::shared_ptr &cam) {\n> +\t\t\t\t\treturn cam->id() == id;\n> +\t\t\t\t});\n> +\tif (iter == cameras_.end()) {\n> \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> \t\treturn -EINVAL;\n> \t}\n> \n> -\tCameraDevice *camera = cameras_[id].get();\n> +\tCameraDevice *camera = iter->get();\n> \n> \tinfo->facing = camera->facing();\n> \tinfo->orientation = camera->orientation();\n> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> {\n> \tcallbacks_ = callbacks;\n> +\n> +\tMutexLocker locker(mutex_);\n> +\t/*\n> +\t * Few cameras might have been hotplugged before setting callbacks_ here.\n> +\t * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> +\t * This hold only for external cameras, as internal cameras are assumed to\n> +\t * be present at module load time, by the framework.\n\n\nJust to try to fix some of the grammar/flow here:\n\n\t/*\n\t * Some external cameras may have been identified before the\n\t * callbacks_ were enabled. Iterate any existing external\n\t * cameras and mark them as CAMERA_DEVICE_STATUS_PRESENT\n\t * explicitly.\n\t *\n\t * Internal cameras are already assumed to be present at module\n\t * load time by the android framework.\n\t */\n\n\nIs this an actual possibility?\nI assume it's just a race... so not expected.\n\n\n> +\t */\n> +\tfor (auto &cam : cameraManager_->cameras()) {\n> +\t\tauto iter = camerasMap_.find(cam->id());\n> +\t\tif (iter == camerasMap_.end())\n> +\t\t\tcontinue;\n> +\n> +\t\tunsigned int id = iter->second;\n> +\t\tconst ControlList &properties = cam->properties();\n> +\t\tif (properties.contains(properties::Location) &&\n> +\t\t properties.get(properties::Location) &\n> +\t\t properties::CameraLocationExternal) {\n> +\t\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> +\t\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> +\t\t}\n> +\t}\n> }\n> diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> index a582f04..7c481d4 100644\n> --- a/src/android/camera_hal_manager.h\n> +++ b/src/android/camera_hal_manager.h\n> @@ -7,6 +7,8 @@\n> #ifndef __ANDROID_CAMERA_MANAGER_H__\n> #define __ANDROID_CAMERA_MANAGER_H__\n> \n> +#include \n> +#include \n> #include \n> #include \n> \n> @@ -18,6 +20,9 @@\n> \n> class CameraDevice;\n> \n> +using Mutex = std::mutex;\n> +using MutexLocker = std::unique_lock;\n\nPerhaps it doesn't matter too much, but should those be in the class\nCameraHalManager scope?\n\nI guess it depends on who includes this file.\n\n\n> +\n> class CameraHalManager\n> {\n> public:\n> @@ -33,10 +38,18 @@ public:\n> \tvoid setCallbacks(const camera_module_callbacks_t *callbacks);\n> \n> private:\n> +\tvoid cameraAdded(std::shared_ptr cam);\n> +\tvoid cameraRemoved(std::shared_ptr cam);\n> +\n> \tlibcamera::CameraManager *cameraManager_;\n> \n> \tconst camera_module_callbacks_t *callbacks_;\n> -\tstd::vector> cameras_;\n> +\tstd::vector> cameras_;\n> +\tstd::map camerasMap_;\n> +\tMutex mutex_;\n> +\n> +\tunsigned int externalCameraCounter_;\n> +\tunsigned int cameraCounter_;\n> };\n> \n> #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4716EBD87D\n\tfor ;\n\tTue, 11 Aug 2020 15:02:49 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BC67C61094;\n\tTue, 11 Aug 2020 17:02:48 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E0476055A\n\tfor ;\n\tTue, 11 Aug 2020 17:02:47 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 469D69A8;\n\tTue, 11 Aug 2020 17:02:46 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vZlqGsNt\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597158166;\n\tbh=oUP9y77EQ3S6hYRANE9J2CDcUaIDk2LFpTDdKFzXBWU=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=vZlqGsNtNk2duFJGWXirk6qNxqTgXBXXAR8xvjgBTA/Lt7INg1fnh/yOo/DOjEcL/\n\tDwho4whzL0GqqUrD2crfM1dbT/vY7i31ZCvqBiVRV2Bh7IF3w7ilb9taJV0vFZ0IOB\n\tJC7Gsbjj14q7D1nIH1cTYhzR2FbaB1PDNnf+gne4=","To":"Umang Jain , libcamera-devel@lists.libcamera.org","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<91d8710b-fd54-0ac0-b37c-8288620b6986@ideasonboard.com>","Date":"Tue, 11 Aug 2020 16:02:44 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200810120406.52654-4-email@uajain.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11982,"web_url":"https://patchwork.libcamera.org/comment/11982/","msgid":"<20200813114418.GE1061689@oden.dyn.berto.se>","date":"2020-08-13T11:44:18","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/people/5/","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"content":"Hi Umang,\n\nThanks for your work.\n\nOn 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> Extend the support for camera hotplug from libcamera's CameraManager\n> to CameraHalManager. Use camera module callbacks to let the framework\n> know about the hotplug events and change the status of cameras being\n> being hotplugged or unplugged via camera_device_status_change().\n> \n> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> past by the CameraHalManager. If the camera is seen for the first time,\n> a new id is assigned to it. If the camera has been seen before by the\n> manager, it's old id is reused. IDs for internal cameras start with\n> '0' and for external cameras, they start with '1000'. Note, for the\n> current implementation, we assume all UVC cameras are external cameras.\n\nI wonder if keeping the cache of previously seen cameras or if we should \nnot just treat any hot-plugged camera as a new one? I can't think of any \nbenefit of preserving the same numerical ID between two plug events, \nwhile I can think of quiet a few cons.\n\n- It's confusing when debugging as un-plugging and then replugging the \n same camera will result in logs where the numerical ID is the same for \n both. This may even result in things working by \"chance\" is it reuses \n an already known numerical ID.\n\n- Looking at the code plugging a UVC camera in a different USB port will \n generate a different Camera::id() result and then be treated as a new \n camera vs plugging it in the same port and it then being treatad as a \n new camera.\n\n- The logic in this patch is more complex due to it both having to deal \n with known and new cameras.\n\nWhat is the benefit of this cache that I'm missing?\n\n> \n> CameraDevice is now a shared object and cameras_ vector stores shared\n> pointers to CameraDevice. This is done in order to introduce reference\n> counting for CameraDevice objects - especially to handle hot-unplug\n> events. Both camerasMap_ and cameras_ are protected by a mutex.\n> \n> Signed-off-by: Umang Jain \n> ---\n> src/android/camera_device.cpp | 15 +++\n> src/android/camera_device.h | 8 +-\n> src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> src/android/camera_hal_manager.h | 15 ++-\n> 4 files changed, 166 insertions(+), 25 deletions(-)\n> \n> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> index d918350..a79bb69 100644\n> --- a/src/android/camera_device.cpp\n> +++ b/src/android/camera_device.cpp\n> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> \t\tdelete it.second;\n> }\n> \n> +std::shared_ptr CameraDevice::create(unsigned int id,\n> +\t\t\t\t\t\t const std::shared_ptr &cam)\n> +{\n> +\tstruct Deleter : std::default_delete {\n> +\t\tvoid operator()(CameraDevice *camera)\n> +\t\t{\n> +\t\t\tdelete camera;\n> +\t\t}\n> +\t};\n> +\n> +\tCameraDevice *camera = new CameraDevice(id, cam);\n> +\n> +\treturn std::shared_ptr(camera, Deleter());\n> +}\n\nAs Kieran points out I think this should be added in a separate as this \none is quiet large and therefore hard to review.\n\n> +\n> /*\n> * Initialize the camera static information.\n> * This method is called before the camera device is opened.\n> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> index 7be9e11..7f9e010 100644\n> --- a/src/android/camera_device.h\n> +++ b/src/android/camera_device.h\n> @@ -47,8 +47,8 @@ struct CameraStream {\n> class CameraDevice : protected libcamera::Loggable\n> {\n> public:\n> -\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> -\t~CameraDevice();\n> +\tstatic std::shared_ptr create(unsigned int id,\n> +\t\t\t\t\t\t const std::shared_ptr &cam);\n> \n> \tint initialize();\n> \n> @@ -57,6 +57,7 @@ public:\n> \n> \tunsigned int id() const { return id_; }\n> \tcamera3_device_t *camera3Device() { return &camera3Device_; }\n> +\tconst libcamera::Camera *getCamera() { return camera_.get(); };\n\nSame here this can be done in a separate commit. Also I think this could \nbe named camera() instead of getCamera()\n\n> \n> \tint facing() const { return facing_; }\n> \tint orientation() const { return orientation_; }\n> @@ -72,6 +73,9 @@ protected:\n> \tstd::string logPrefix() const override;\n> \n> private:\n> +\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> +\t~CameraDevice();\n> +\n> \tstruct Camera3RequestDescriptor {\n> \t\tCamera3RequestDescriptor(unsigned int frameNumber,\n> \t\t\t\t\t unsigned int numBuffers);\n> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> index 3d6d2b4..fdde2c0 100644\n> --- a/src/android/camera_hal_manager.cpp\n> +++ b/src/android/camera_hal_manager.cpp\n> @@ -8,6 +8,7 @@\n> #include \"camera_hal_manager.h\"\n> \n> #include \n> +#include \n> \n> #include \"libcamera/internal/log.h\"\n> \n> @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> CameraHalManager::~CameraHalManager()\n> {\n> \tcameras_.clear();\n> +\tcamerasMap_.clear();\n> \n> \tif (cameraManager_) {\n> \t\tcameraManager_->stop();\n> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> {\n> \tcameraManager_ = new CameraManager();\n> \n> +\t/* Support camera hotplug. */\n> +\tcameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> +\tcameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> +\n> +\tcameraCounter_ = 0;\n> +\texternalCameraCounter_ = 1000;\n> +\n> \tint ret = cameraManager_->start();\n> \tif (ret) {\n> \t\tLOG(HAL, Error) << \"Failed to start camera manager: \"\n> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> \t\treturn ret;\n> \t}\n> \n> -\t/*\n> -\t * For each Camera registered in the system, a CameraDevice\n> -\t * gets created here to wraps a libcamera Camera instance.\n> -\t *\n> -\t * \\todo Support camera hotplug.\n> -\t */\n> -\tunsigned int index = 0;\n> -\tfor (auto &cam : cameraManager_->cameras()) {\n> -\t\tCameraDevice *camera = new CameraDevice(index, cam);\n> -\t\tret = camera->initialize();\n> -\t\tif (ret)\n> -\t\t\tcontinue;\n> -\n> -\t\tcameras_.emplace_back(camera);\n> -\t\t++index;\n> -\t}\n> -\n> \treturn 0;\n> }\n> \n> CameraDevice *CameraHalManager::open(unsigned int id,\n> \t\t\t\t const hw_module_t *hardwareModule)\n> {\n> -\tif (id >= numCameras()) {\n> +\tMutexLocker locker(mutex_);\n> +\n> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> +\t\t\t\t [id](std::shared_ptr &cam) {\n> +\t\t\t\t\treturn cam->id() == id;\n> +\t\t\t\t});\n> +\tif (iter == cameras_.end()) {\n\nI think you should break this (and the similar ones below) into private \nhelpers instead if implementing the logic in-place\n\n CameraHalManager::cameraFromAndroidId(..);\n CameraHalManager::androidIdFromCamera(..);\n\n> \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> \t\treturn nullptr;\n> \t}\n> \n> -\tCameraDevice *camera = cameras_[id].get();\n> +\tCameraDevice *camera = iter->get();\n> +\n> \tif (camera->open(hardwareModule))\n> \t\treturn nullptr;\n> \n> @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> \treturn camera;\n> }\n> \n> +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> +{\n> +\tunsigned int id;\n> +\tbool isCameraExternal = false;\n> +\tbool isCameraNew = false;\n> +\n> +\tMutexLocker locker(mutex_);\n> +\n> +\t/* Each camera is assigned a unique integer id when it is seen for the\n> +\t * first time. If the camera has been seen before, the id is reused and\n> +\t * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> +\t *\n> +\t * ID starts from '0' for internal cameras and '1000' for external cameras.\n> +\t */\n> +\tauto iter = camerasMap_.find(cam->id());\n> +\tif (iter != camerasMap_.end()) {\n> +\t\tid = iter->second;\n> +\t} else {\n> +\t\tisCameraNew = true;\n> +\n> +\t\t/*\n> +\t\t * Now check if this is an external camera and assign\n> +\t\t * its id accordingly.\n> +\t\t */\n> +\t\tconst ControlList &properties = cam->properties();\n> +\t\tif (properties.contains(properties::Location) &&\n> +\t\t properties.get(properties::Location) &\n> +\t\t properties::CameraLocationExternal) {\n> +\t\t\tisCameraExternal = true;\n> +\t\t\tid = externalCameraCounter_;\n> +\t\t} else {\n> +\t\t\tid = cameraCounter_;\n> +\t\t}\n\nAs I understand it the information the HAL wants is whether or not the \ncamera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n\nIf so I wonder if using the camera location as a judgment of the camera \nis hot-plugged or not is the way to go here? Imagine a device where the \ncamera is permanently attached (not hot-unpluggable) but not fixated in \na location. I'm thinking cameras mounted at the end of instruments \n(medical instruments, hand held tools) or robotics (mounted at the arm \nof a welding robot). I would imagine those cameras would be marked as \nlocated externally but they would not really be hot-pluggable.\n\nI understand we have no other way to report or detect this at the moment \nand I'm not pushing hard for this to be solved as part of this series if \nit's not easy. But I think a bigger comment here is needed explaining \nthat the HAL wants to know if a camera is hot-pluggable or not and does \nnot really care if it's located internally or externally. I also think a \n\\todo should be added so it's not forgotten.\n\n> +\t}\n> +\n> +\t/*\n> +\t * For each Camera registered in the system, a CameraDevice\n> +\t * gets created here to wraps a libcamera Camera instance.\n> +\t */\n> +\tstd::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> +\tint ret = camera->initialize();\n> +\tif (ret) {\n> +\t\tLOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> +\t\treturn;\n> +\t}\n> +\n> +\tif (isCameraNew) {\n> +\t\tcamerasMap_.emplace(cam->id(), id);\n> +\n> +\t\tif (isCameraExternal)\n> +\t\t\texternalCameraCounter_++;\n> +\t\telse\n> +\t\t\tcameraCounter_++;\n> +\t}\n> +\n> +\tcameras_.emplace_back(std::move(camera));\n> +\n> +\tif (callbacks_)\n> +\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> +\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> +}\n> +\n> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> +{\n> +\tMutexLocker locker(mutex_);\n> +\n> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> +\t\t\t\t [cam](std::shared_ptr &camera) {\n> +\t\t\t\t\treturn cam.get() == camera->getCamera();\n> +\t\t\t\t});\n> +\tif (iter == cameras_.end())\n> +\t\treturn;\n> +\n> +\tunsigned int id = (*iter)->id();\n> +\tcallbacks_->camera_device_status_change(callbacks_, id,\n> +\t\t\t\t\t\tCAMERA_DEVICE_STATUS_NOT_PRESENT);\n> +\tcameras_.erase(iter);\n> +\n> +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> +}\n> +\n> unsigned int CameraHalManager::numCameras() const\n> {\n> -\treturn cameraManager_->cameras().size();\n> +\treturn cameraCounter_;\n> }\n> \n> int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> \tif (!info)\n> \t\treturn -EINVAL;\n> \n> -\tif (id >= numCameras()) {\n> +\tMutexLocker locker(mutex_);\n> +\n> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> +\t\t\t\t [id](std::shared_ptr &cam) {\n> +\t\t\t\t\treturn cam->id() == id;\n> +\t\t\t\t});\n> +\tif (iter == cameras_.end()) {\n> \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> \t\treturn -EINVAL;\n> \t}\n> \n> -\tCameraDevice *camera = cameras_[id].get();\n> +\tCameraDevice *camera = iter->get();\n> \n> \tinfo->facing = camera->facing();\n> \tinfo->orientation = camera->orientation();\n> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> {\n> \tcallbacks_ = callbacks;\n> +\n> +\tMutexLocker locker(mutex_);\n> +\t/*\n> +\t * Few cameras might have been hotplugged before setting callbacks_ here.\n> +\t * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> +\t * This hold only for external cameras, as internal cameras are assumed to\n> +\t * be present at module load time, by the framework.\n> +\t */\n> +\tfor (auto &cam : cameraManager_->cameras()) {\n> +\t\tauto iter = camerasMap_.find(cam->id());\n> +\t\tif (iter == camerasMap_.end())\n> +\t\t\tcontinue;\n> +\n> +\t\tunsigned int id = iter->second;\n> +\t\tconst ControlList &properties = cam->properties();\n> +\t\tif (properties.contains(properties::Location) &&\n> +\t\t properties.get(properties::Location) &\n> +\t\t properties::CameraLocationExternal) {\n> +\t\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> +\t\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> +\t\t}\n> +\t}\n> }\n> diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> index a582f04..7c481d4 100644\n> --- a/src/android/camera_hal_manager.h\n> +++ b/src/android/camera_hal_manager.h\n> @@ -7,6 +7,8 @@\n> #ifndef __ANDROID_CAMERA_MANAGER_H__\n> #define __ANDROID_CAMERA_MANAGER_H__\n> \n> +#include \n> +#include \n> #include \n> #include \n> \n> @@ -18,6 +20,9 @@\n> \n> class CameraDevice;\n> \n> +using Mutex = std::mutex;\n> +using MutexLocker = std::unique_lock;\n> +\n> class CameraHalManager\n> {\n> public:\n> @@ -33,10 +38,18 @@ public:\n> \tvoid setCallbacks(const camera_module_callbacks_t *callbacks);\n> \n> private:\n> +\tvoid cameraAdded(std::shared_ptr cam);\n> +\tvoid cameraRemoved(std::shared_ptr cam);\n> +\n> \tlibcamera::CameraManager *cameraManager_;\n> \n> \tconst camera_module_callbacks_t *callbacks_;\n> -\tstd::vector> cameras_;\n> +\tstd::vector> cameras_;\n> +\tstd::map camerasMap_;\n\nIf each hot-plugged camera where treated as a new camera cameras_ and \ncamerasMap_ could be merged to a\n\n std::map> cameras_;\n\nWhich would eliminate the possibility of them going out-of-sync.\n\n> +\tMutex mutex_;\n> +\n> +\tunsigned int externalCameraCounter_;\n> +\tunsigned int cameraCounter_;\n> };\n> \n> #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n> -- \n> 2.26.2\n> \n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 70202BD87D\n\tfor ;\n\tThu, 13 Aug 2020 11:44:22 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EC2CA6055A;\n\tThu, 13 Aug 2020 13:44:21 +0200 (CEST)","from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com\n\t[IPv6:2a00:1450:4864:20::12b])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 28BCF6038C\n\tfor ;\n\tThu, 13 Aug 2020 13:44:20 +0200 (CEST)","by mail-lf1-x12b.google.com with SMTP id x24so2842882lfe.11\n\tfor ;\n\tThu, 13 Aug 2020 04:44:20 -0700 (PDT)","from localhost (h-209-203.A463.priv.bahnhof.se. [155.4.209.203])\n\tby smtp.gmail.com with ESMTPSA id\n\tl26sm1039213ljc.133.2020.08.13.04.44.18\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 13 Aug 2020 04:44:18 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ragnatech-se.20150623.gappssmtp.com\n\theader.i=@ragnatech-se.20150623.gappssmtp.com\n\theader.b=\"mKv+bkqu\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ragnatech-se.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to;\n\tbh=Nk/HcmeT0ToC2tsfaAtRlpFua61lbkfYJYdMPIzDH+g=;\n\tb=mKv+bkqu8QYeYstHMMW7pXtPlhd2/4aU+StV5m227lm1HO4jBp07g2Zl9rwB13RSGG\n\tWpGY17kQKu/HQh6rFcvwfEUVTXcSURyZSEo7G4ma96LsduD6XkfdowfFt6xlJXMydiNQ\n\tEMFCQaSrKtqM7kmOO8gexApbw8N/+7YABSUPLkOxp+b+oUyT9JRQhP5Sr4iJEpiAEtBa\n\t2gSzCWnQ89EzPmt2f0iP7a8ievGTTy1KEUhbYT6ViELme9UXtibSKUvKJNOHbOghozQQ\n\trsTuZirKBXAtGwtQwfDxbv55ENvBj8yK2RrQkH61hmqF8NOxRknSHYiqPM64wspsAH/B\n\tlcYw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to;\n\tbh=Nk/HcmeT0ToC2tsfaAtRlpFua61lbkfYJYdMPIzDH+g=;\n\tb=n5YxM0ck/zF0lEQ3k6QJE1wdqLtbs/oHxV1uwtJPVuqxmtooStAwMy+VegCSLgL/zw\n\tIwRXQZA05M17xY7+dguzCxHE++SBooB+Gsf3yRNN8l+C7nsqZtFBkArHnpc6jLWD5ZTD\n\thzvh62XQH8NtIgzB3j1v4SfMx5bHN8G6axs/mBtiOYKrSrufg0UDw38gdylZAcyQ3EsN\n\txZ+BTSyDq2BwS70/CisfrIbmGd+R3a87Jy+1MAql0PN9JBRnQ5Pw2GRerBM/vjKVYOva\n\tMwN97MrlYe1gOAMpw+baMoyPwvDTUu3fLeDmQt+tsue3lsbguv/vKnjZAy+rKmfELPJM\n\tAb2A==","X-Gm-Message-State":"AOAM532B0pNobTkXAoEA0JlxR8e7XuGqIGzkQnNmAvEmh8VkfhGVURrP\n\tiRwIB/WrxmE6z15Eiacc8ZH790uLJmU=","X-Google-Smtp-Source":"ABdhPJx1v+TKq4fN3yTkpDIDQiOmLcDxtavrppXq7OrLkoQgMFdo8hyj4I7IcQI5zowKsfevdZF1mg==","X-Received":"by 2002:ac2:43ca:: with SMTP id\n\tu10mr1979376lfl.147.1597319059188; \n\tThu, 13 Aug 2020 04:44:19 -0700 (PDT)","Date":"Thu, 13 Aug 2020 13:44:18 +0200","From":"Niklas =?iso-8859-1?q?S=F6derlund?= ","To":"Umang Jain ","Message-ID":"<20200813114418.GE1061689@oden.dyn.berto.se>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200810120406.52654-4-email@uajain.com>","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"iso-8859-1\"","Content-Transfer-Encoding":"quoted-printable","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11983,"web_url":"https://patchwork.libcamera.org/comment/11983/","msgid":"<20200813120529.GE6057@pendragon.ideasonboard.com>","date":"2020-08-13T12:05:29","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Niklas,\n\n(CC'ing Shik)\n\nOn Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > Extend the support for camera hotplug from libcamera's CameraManager\n> > to CameraHalManager. Use camera module callbacks to let the framework\n> > know about the hotplug events and change the status of cameras being\n> > being hotplugged or unplugged via camera_device_status_change().\n> > \n> > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > past by the CameraHalManager. If the camera is seen for the first time,\n> > a new id is assigned to it. If the camera has been seen before by the\n> > manager, it's old id is reused. IDs for internal cameras start with\n> > '0' and for external cameras, they start with '1000'. Note, for the\n> > current implementation, we assume all UVC cameras are external cameras.\n> \n> I wonder if keeping the cache of previously seen cameras or if we should \n> not just treat any hot-plugged camera as a new one? I can't think of any \n> benefit of preserving the same numerical ID between two plug events, \n> while I can think of quiet a few cons.\n> \n> - It's confusing when debugging as un-plugging and then replugging the \n> same camera will result in logs where the numerical ID is the same for \n> both. This may even result in things working by \"chance\" is it reuses \n> an already known numerical ID.\n> \n> - Looking at the code plugging a UVC camera in a different USB port will \n> generate a different Camera::id() result and then be treated as a new \n> camera vs plugging it in the same port and it then being treatad as a \n> new camera.\n> \n> - The logic in this patch is more complex due to it both having to deal \n> with known and new cameras.\n> \n> What is the benefit of this cache that I'm missing?\n\nIt's not clear whether Android requires it. It requires different\ncameras to have different IDs, but I couldn't find a mention of how\nidentical cameras should be treated. The Chrome OS UVC HAL caches the\nID, I don't know if it's mandatory though.\n\nShik, as you've worked on the UVC HAL, would you happen to know if there\nare requirements to keep the same ID when a camera is unplugged and\nreplugged ?\n\n> > CameraDevice is now a shared object and cameras_ vector stores shared\n> > pointers to CameraDevice. This is done in order to introduce reference\n> > counting for CameraDevice objects - especially to handle hot-unplug\n> > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > \n> > Signed-off-by: Umang Jain \n> > ---\n> > src/android/camera_device.cpp | 15 +++\n> > src/android/camera_device.h | 8 +-\n> > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > src/android/camera_hal_manager.h | 15 ++-\n> > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > \n> > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > index d918350..a79bb69 100644\n> > --- a/src/android/camera_device.cpp\n> > +++ b/src/android/camera_device.cpp\n> > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > \t\tdelete it.second;\n> > }\n> > \n> > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > +\t\t\t\t\t\t const std::shared_ptr &cam)\n> > +{\n> > +\tstruct Deleter : std::default_delete {\n> > +\t\tvoid operator()(CameraDevice *camera)\n> > +\t\t{\n> > +\t\t\tdelete camera;\n> > +\t\t}\n> > +\t};\n> > +\n> > +\tCameraDevice *camera = new CameraDevice(id, cam);\n> > +\n> > +\treturn std::shared_ptr(camera, Deleter());\n> > +}\n> \n> As Kieran points out I think this should be added in a separate as this \n> one is quiet large and therefore hard to review.\n> \n> > +\n> > /*\n> > * Initialize the camera static information.\n> > * This method is called before the camera device is opened.\n> > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > index 7be9e11..7f9e010 100644\n> > --- a/src/android/camera_device.h\n> > +++ b/src/android/camera_device.h\n> > @@ -47,8 +47,8 @@ struct CameraStream {\n> > class CameraDevice : protected libcamera::Loggable\n> > {\n> > public:\n> > -\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > -\t~CameraDevice();\n> > +\tstatic std::shared_ptr create(unsigned int id,\n> > +\t\t\t\t\t\t const std::shared_ptr &cam);\n> > \n> > \tint initialize();\n> > \n> > @@ -57,6 +57,7 @@ public:\n> > \n> > \tunsigned int id() const { return id_; }\n> > \tcamera3_device_t *camera3Device() { return &camera3Device_; }\n> > +\tconst libcamera::Camera *getCamera() { return camera_.get(); };\n> \n> Same here this can be done in a separate commit. Also I think this could \n> be named camera() instead of getCamera()\n> \n> > \n> > \tint facing() const { return facing_; }\n> > \tint orientation() const { return orientation_; }\n> > @@ -72,6 +73,9 @@ protected:\n> > \tstd::string logPrefix() const override;\n> > \n> > private:\n> > +\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > +\t~CameraDevice();\n> > +\n> > \tstruct Camera3RequestDescriptor {\n> > \t\tCamera3RequestDescriptor(unsigned int frameNumber,\n> > \t\t\t\t\t unsigned int numBuffers);\n> > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > index 3d6d2b4..fdde2c0 100644\n> > --- a/src/android/camera_hal_manager.cpp\n> > +++ b/src/android/camera_hal_manager.cpp\n> > @@ -8,6 +8,7 @@\n> > #include \"camera_hal_manager.h\"\n> > \n> > #include \n> > +#include \n> > \n> > #include \"libcamera/internal/log.h\"\n> > \n> > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > CameraHalManager::~CameraHalManager()\n> > {\n> > \tcameras_.clear();\n> > +\tcamerasMap_.clear();\n> > \n> > \tif (cameraManager_) {\n> > \t\tcameraManager_->stop();\n> > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > {\n> > \tcameraManager_ = new CameraManager();\n> > \n> > +\t/* Support camera hotplug. */\n> > +\tcameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > +\tcameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > +\n> > +\tcameraCounter_ = 0;\n> > +\texternalCameraCounter_ = 1000;\n> > +\n> > \tint ret = cameraManager_->start();\n> > \tif (ret) {\n> > \t\tLOG(HAL, Error) << \"Failed to start camera manager: \"\n> > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > \t\treturn ret;\n> > \t}\n> > \n> > -\t/*\n> > -\t * For each Camera registered in the system, a CameraDevice\n> > -\t * gets created here to wraps a libcamera Camera instance.\n> > -\t *\n> > -\t * \\todo Support camera hotplug.\n> > -\t */\n> > -\tunsigned int index = 0;\n> > -\tfor (auto &cam : cameraManager_->cameras()) {\n> > -\t\tCameraDevice *camera = new CameraDevice(index, cam);\n> > -\t\tret = camera->initialize();\n> > -\t\tif (ret)\n> > -\t\t\tcontinue;\n> > -\n> > -\t\tcameras_.emplace_back(camera);\n> > -\t\t++index;\n> > -\t}\n> > -\n> > \treturn 0;\n> > }\n> > \n> > CameraDevice *CameraHalManager::open(unsigned int id,\n> > \t\t\t\t const hw_module_t *hardwareModule)\n> > {\n> > -\tif (id >= numCameras()) {\n> > +\tMutexLocker locker(mutex_);\n> > +\n> > +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > +\t\t\t\t [id](std::shared_ptr &cam) {\n> > +\t\t\t\t\treturn cam->id() == id;\n> > +\t\t\t\t});\n> > +\tif (iter == cameras_.end()) {\n> \n> I think you should break this (and the similar ones below) into private \n> helpers instead if implementing the logic in-place\n> \n> CameraHalManager::cameraFromAndroidId(..);\n> CameraHalManager::androidIdFromCamera(..);\n> \n> > \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > \t\treturn nullptr;\n> > \t}\n> > \n> > -\tCameraDevice *camera = cameras_[id].get();\n> > +\tCameraDevice *camera = iter->get();\n> > +\n> > \tif (camera->open(hardwareModule))\n> > \t\treturn nullptr;\n> > \n> > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > \treturn camera;\n> > }\n> > \n> > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > +{\n> > +\tunsigned int id;\n> > +\tbool isCameraExternal = false;\n> > +\tbool isCameraNew = false;\n> > +\n> > +\tMutexLocker locker(mutex_);\n> > +\n> > +\t/* Each camera is assigned a unique integer id when it is seen for the\n> > +\t * first time. If the camera has been seen before, the id is reused and\n> > +\t * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > +\t *\n> > +\t * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > +\t */\n> > +\tauto iter = camerasMap_.find(cam->id());\n> > +\tif (iter != camerasMap_.end()) {\n> > +\t\tid = iter->second;\n> > +\t} else {\n> > +\t\tisCameraNew = true;\n> > +\n> > +\t\t/*\n> > +\t\t * Now check if this is an external camera and assign\n> > +\t\t * its id accordingly.\n> > +\t\t */\n> > +\t\tconst ControlList &properties = cam->properties();\n> > +\t\tif (properties.contains(properties::Location) &&\n> > +\t\t properties.get(properties::Location) &\n> > +\t\t properties::CameraLocationExternal) {\n> > +\t\t\tisCameraExternal = true;\n> > +\t\t\tid = externalCameraCounter_;\n> > +\t\t} else {\n> > +\t\t\tid = cameraCounter_;\n> > +\t\t}\n> \n> As I understand it the information the HAL wants is whether or not the \n> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> \n> If so I wonder if using the camera location as a judgment of the camera \n> is hot-plugged or not is the way to go here? Imagine a device where the \n> camera is permanently attached (not hot-unpluggable) but not fixated in \n> a location. I'm thinking cameras mounted at the end of instruments \n> (medical instruments, hand held tools) or robotics (mounted at the arm \n> of a welding robot). I would imagine those cameras would be marked as \n> located externally but they would not really be hot-pluggable.\n> \n> I understand we have no other way to report or detect this at the moment \n> and I'm not pushing hard for this to be solved as part of this series if \n> it's not easy. But I think a bigger comment here is needed explaining \n> that the HAL wants to know if a camera is hot-pluggable or not and does \n> not really care if it's located internally or externally. I also think a \n> \\todo should be added so it's not forgotten.\n> \n> > +\t}\n> > +\n> > +\t/*\n> > +\t * For each Camera registered in the system, a CameraDevice\n> > +\t * gets created here to wraps a libcamera Camera instance.\n> > +\t */\n> > +\tstd::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > +\tint ret = camera->initialize();\n> > +\tif (ret) {\n> > +\t\tLOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > +\t\treturn;\n> > +\t}\n> > +\n> > +\tif (isCameraNew) {\n> > +\t\tcamerasMap_.emplace(cam->id(), id);\n> > +\n> > +\t\tif (isCameraExternal)\n> > +\t\t\texternalCameraCounter_++;\n> > +\t\telse\n> > +\t\t\tcameraCounter_++;\n> > +\t}\n> > +\n> > +\tcameras_.emplace_back(std::move(camera));\n> > +\n> > +\tif (callbacks_)\n> > +\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> > +\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> > +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > +}\n> > +\n> > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > +{\n> > +\tMutexLocker locker(mutex_);\n> > +\n> > +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > +\t\t\t\t [cam](std::shared_ptr &camera) {\n> > +\t\t\t\t\treturn cam.get() == camera->getCamera();\n> > +\t\t\t\t});\n> > +\tif (iter == cameras_.end())\n> > +\t\treturn;\n> > +\n> > +\tunsigned int id = (*iter)->id();\n> > +\tcallbacks_->camera_device_status_change(callbacks_, id,\n> > +\t\t\t\t\t\tCAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > +\tcameras_.erase(iter);\n> > +\n> > +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > +}\n> > +\n> > unsigned int CameraHalManager::numCameras() const\n> > {\n> > -\treturn cameraManager_->cameras().size();\n> > +\treturn cameraCounter_;\n> > }\n> > \n> > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > \tif (!info)\n> > \t\treturn -EINVAL;\n> > \n> > -\tif (id >= numCameras()) {\n> > +\tMutexLocker locker(mutex_);\n> > +\n> > +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > +\t\t\t\t [id](std::shared_ptr &cam) {\n> > +\t\t\t\t\treturn cam->id() == id;\n> > +\t\t\t\t});\n> > +\tif (iter == cameras_.end()) {\n> > \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > \t\treturn -EINVAL;\n> > \t}\n> > \n> > -\tCameraDevice *camera = cameras_[id].get();\n> > +\tCameraDevice *camera = iter->get();\n> > \n> > \tinfo->facing = camera->facing();\n> > \tinfo->orientation = camera->orientation();\n> > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > {\n> > \tcallbacks_ = callbacks;\n> > +\n> > +\tMutexLocker locker(mutex_);\n> > +\t/*\n> > +\t * Few cameras might have been hotplugged before setting callbacks_ here.\n> > +\t * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > +\t * This hold only for external cameras, as internal cameras are assumed to\n> > +\t * be present at module load time, by the framework.\n> > +\t */\n> > +\tfor (auto &cam : cameraManager_->cameras()) {\n> > +\t\tauto iter = camerasMap_.find(cam->id());\n> > +\t\tif (iter == camerasMap_.end())\n> > +\t\t\tcontinue;\n> > +\n> > +\t\tunsigned int id = iter->second;\n> > +\t\tconst ControlList &properties = cam->properties();\n> > +\t\tif (properties.contains(properties::Location) &&\n> > +\t\t properties.get(properties::Location) &\n> > +\t\t properties::CameraLocationExternal) {\n> > +\t\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> > +\t\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> > +\t\t}\n> > +\t}\n> > }\n> > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > index a582f04..7c481d4 100644\n> > --- a/src/android/camera_hal_manager.h\n> > +++ b/src/android/camera_hal_manager.h\n> > @@ -7,6 +7,8 @@\n> > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > #define __ANDROID_CAMERA_MANAGER_H__\n> > \n> > +#include \n> > +#include \n> > #include \n> > #include \n> > \n> > @@ -18,6 +20,9 @@\n> > \n> > class CameraDevice;\n> > \n> > +using Mutex = std::mutex;\n> > +using MutexLocker = std::unique_lock;\n> > +\n> > class CameraHalManager\n> > {\n> > public:\n> > @@ -33,10 +38,18 @@ public:\n> > \tvoid setCallbacks(const camera_module_callbacks_t *callbacks);\n> > \n> > private:\n> > +\tvoid cameraAdded(std::shared_ptr cam);\n> > +\tvoid cameraRemoved(std::shared_ptr cam);\n> > +\n> > \tlibcamera::CameraManager *cameraManager_;\n> > \n> > \tconst camera_module_callbacks_t *callbacks_;\n> > -\tstd::vector> cameras_;\n> > +\tstd::vector> cameras_;\n> > +\tstd::map camerasMap_;\n> \n> If each hot-plugged camera where treated as a new camera cameras_ and \n> camerasMap_ could be merged to a\n> \n> std::map> cameras_;\n> \n> Which would eliminate the possibility of them going out-of-sync.\n> \n> > +\tMutex mutex_;\n> > +\n> > +\tunsigned int externalCameraCounter_;\n> > +\tunsigned int cameraCounter_;\n> > };\n> > \n> > #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8AC4ABD87C\n\tfor ;\n\tThu, 13 Aug 2020 12:05:45 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DE95C613C0;\n\tThu, 13 Aug 2020 14:05:44 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 24D816038C\n\tfor ;\n\tThu, 13 Aug 2020 14:05:44 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 415A82B7;\n\tThu, 13 Aug 2020 14:05:43 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"lFywM6I0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597320343;\n\tbh=fUbgcsTR1QLK60YNhz/xAM2eqzfKFxP9D1q0gmLTXfY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=lFywM6I0SqlB3BhECCntRxrjRzmxy3mxMkL4WHVDv423L+1ck2BL2uSWlLQA4gHE0\n\t5u9sySkLqccAhb5j8JtlFNOgn+/JSob8Z3bNcDZ3UFhK24oP7gFZQSs7C4hNuDMFCZ\n\t8+0lop0aqK74NaX0/QSmvwql+KprTxxX0kuCqAwc=","Date":"Thu, 13 Aug 2020 15:05:29 +0300","From":"Laurent Pinchart ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Message-ID":"<20200813120529.GE6057@pendragon.ideasonboard.com>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200813114418.GE1061689@oden.dyn.berto.se>","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":11984,"web_url":"https://patchwork.libcamera.org/comment/11984/","msgid":"<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>","date":"2020-08-13T12:22:22","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":1,"url":"https://patchwork.libcamera.org/api/people/1/","name":"Umang Jain","email":"email@uajain.com"},"content":"Hi Niklas\n\nOn 8/13/20 5:14 PM, Niklas Söderlund wrote:\n> Hi Umang,\n>\n> Thanks for your work.\n>\n> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n>> Extend the support for camera hotplug from libcamera's CameraManager\n>> to CameraHalManager. Use camera module callbacks to let the framework\n>> know about the hotplug events and change the status of cameras being\n>> being hotplugged or unplugged via camera_device_status_change().\n>>\n>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n>> past by the CameraHalManager. If the camera is seen for the first time,\n>> a new id is assigned to it. If the camera has been seen before by the\n>> manager, it's old id is reused. IDs for internal cameras start with\n>> '0' and for external cameras, they start with '1000'. Note, for the\n>> current implementation, we assume all UVC cameras are external cameras.\n> I wonder if keeping the cache of previously seen cameras or if we should\n> not just treat any hot-plugged camera as a new one? I can't think of any\n> benefit of preserving the same numerical ID between two plug events,\n> while I can think of quiet a few cons.\n>\n> - It's confusing when debugging as un-plugging and then replugging the\n> same camera will result in logs where the numerical ID is the same for\n> both. This may even result in things working by \"chance\" is it reuses\n> an already known numerical ID.\n>\n> - Looking at the code plugging a UVC camera in a different USB port will\n> generate a different Camera::id() result and then be treated as a new\n> camera vs plugging it in the same port and it then being treatad as a\n> new camera.\nAh, I just noticed this point. I assumed each camera will generate it's\nown unique ID, plugged in for any of the ports. I am not sure, what actions/\nmeasure we can take here to address this.\n>\n> - The logic in this patch is more complex due to it both having to deal\n> with known and new cameras.\n>\n> What is the benefit of this cache that I'm missing?\nLaurent is following up too here, so I'll wait.\nI basically took his suggestion from the v1 review [1] and steered the \npatch into that\ndirection.\n\n>\n>> CameraDevice is now a shared object and cameras_ vector stores shared\n>> pointers to CameraDevice. This is done in order to introduce reference\n>> counting for CameraDevice objects - especially to handle hot-unplug\n>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n>>\n>> Signed-off-by: Umang Jain \n>> ---\n>> src/android/camera_device.cpp | 15 +++\n>> src/android/camera_device.h | 8 +-\n>> src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n>> src/android/camera_hal_manager.h | 15 ++-\n>> 4 files changed, 166 insertions(+), 25 deletions(-)\n>>\n>> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n>> index d918350..a79bb69 100644\n>> --- a/src/android/camera_device.cpp\n>> +++ b/src/android/camera_device.cpp\n>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n>> \t\tdelete it.second;\n>> }\n>> \n>> +std::shared_ptr CameraDevice::create(unsigned int id,\n>> +\t\t\t\t\t\t const std::shared_ptr &cam)\n>> +{\n>> +\tstruct Deleter : std::default_delete {\n>> +\t\tvoid operator()(CameraDevice *camera)\n>> +\t\t{\n>> +\t\t\tdelete camera;\n>> +\t\t}\n>> +\t};\n>> +\n>> +\tCameraDevice *camera = new CameraDevice(id, cam);\n>> +\n>> +\treturn std::shared_ptr(camera, Deleter());\n>> +}\n> As Kieran points out I think this should be added in a separate as this\n> one is quiet large and therefore hard to review.\n>\n>> +\n>> /*\n>> * Initialize the camera static information.\n>> * This method is called before the camera device is opened.\n>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n>> index 7be9e11..7f9e010 100644\n>> --- a/src/android/camera_device.h\n>> +++ b/src/android/camera_device.h\n>> @@ -47,8 +47,8 @@ struct CameraStream {\n>> class CameraDevice : protected libcamera::Loggable\n>> {\n>> public:\n>> -\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n>> -\t~CameraDevice();\n>> +\tstatic std::shared_ptr create(unsigned int id,\n>> +\t\t\t\t\t\t const std::shared_ptr &cam);\n>> \n>> \tint initialize();\n>> \n>> @@ -57,6 +57,7 @@ public:\n>> \n>> \tunsigned int id() const { return id_; }\n>> \tcamera3_device_t *camera3Device() { return &camera3Device_; }\n>> +\tconst libcamera::Camera *getCamera() { return camera_.get(); };\n> Same here this can be done in a separate commit. Also I think this could\n> be named camera() instead of getCamera()\nNoted. Well, this would then be a single-line patch and we won't be able \nto see where\nit's used. I am under the impression that API introduction and it's \nrelevant usage should\nbe done in a single commit(this patch uses the ->getCamera()) so .... \none can the entire\npicture of the diff change.\n>\n>> \n>> \tint facing() const { return facing_; }\n>> \tint orientation() const { return orientation_; }\n>> @@ -72,6 +73,9 @@ protected:\n>> \tstd::string logPrefix() const override;\n>> \n>> private:\n>> +\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n>> +\t~CameraDevice();\n>> +\n>> \tstruct Camera3RequestDescriptor {\n>> \t\tCamera3RequestDescriptor(unsigned int frameNumber,\n>> \t\t\t\t\t unsigned int numBuffers);\n>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n>> index 3d6d2b4..fdde2c0 100644\n>> --- a/src/android/camera_hal_manager.cpp\n>> +++ b/src/android/camera_hal_manager.cpp\n>> @@ -8,6 +8,7 @@\n>> #include \"camera_hal_manager.h\"\n>> \n>> #include \n>> +#include \n>> \n>> #include \"libcamera/internal/log.h\"\n>> \n>> @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n>> CameraHalManager::~CameraHalManager()\n>> {\n>> \tcameras_.clear();\n>> +\tcamerasMap_.clear();\n>> \n>> \tif (cameraManager_) {\n>> \t\tcameraManager_->stop();\n>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n>> {\n>> \tcameraManager_ = new CameraManager();\n>> \n>> +\t/* Support camera hotplug. */\n>> +\tcameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n>> +\tcameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n>> +\n>> +\tcameraCounter_ = 0;\n>> +\texternalCameraCounter_ = 1000;\n>> +\n>> \tint ret = cameraManager_->start();\n>> \tif (ret) {\n>> \t\tLOG(HAL, Error) << \"Failed to start camera manager: \"\n>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n>> \t\treturn ret;\n>> \t}\n>> \n>> -\t/*\n>> -\t * For each Camera registered in the system, a CameraDevice\n>> -\t * gets created here to wraps a libcamera Camera instance.\n>> -\t *\n>> -\t * \\todo Support camera hotplug.\n>> -\t */\n>> -\tunsigned int index = 0;\n>> -\tfor (auto &cam : cameraManager_->cameras()) {\n>> -\t\tCameraDevice *camera = new CameraDevice(index, cam);\n>> -\t\tret = camera->initialize();\n>> -\t\tif (ret)\n>> -\t\t\tcontinue;\n>> -\n>> -\t\tcameras_.emplace_back(camera);\n>> -\t\t++index;\n>> -\t}\n>> -\n>> \treturn 0;\n>> }\n>> \n>> CameraDevice *CameraHalManager::open(unsigned int id,\n>> \t\t\t\t const hw_module_t *hardwareModule)\n>> {\n>> -\tif (id >= numCameras()) {\n>> +\tMutexLocker locker(mutex_);\n>> +\n>> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>> +\t\t\t\t [id](std::shared_ptr &cam) {\n>> +\t\t\t\t\treturn cam->id() == id;\n>> +\t\t\t\t});\n>> +\tif (iter == cameras_.end()) {\n> I think you should break this (and the similar ones below) into private\n> helpers instead if implementing the logic in-place\n>\n> CameraHalManager::cameraFromAndroidId(..);\n> CameraHalManager::androidIdFromCamera(..);\n>> \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>> \t\treturn nullptr;\n>> \t}\n>> \n>> -\tCameraDevice *camera = cameras_[id].get();\n>> +\tCameraDevice *camera = iter->get();\n>> +\n>> \tif (camera->open(hardwareModule))\n>> \t\treturn nullptr;\n>> \n>> @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n>> \treturn camera;\n>> }\n>> \n>> +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n>> +{\n>> +\tunsigned int id;\n>> +\tbool isCameraExternal = false;\n>> +\tbool isCameraNew = false;\n>> +\n>> +\tMutexLocker locker(mutex_);\n>> +\n>> +\t/* Each camera is assigned a unique integer id when it is seen for the\n>> +\t * first time. If the camera has been seen before, the id is reused and\n>> +\t * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n>> +\t *\n>> +\t * ID starts from '0' for internal cameras and '1000' for external cameras.\n>> +\t */\n>> +\tauto iter = camerasMap_.find(cam->id());\n>> +\tif (iter != camerasMap_.end()) {\n>> +\t\tid = iter->second;\n>> +\t} else {\n>> +\t\tisCameraNew = true;\n>> +\n>> +\t\t/*\n>> +\t\t * Now check if this is an external camera and assign\n>> +\t\t * its id accordingly.\n>> +\t\t */\n>> +\t\tconst ControlList &properties = cam->properties();\n>> +\t\tif (properties.contains(properties::Location) &&\n>> +\t\t properties.get(properties::Location) &\n>> +\t\t properties::CameraLocationExternal) {\n>> +\t\t\tisCameraExternal = true;\n>> +\t\t\tid = externalCameraCounter_;\n>> +\t\t} else {\n>> +\t\t\tid = cameraCounter_;\n>> +\t\t}\n> As I understand it the information the HAL wants is whether or not the\n> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\nAgain, the spec is not clear on the hot-plugged / removable camera IDs \nfront.\nIt is though clear on the internal cameras IDs - should start from 0, hence,\nwe thought 1000 is a good number for assigning IDs for external cameras.\n>\n> If so I wonder if using the camera location as a judgment of the camera\n> is hot-plugged or not is the way to go here? Imagine a device where the\n> camera is permanently attached (not hot-unpluggable) but not fixated in\n> a location. I'm thinking cameras mounted at the end of instruments\n> (medical instruments, hand held tools) or robotics (mounted at the arm\n> of a welding robot). I would imagine those cameras would be marked as\n> located externally but they would not really be hot-pluggable.\nYes, I get your point. Their location is external, but from the \npoint-of-view\nof HAL, they are still needed to be treated as \"internal\" cameras, no?\nI guess, in HAL, internal cameras are the ones, which cannot be detached\nfrom the device, whereas external cameras are the one, which can,\n(Just a guess, I might be entirely wrong)\n>\n> I understand we have no other way to report or detect this at the moment\n> and I'm not pushing hard for this to be solved as part of this series if\n> it's not easy. But I think a bigger comment here is needed explaining\n> that the HAL wants to know if a camera is hot-pluggable or not and does\n> not really care if it's located internally or externally. I also think a\n> \\todo should be added so it's not forgotten.\n>\n>> +\t}\n>> +\n>> +\t/*\n>> +\t * For each Camera registered in the system, a CameraDevice\n>> +\t * gets created here to wraps a libcamera Camera instance.\n>> +\t */\n>> +\tstd::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n>> +\tint ret = camera->initialize();\n>> +\tif (ret) {\n>> +\t\tLOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n>> +\t\treturn;\n>> +\t}\n>> +\n>> +\tif (isCameraNew) {\n>> +\t\tcamerasMap_.emplace(cam->id(), id);\n>> +\n>> +\t\tif (isCameraExternal)\n>> +\t\t\texternalCameraCounter_++;\n>> +\t\telse\n>> +\t\t\tcameraCounter_++;\n>> +\t}\n>> +\n>> +\tcameras_.emplace_back(std::move(camera));\n>> +\n>> +\tif (callbacks_)\n>> +\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n>> +\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n>> +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n>> +}\n>> +\n>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n>> +{\n>> +\tMutexLocker locker(mutex_);\n>> +\n>> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>> +\t\t\t\t [cam](std::shared_ptr &camera) {\n>> +\t\t\t\t\treturn cam.get() == camera->getCamera();\n>> +\t\t\t\t});\n>> +\tif (iter == cameras_.end())\n>> +\t\treturn;\n>> +\n>> +\tunsigned int id = (*iter)->id();\n>> +\tcallbacks_->camera_device_status_change(callbacks_, id,\n>> +\t\t\t\t\t\tCAMERA_DEVICE_STATUS_NOT_PRESENT);\n>> +\tcameras_.erase(iter);\n>> +\n>> +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n>> +}\n>> +\n>> unsigned int CameraHalManager::numCameras() const\n>> {\n>> -\treturn cameraManager_->cameras().size();\n>> +\treturn cameraCounter_;\n>> }\n>> \n>> int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n>> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n>> \tif (!info)\n>> \t\treturn -EINVAL;\n>> \n>> -\tif (id >= numCameras()) {\n>> +\tMutexLocker locker(mutex_);\n>> +\n>> +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>> +\t\t\t\t [id](std::shared_ptr &cam) {\n>> +\t\t\t\t\treturn cam->id() == id;\n>> +\t\t\t\t});\n>> +\tif (iter == cameras_.end()) {\n>> \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>> \t\treturn -EINVAL;\n>> \t}\n>> \n>> -\tCameraDevice *camera = cameras_[id].get();\n>> +\tCameraDevice *camera = iter->get();\n>> \n>> \tinfo->facing = camera->facing();\n>> \tinfo->orientation = camera->orientation();\n>> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n>> void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n>> {\n>> \tcallbacks_ = callbacks;\n>> +\n>> +\tMutexLocker locker(mutex_);\n>> +\t/*\n>> +\t * Few cameras might have been hotplugged before setting callbacks_ here.\n>> +\t * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n>> +\t * This hold only for external cameras, as internal cameras are assumed to\n>> +\t * be present at module load time, by the framework.\n>> +\t */\n>> +\tfor (auto &cam : cameraManager_->cameras()) {\n>> +\t\tauto iter = camerasMap_.find(cam->id());\n>> +\t\tif (iter == camerasMap_.end())\n>> +\t\t\tcontinue;\n>> +\n>> +\t\tunsigned int id = iter->second;\n>> +\t\tconst ControlList &properties = cam->properties();\n>> +\t\tif (properties.contains(properties::Location) &&\n>> +\t\t properties.get(properties::Location) &\n>> +\t\t properties::CameraLocationExternal) {\n>> +\t\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n>> +\t\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n>> +\t\t}\n>> +\t}\n>> }\n>> diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n>> index a582f04..7c481d4 100644\n>> --- a/src/android/camera_hal_manager.h\n>> +++ b/src/android/camera_hal_manager.h\n>> @@ -7,6 +7,8 @@\n>> #ifndef __ANDROID_CAMERA_MANAGER_H__\n>> #define __ANDROID_CAMERA_MANAGER_H__\n>> \n>> +#include \n>> +#include \n>> #include \n>> #include \n>> \n>> @@ -18,6 +20,9 @@\n>> \n>> class CameraDevice;\n>> \n>> +using Mutex = std::mutex;\n>> +using MutexLocker = std::unique_lock;\n>> +\n>> class CameraHalManager\n>> {\n>> public:\n>> @@ -33,10 +38,18 @@ public:\n>> \tvoid setCallbacks(const camera_module_callbacks_t *callbacks);\n>> \n>> private:\n>> +\tvoid cameraAdded(std::shared_ptr cam);\n>> +\tvoid cameraRemoved(std::shared_ptr cam);\n>> +\n>> \tlibcamera::CameraManager *cameraManager_;\n>> \n>> \tconst camera_module_callbacks_t *callbacks_;\n>> -\tstd::vector> cameras_;\n>> +\tstd::vector> cameras_;\n>> +\tstd::map camerasMap_;\n> If each hot-plugged camera where treated as a new camera cameras_ and\n> camerasMap_ could be merged to a\n>\n> std::map> cameras_;\n>\n> Which would eliminate the possibility of them going out-of-sync.\nGreat point. I hope it's not too cubersome to iterate over all the entires,\nto find the exact CameraDevice we want for various plug events handler.\nI will look into it.\n>\n>> +\tMutex mutex_;\n>> +\n>> +\tunsigned int externalCameraCounter_;\n>> +\tunsigned int cameraCounter_;\n>> };\n>> \n>> #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>> -- \n>> 2.26.2\n>>\n>> _______________________________________________\n>> libcamera-devel mailing list\n>> libcamera-devel@lists.libcamera.org\n>> https://lists.libcamera.org/listinfo/libcamera-devel\n[1] : \nhttps://lists.libcamera.org/pipermail/libcamera-devel/2020-August/011857.html","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 84854BD87D\n\tfor ;\n\tThu, 13 Aug 2020 12:22:26 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EA9D4613A9;\n\tThu, 13 Aug 2020 14:22:25 +0200 (CEST)","from o1.f.az.sendgrid.net (o1.f.az.sendgrid.net [208.117.55.132])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 324566038C\n\tfor ;\n\tThu, 13 Aug 2020 14:22:23 +0200 (CEST)","by filterdrecv-p3iad2-6f4d87c464-6bkjq with SMTP id\n\tfilterdrecv-p3iad2-6f4d87c464-6bkjq-20-5F35307E-2D\n\t2020-08-13 12:22:22.582393638 +0000 UTC m=+65162.402227250","from mail.uajain.com (unknown)\n\tby ismtpd0003p1maa1.sendgrid.net (SG) with ESMTP\n\tid 0WOjM3cyT1Ov6ZmW_LgJpA Thu, 13 Aug 2020 12:22:21.865 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=uajain.com header.i=@uajain.com\n\theader.b=\"xxtLjUnc\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=uajain.com;\n\th=subject:references:from:mime-version:in-reply-to:to:cc:content-type:\n\tcontent-transfer-encoding;\n\ts=s1; bh=2mos/0Rvd0hwb9Pb72HzUZf7DKT5SIAQf/ZABt/SguI=;\n\tb=xxtLjUncFKyuyzYhf6Iu8iR4CCPMEWI2ObNwbWnNAcga9Xi7Cho7sXGuouU1JoWMcqqB\n\t/+gS3MJRaF8kPj2dDYCJsu/ucUCEfY3rE7vg1vUAg4CACmLPfkQexeLvemyYFJtVfSCVbt\n\tLHppIKRqiCGmjX6wUSqwQwRvNa2/RmmIA=","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>","From":"Umang Jain ","Message-ID":"<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>","Date":"Thu, 13 Aug 2020 12:22:22 +0000 (UTC)","Mime-Version":"1.0","In-Reply-To":"<20200813114418.GE1061689@oden.dyn.berto.se>","X-SG-EID":"1Q40EQ7YGir8a9gjSIAdTjhngY657NMk9ckeo4dbHZDiOpywc/L3L9rFqlwE4KPcu+0qlnDlrDbBcRldMV+9DsFsD/DGgaSFuaiouMd5D6SxhJc3/Doasw3Wd1FVsjlQw3RLmfJZN+SquSRN9JiWhhFea0801754m6DHKk0dY6bIPZ+dOUU+lApOS6JyHTvyvvDZC6X+MuLobcWv4VyLh0r0o7HOnkNINWb+HBjygR3Z/0AIAYI1eztG3EJs1fCFgzPJ8GB4zBx5xzCK93Rljw==","To":"Niklas =?iso-8859-1?q?S=F6derlund?= ","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Transfer-Encoding":"quoted-printable","Content-Type":"text/plain; charset=\"iso-8859-1\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12007,"web_url":"https://patchwork.libcamera.org/comment/12007/","msgid":"<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>","date":"2020-08-14T08:29:58","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":1,"url":"https://patchwork.libcamera.org/api/people/1/","name":"Umang Jain","email":"email@uajain.com"},"content":"Hi Niklas,\n\nOn 8/13/20 5:52 PM, Umang Jain wrote:\n> Hi Niklas\n>\n> On 8/13/20 5:14 PM, Niklas Söderlund wrote:\n>> Hi Umang,\n>>\n>> Thanks for your work.\n>>\n>> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n>>> Extend the support for camera hotplug from libcamera's CameraManager\n>>> to CameraHalManager. Use camera module callbacks to let the framework\n>>> know about the hotplug events and change the status of cameras being\n>>> being hotplugged or unplugged via camera_device_status_change().\n>>>\n>>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n>>> past by the CameraHalManager. If the camera is seen for the first time,\n>>> a new id is assigned to it. If the camera has been seen before by the\n>>> manager, it's old id is reused. IDs for internal cameras start with\n>>> '0' and for external cameras, they start with '1000'. Note, for the\n>>> current implementation, we assume all UVC cameras are external cameras.\n>> I wonder if keeping the cache of previously seen cameras or if we should\n>> not just treat any hot-plugged camera as a new one? I can't think of any\n>> benefit of preserving the same numerical ID between two plug events,\n>> while I can think of quiet a few cons.\n>>\n>> - It's confusing when debugging as un-plugging and then replugging the\n>>    same camera will result in logs where the numerical ID is the same \n>> for\n>>    both. This may even result in things working by \"chance\" is it reuses\n>>    an already known numerical ID.\n>>\n>> - Looking at the code plugging a UVC camera in a different USB port will\n>>    generate a different Camera::id() result and then be treated as a new\n>>    camera vs plugging it in the same port and it then being treatad as a\n>>    new camera.\n> Ah, I just noticed this point. I assumed each camera will generate it's\n> own unique ID, plugged in for any of the ports. I am not sure, what \n> actions/\n> measure we can take here to address this.\n>>\n>> - The logic in this patch is more complex due to it both having to deal\n>>    with known and new cameras.\n>>\n>> What is the benefit of this cache that I'm missing?\n> Laurent is following up too here, so I'll wait.\n> I basically took his suggestion from the v1 review [1] and steered the \n> patch into that\n> direction.\n>\n>>\n>>> CameraDevice is now a shared object and cameras_ vector stores shared\n>>> pointers to CameraDevice. This is done in order to introduce reference\n>>> counting for CameraDevice objects - especially to handle hot-unplug\n>>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n>>>\n>>> Signed-off-by: Umang Jain \n>>> ---\n>>>   src/android/camera_device.cpp      |  15 +++\n>>>   src/android/camera_device.h        |   8 +-\n>>>   src/android/camera_hal_manager.cpp | 153 \n>>> ++++++++++++++++++++++++-----\n>>>   src/android/camera_hal_manager.h   |  15 ++-\n>>>   4 files changed, 166 insertions(+), 25 deletions(-)\n>>>\n>>> diff --git a/src/android/camera_device.cpp \n>>> b/src/android/camera_device.cpp\n>>> index d918350..a79bb69 100644\n>>> --- a/src/android/camera_device.cpp\n>>> +++ b/src/android/camera_device.cpp\n>>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n>>>           delete it.second;\n>>>   }\n>>>   +std::shared_ptr CameraDevice::create(unsigned int id,\n>>> +                            const std::shared_ptr &cam)\n>>> +{\n>>> +    struct Deleter : std::default_delete {\n>>> +        void operator()(CameraDevice *camera)\n>>> +        {\n>>> +            delete camera;\n>>> +        }\n>>> +    };\n>>> +\n>>> +    CameraDevice *camera = new CameraDevice(id, cam);\n>>> +\n>>> +    return std::shared_ptr(camera, Deleter());\n>>> +}\n>> As Kieran points out I think this should be added in a separate as this\n>> one is quiet large and therefore hard to review.\n>>\n>>> +\n>>>   /*\n>>>    * Initialize the camera static information.\n>>>    * This method is called before the camera device is opened.\n>>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n>>> index 7be9e11..7f9e010 100644\n>>> --- a/src/android/camera_device.h\n>>> +++ b/src/android/camera_device.h\n>>> @@ -47,8 +47,8 @@ struct CameraStream {\n>>>   class CameraDevice : protected libcamera::Loggable\n>>>   {\n>>>   public:\n>>> -    CameraDevice(unsigned int id, const \n>>> std::shared_ptr &camera);\n>>> -    ~CameraDevice();\n>>> +    static std::shared_ptr create(unsigned int id,\n>>> +                            const \n>>> std::shared_ptr &cam);\n>>>         int initialize();\n>>>   @@ -57,6 +57,7 @@ public:\n>>>         unsigned int id() const { return id_; }\n>>>       camera3_device_t *camera3Device() { return &camera3Device_; }\n>>> +    const libcamera::Camera *getCamera() { return camera_.get(); };\n>> Same here this can be done in a separate commit. Also I think this could\n>> be named camera() instead of getCamera()\n> Noted. Well, this would then be a single-line patch and we won't be \n> able to see where\n> it's used. I am under the impression that API introduction and it's \n> relevant usage should\n> be done in a single commit(this patch uses the ->getCamera()) so .... \n> one can the entire\n> picture of the diff change.\n>>\n>>>         int facing() const { return facing_; }\n>>>       int orientation() const { return orientation_; }\n>>> @@ -72,6 +73,9 @@ protected:\n>>>       std::string logPrefix() const override;\n>>>     private:\n>>> +    CameraDevice(unsigned int id, const \n>>> std::shared_ptr &camera);\n>>> +    ~CameraDevice();\n>>> +\n>>>       struct Camera3RequestDescriptor {\n>>>           Camera3RequestDescriptor(unsigned int frameNumber,\n>>>                        unsigned int numBuffers);\n>>> diff --git a/src/android/camera_hal_manager.cpp \n>>> b/src/android/camera_hal_manager.cpp\n>>> index 3d6d2b4..fdde2c0 100644\n>>> --- a/src/android/camera_hal_manager.cpp\n>>> +++ b/src/android/camera_hal_manager.cpp\n>>> @@ -8,6 +8,7 @@\n>>>   #include \"camera_hal_manager.h\"\n>>>     #include \n>>> +#include \n>>>     #include \"libcamera/internal/log.h\"\n>>>   @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n>>>   CameraHalManager::~CameraHalManager()\n>>>   {\n>>>       cameras_.clear();\n>>> +    camerasMap_.clear();\n>>>         if (cameraManager_) {\n>>>           cameraManager_->stop();\n>>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n>>>   {\n>>>       cameraManager_ = new CameraManager();\n>>>   +    /* Support camera hotplug. */\n>>> +    cameraManager_->cameraAdded.connect(this, \n>>> &CameraHalManager::cameraAdded);\n>>> +    cameraManager_->cameraRemoved.connect(this, \n>>> &CameraHalManager::cameraRemoved);\n>>> +\n>>> +    cameraCounter_ = 0;\n>>> +    externalCameraCounter_ = 1000;\n>>> +\n>>>       int ret = cameraManager_->start();\n>>>       if (ret) {\n>>>           LOG(HAL, Error) << \"Failed to start camera manager: \"\n>>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n>>>           return ret;\n>>>       }\n>>>   -    /*\n>>> -     * For each Camera registered in the system, a CameraDevice\n>>> -     * gets created here to wraps a libcamera Camera instance.\n>>> -     *\n>>> -     * \\todo Support camera hotplug.\n>>> -     */\n>>> -    unsigned int index = 0;\n>>> -    for (auto &cam : cameraManager_->cameras()) {\n>>> -        CameraDevice *camera = new CameraDevice(index, cam);\n>>> -        ret = camera->initialize();\n>>> -        if (ret)\n>>> -            continue;\n>>> -\n>>> -        cameras_.emplace_back(camera);\n>>> -        ++index;\n>>> -    }\n>>> -\n>>>       return 0;\n>>>   }\n>>>     CameraDevice *CameraHalManager::open(unsigned int id,\n>>>                        const hw_module_t *hardwareModule)\n>>>   {\n>>> -    if (id >= numCameras()) {\n>>> +    MutexLocker locker(mutex_);\n>>> +\n>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>> +                 [id](std::shared_ptr &cam) {\n>>> +                    return cam->id() == id;\n>>> +                });\n>>> +    if (iter == cameras_.end()) {\n>> I think you should break this (and the similar ones below) into private\n>> helpers instead if implementing the logic in-place\n>>\n>>      CameraHalManager::cameraFromAndroidId(..);\n>>      CameraHalManager::androidIdFromCamera(..);\n>>>           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>>>           return nullptr;\n>>>       }\n>>>   -    CameraDevice *camera = cameras_[id].get();\n>>> +    CameraDevice *camera = iter->get();\n>>> +\n>>>       if (camera->open(hardwareModule))\n>>>           return nullptr;\n>>>   @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned \n>>> int id,\n>>>       return camera;\n>>>   }\n>>>   +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n>>> +{\n>>> +    unsigned int id;\n>>> +    bool isCameraExternal = false;\n>>> +    bool isCameraNew = false;\n>>> +\n>>> +    MutexLocker locker(mutex_);\n>>> +\n>>> +    /* Each camera is assigned a unique integer id when it is seen \n>>> for the\n>>> +     * first time. If the camera has been seen before, the id is \n>>> reused and\n>>> +     * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT \n>>> subsequently.\n>>> +     *\n>>> +     * ID starts from '0' for internal cameras and '1000' for \n>>> external cameras.\n>>> +     */\n>>> +    auto iter = camerasMap_.find(cam->id());\n>>> +    if (iter != camerasMap_.end()) {\n>>> +        id = iter->second;\n>>> +    } else {\n>>> +        isCameraNew = true;\n>>> +\n>>> +        /*\n>>> +         *  Now check if this is an external camera and assign\n>>> +         *  its id accordingly.\n>>> +         */\n>>> +        const ControlList &properties = cam->properties();\n>>> +        if (properties.contains(properties::Location) &&\n>>> +            properties.get(properties::Location) &\n>>> +            properties::CameraLocationExternal) {\n>>> +            isCameraExternal = true;\n>>> +            id = externalCameraCounter_;\n>>> +        } else {\n>>> +            id = cameraCounter_;\n>>> +        }\n>> As I understand it the information the HAL wants is whether or not the\n>> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> Again, the spec is not clear on the hot-plugged / removable camera IDs \n> front.\n> It is though clear on the internal cameras IDs - should start from 0, \n> hence,\n> we thought 1000 is a good number for assigning IDs for external cameras.\n>>\n>> If so I wonder if using the camera location as a judgment of the camera\n>> is hot-plugged or not is the way to go here? Imagine a device where the\n>> camera is permanently attached (not hot-unpluggable) but not fixated in\n>> a location. I'm thinking cameras mounted at the end of instruments\n>> (medical instruments, hand held tools) or robotics (mounted at the arm\n>> of a welding robot). I would imagine those cameras would be marked as\n>> located externally but they would not really be hot-pluggable.\n> Yes, I get your point. Their location is external, but from the \n> point-of-view\n> of HAL, they are still needed to be treated as \"internal\" cameras, no?\n> I guess, in HAL, internal cameras are the ones, which cannot be detached\n> from the device, whereas external cameras are the one, which can,\n> (Just a guess, I might be entirely wrong)\n>>\n>> I understand we have no other way to report or detect this at the moment\n>> and I'm not pushing hard for this to be solved as part of this series if\n>> it's not easy. But I think a bigger comment here is needed explaining\n>> that the HAL wants to know if a camera is hot-pluggable or not and does\n>> not really care if it's located internally or externally. I also think a\n>> \\todo should be added so it's not forgotten.\n>>\n>>> +    }\n>>> +\n>>> +    /*\n>>> +     * For each Camera registered in the system, a CameraDevice\n>>> +     * gets created here to wraps a libcamera Camera instance.\n>>> +     */\n>>> +    std::shared_ptr camera = CameraDevice::create(id, \n>>> std::move(cam));\n>>> +    int ret = camera->initialize();\n>>> +    if (ret) {\n>>> +        LOG(HAL, Error) << \"Failed to initialize camera: \" << \n>>> cam->id();\n>>> +        return;\n>>> +    }\n>>> +\n>>> +    if (isCameraNew) {\n>>> +        camerasMap_.emplace(cam->id(), id);\n>>> +\n>>> +        if (isCameraExternal)\n>>> +            externalCameraCounter_++;\n>>> +        else\n>>> +            cameraCounter_++;\n>>> +    }\n>>> +\n>>> +    cameras_.emplace_back(std::move(camera));\n>>> +\n>>> +    if (callbacks_)\n>>> + callbacks_->camera_device_status_change(callbacks_, id,\n>>> +                            CAMERA_DEVICE_STATUS_PRESENT);\n>>> +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n>>> +}\n>>> +\n>>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n>>> +{\n>>> +    MutexLocker locker(mutex_);\n>>> +\n>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>> +                 [cam](std::shared_ptr &camera) {\n>>> +                    return cam.get() == camera->getCamera();\n>>> +                });\n>>> +    if (iter == cameras_.end())\n>>> +        return;\n>>> +\n>>> +    unsigned int id = (*iter)->id();\n>>> +    callbacks_->camera_device_status_change(callbacks_, id,\n>>> +                        CAMERA_DEVICE_STATUS_NOT_PRESENT);\n>>> +    cameras_.erase(iter);\n>>> +\n>>> +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed \n>>> successfully.\";\n>>> +}\n>>> +\n>>>   unsigned int CameraHalManager::numCameras() const\n>>>   {\n>>> -    return cameraManager_->cameras().size();\n>>> +    return cameraCounter_;\n>>>   }\n>>>     int CameraHalManager::getCameraInfo(unsigned int id, struct \n>>> camera_info *info)\n>>> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned \n>>> int id, struct camera_info *info)\n>>>       if (!info)\n>>>           return -EINVAL;\n>>>   -    if (id >= numCameras()) {\n>>> +    MutexLocker locker(mutex_);\n>>> +\n>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>> +                 [id](std::shared_ptr &cam) {\n>>> +                    return cam->id() == id;\n>>> +                });\n>>> +    if (iter == cameras_.end()) {\n>>>           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>>>           return -EINVAL;\n>>>       }\n>>>   -    CameraDevice *camera = cameras_[id].get();\n>>> +    CameraDevice *camera = iter->get();\n>>>         info->facing = camera->facing();\n>>>       info->orientation = camera->orientation();\n>>> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned \n>>> int id, struct camera_info *info)\n>>>   void CameraHalManager::setCallbacks(const \n>>> camera_module_callbacks_t *callbacks)\n>>>   {\n>>>       callbacks_ = callbacks;\n>>> +\n>>> +    MutexLocker locker(mutex_);\n>>> +    /*\n>>> +     * Few cameras might have been hotplugged before setting \n>>> callbacks_ here.\n>>> +     * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them \n>>> explicitly.\n>>> +     * This hold only for external cameras, as internal cameras are \n>>> assumed to\n>>> +     * be present at module load time, by the framework.\n>>> +     */\n>>> +    for (auto &cam : cameraManager_->cameras()) {\n>>> +        auto iter = camerasMap_.find(cam->id());\n>>> +        if (iter == camerasMap_.end())\n>>> +            continue;\n>>> +\n>>> +        unsigned int id = iter->second;\n>>> +        const ControlList &properties = cam->properties();\n>>> +        if (properties.contains(properties::Location) &&\n>>> +            properties.get(properties::Location) &\n>>> +            properties::CameraLocationExternal) {\n>>> + callbacks_->camera_device_status_change(callbacks_, id,\n>>> + CAMERA_DEVICE_STATUS_PRESENT);\n>>> +        }\n>>> +    }\n>>>   }\n>>> diff --git a/src/android/camera_hal_manager.h \n>>> b/src/android/camera_hal_manager.h\n>>> index a582f04..7c481d4 100644\n>>> --- a/src/android/camera_hal_manager.h\n>>> +++ b/src/android/camera_hal_manager.h\n>>> @@ -7,6 +7,8 @@\n>>>   #ifndef __ANDROID_CAMERA_MANAGER_H__\n>>>   #define __ANDROID_CAMERA_MANAGER_H__\n>>>   +#include \n>>> +#include \n>>>   #include \n>>>   #include \n>>>   @@ -18,6 +20,9 @@\n>>>     class CameraDevice;\n>>>   +using Mutex = std::mutex;\n>>> +using MutexLocker = std::unique_lock;\n>>> +\n>>>   class CameraHalManager\n>>>   {\n>>>   public:\n>>> @@ -33,10 +38,18 @@ public:\n>>>       void setCallbacks(const camera_module_callbacks_t *callbacks);\n>>>     private:\n>>> +    void cameraAdded(std::shared_ptr cam);\n>>> +    void cameraRemoved(std::shared_ptr cam);\n>>> +\n>>>       libcamera::CameraManager *cameraManager_;\n>>>         const camera_module_callbacks_t *callbacks_;\n>>> -    std::vector> cameras_;\n>>> +    std::vector> cameras_;\n>>> +    std::map camerasMap_;\n>> If each hot-plugged camera where treated as a new camera cameras_ and\n>> camerasMap_ could be merged to a\n>>\n>>      std::map> cameras_;\n>>\n>> Which would eliminate the possibility of them going out-of-sync.\n> Great point. I hope it's not too cubersome to iterate over all the \n> entires,\n> to find the exact CameraDevice we want for various plug events handler.\n> I will look into it.\nI found this to be a bit cubersome to implement. The foundation issue \nhere is:\nwe do need to cache the android HAL's ID<->libcamera::Camera::id mapping\nto identify if we are replugging an already-seen camera (I am assuming that\nlibcamera::Camera::id is a unique ID generated by libcamera for each camera\nand is constant whenever the camera is replugged).\n\nIf I just maintain a single map std::map>\nthe CameraDevice will be deleted on hot-unplug. Then I do not have a \ncomparator to\ncompare against, to know if the ID key should be reused or not, when \nhotplug event\nhandlers.\n>>\n>>> +    Mutex mutex_;\n>>> +\n>>> +    unsigned int externalCameraCounter_;\n>>> +    unsigned int cameraCounter_;\n>>>   };\n>>>     #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>>> -- \n>>> 2.26.2\n>>>\n>>> _______________________________________________\n>>> libcamera-devel mailing list\n>>> libcamera-devel@lists.libcamera.org\n>>> https://lists.libcamera.org/listinfo/libcamera-devel\n> [1] : \n> https://lists.libcamera.org/pipermail/libcamera-devel/2020-August/011857.html \n>\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0CAB2BD87C\n\tfor ;\n\tFri, 14 Aug 2020 08:30:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C36DC613D6;\n\tFri, 14 Aug 2020 10:30:01 +0200 (CEST)","from o1.f.az.sendgrid.net (o1.f.az.sendgrid.net [208.117.55.132])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4920160386\n\tfor ;\n\tFri, 14 Aug 2020 10:30:00 +0200 (CEST)","by filterdrecv-p3mdw1-7fdf8bccc5-nzk6l with SMTP id\n\tfilterdrecv-p3mdw1-7fdf8bccc5-nzk6l-20-5F364B86-1D\n\t2020-08-14 08:29:58.680629806 +0000 UTC m=+57625.092742644","from mail.uajain.com (unknown)\n\tby ismtpd0006p1maa1.sendgrid.net (SG) with ESMTP\n\tid uozWea7BQdq4iQBYQyu6fg Fri, 14 Aug 2020 08:29:58.081 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=uajain.com header.i=@uajain.com\n\theader.b=\"OeYOI9Gy\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=uajain.com;\n\th=subject:from:references:mime-version:in-reply-to:to:cc:content-type:\n\tcontent-transfer-encoding;\n\ts=s1; bh=JCbHfDwZGg3bRy4SdDO61aYECjCj2wulgWUZRfRgSjo=;\n\tb=OeYOI9GySsYRIRduL6t8WotbdAd+5j8bg1VxEjW8JEeG/DMxB9lxTHpL2DyBcVfvlkjj\n\tgcdLVvNWm94HkFO0xxfAIn/UglKSjXYoeKN1g07GrM7rQQ8OpXYdhYjiX98VuA32yLHNrQ\n\txmpzhe+BBHc2YOUddeaPBY7plfv0drkWQ=","From":"Umang Jain ","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>","Message-ID":"<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>","Date":"Fri, 14 Aug 2020 08:29:58 +0000 (UTC)","Mime-Version":"1.0","In-Reply-To":"<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>","X-SG-EID":"1Q40EQ7YGir8a9gjSIAdTjhngY657NMk9ckeo4dbHZDiOpywc/L3L9rFqlwE4KPcbjgBrPKwU6zsq5Nc1XDaVnJuL4azuoq40HKR73lcHmRVnuAh9R3dqLMM3wMIz2VuVxMi0VnLf06m/y7fOIxZpK+BG2J3cIxOHpTWa1KIYMoTVuTYZ2I0gL9TevNR90KTuM7DlXojtHr5YcfkiNq2NEiR+h8QBRBtpgzKF96knXGdb3hO7RXSLyujPZuEvILNon7U4vN3Cqm1Q8acrf0WlA==","To":"Niklas =?iso-8859-1?q?S=F6derlund?= ","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Transfer-Encoding":"quoted-printable","Content-Type":"text/plain; charset=\"iso-8859-1\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12008,"web_url":"https://patchwork.libcamera.org/comment/12008/","msgid":"<2e465494-31e1-88d4-9752-6339dee02bea@ideasonboard.com>","date":"2020-08-14T08:45:58","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Umang,\n\nOn 14/08/2020 09:29, Umang Jain wrote:\n> Hi Niklas,\n> \n> On 8/13/20 5:52 PM, Umang Jain wrote:\n>> Hi Niklas\n>>\n>> On 8/13/20 5:14 PM, Niklas Söderlund wrote:\n>>> Hi Umang,\n>>>\n>>> Thanks for your work.\n>>>\n>>> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n>>>> Extend the support for camera hotplug from libcamera's CameraManager\n>>>> to CameraHalManager. Use camera module callbacks to let the framework\n>>>> know about the hotplug events and change the status of cameras being\n>>>> being hotplugged or unplugged via camera_device_status_change().\n>>>>\n>>>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n>>>> past by the CameraHalManager. If the camera is seen for the first time,\n>>>> a new id is assigned to it. If the camera has been seen before by the\n>>>> manager, it's old id is reused. IDs for internal cameras start with\n>>>> '0' and for external cameras, they start with '1000'. Note, for the\n>>>> current implementation, we assume all UVC cameras are external cameras.\n>>> I wonder if keeping the cache of previously seen cameras or if we should\n>>> not just treat any hot-plugged camera as a new one? I can't think of any\n>>> benefit of preserving the same numerical ID between two plug events,\n>>> while I can think of quiet a few cons.\n>>>\n>>> - It's confusing when debugging as un-plugging and then replugging the\n>>>    same camera will result in logs where the numerical ID is the same\n>>> for\n>>>    both. This may even result in things working by \"chance\" is it reuses\n>>>    an already known numerical ID.\n>>>\n>>> - Looking at the code plugging a UVC camera in a different USB port will\n>>>    generate a different Camera::id() result and then be treated as a new\n>>>    camera vs plugging it in the same port and it then being treatad as a\n>>>    new camera.\n>> Ah, I just noticed this point. I assumed each camera will generate it's\n>> own unique ID, plugged in for any of the ports. I am not sure, what\n>> actions/\n>> measure we can take here to address this.\n\n\nThe 'same' camera plugged into a different USB port will be treated as a\nnew camera. I don't think we can do anything to prevent that.\n\nThere is as far as I know, no way to uniquely identify an externally\nconnected camera. Nothing is trusted, as serial numbers are rarely set\nuniquely and there is no specification on UVC to have any unique identifier.\n\nTherefore, I think matching to the unique ID we have from libcamera is\nthe best thing we can do at this layer.\n\nThat leaves any decision as to how to identify a camera to libcamera,\nnot the android layer.\n\n\n>>>\n>>> - The logic in this patch is more complex due to it both having to deal\n>>>    with known and new cameras.\n>>>\n>>> What is the benefit of this cache that I'm missing?\n>> Laurent is following up too here, so I'll wait.\n>> I basically took his suggestion from the v1 review [1] and steered the\n>> patch into that\n>> direction.\n>>\n>>>\n>>>> CameraDevice is now a shared object and cameras_ vector stores shared\n>>>> pointers to CameraDevice. This is done in order to introduce reference\n>>>> counting for CameraDevice objects - especially to handle hot-unplug\n>>>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n>>>>\n>>>> Signed-off-by: Umang Jain \n>>>> ---\n>>>>   src/android/camera_device.cpp      |  15 +++\n>>>>   src/android/camera_device.h        |   8 +-\n>>>>   src/android/camera_hal_manager.cpp | 153\n>>>> ++++++++++++++++++++++++-----\n>>>>   src/android/camera_hal_manager.h   |  15 ++-\n>>>>   4 files changed, 166 insertions(+), 25 deletions(-)\n>>>>\n>>>> diff --git a/src/android/camera_device.cpp\n>>>> b/src/android/camera_device.cpp\n>>>> index d918350..a79bb69 100644\n>>>> --- a/src/android/camera_device.cpp\n>>>> +++ b/src/android/camera_device.cpp\n>>>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n>>>>           delete it.second;\n>>>>   }\n>>>>   +std::shared_ptr CameraDevice::create(unsigned int id,\n>>>> +                            const std::shared_ptr &cam)\n>>>> +{\n>>>> +    struct Deleter : std::default_delete {\n>>>> +        void operator()(CameraDevice *camera)\n>>>> +        {\n>>>> +            delete camera;\n>>>> +        }\n>>>> +    };\n>>>> +\n>>>> +    CameraDevice *camera = new CameraDevice(id, cam);\n>>>> +\n>>>> +    return std::shared_ptr(camera, Deleter());\n>>>> +}\n>>> As Kieran points out I think this should be added in a separate as this\n>>> one is quiet large and therefore hard to review.\n>>>\n>>>> +\n>>>>   /*\n>>>>    * Initialize the camera static information.\n>>>>    * This method is called before the camera device is opened.\n>>>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n>>>> index 7be9e11..7f9e010 100644\n>>>> --- a/src/android/camera_device.h\n>>>> +++ b/src/android/camera_device.h\n>>>> @@ -47,8 +47,8 @@ struct CameraStream {\n>>>>   class CameraDevice : protected libcamera::Loggable\n>>>>   {\n>>>>   public:\n>>>> -    CameraDevice(unsigned int id, const\n>>>> std::shared_ptr &camera);\n>>>> -    ~CameraDevice();\n>>>> +    static std::shared_ptr create(unsigned int id,\n>>>> +                            const\n>>>> std::shared_ptr &cam);\n>>>>         int initialize();\n>>>>   @@ -57,6 +57,7 @@ public:\n>>>>         unsigned int id() const { return id_; }\n>>>>       camera3_device_t *camera3Device() { return &camera3Device_; }\n>>>> +    const libcamera::Camera *getCamera() { return camera_.get(); };\n>>> Same here this can be done in a separate commit. Also I think this could\n>>> be named camera() instead of getCamera()\n>> Noted. Well, this would then be a single-line patch and we won't be\n>> able to see where\n>> it's used. I am under the impression that API introduction and it's\n>> relevant usage should\n>> be done in a single commit(this patch uses the ->getCamera()) so ....\n>> one can the entire\n>> picture of the diff change.\n\n\nAs it's a single line addition, this can be in the patch that adds it, I\nthink the aim is just to try to simplify the changes in this patch a bit\n- as there are quite a few things going on so it's harder to parse.\n\n\n>>>\n>>>>         int facing() const { return facing_; }\n>>>>       int orientation() const { return orientation_; }\n>>>> @@ -72,6 +73,9 @@ protected:\n>>>>       std::string logPrefix() const override;\n>>>>     private:\n>>>> +    CameraDevice(unsigned int id, const\n>>>> std::shared_ptr &camera);\n>>>> +    ~CameraDevice();\n>>>> +\n>>>>       struct Camera3RequestDescriptor {\n>>>>           Camera3RequestDescriptor(unsigned int frameNumber,\n>>>>                        unsigned int numBuffers);\n>>>> diff --git a/src/android/camera_hal_manager.cpp\n>>>> b/src/android/camera_hal_manager.cpp\n>>>> index 3d6d2b4..fdde2c0 100644\n>>>> --- a/src/android/camera_hal_manager.cpp\n>>>> +++ b/src/android/camera_hal_manager.cpp\n>>>> @@ -8,6 +8,7 @@\n>>>>   #include \"camera_hal_manager.h\"\n>>>>     #include \n>>>> +#include \n>>>>     #include \"libcamera/internal/log.h\"\n>>>>   @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n>>>>   CameraHalManager::~CameraHalManager()\n>>>>   {\n>>>>       cameras_.clear();\n>>>> +    camerasMap_.clear();\n>>>>         if (cameraManager_) {\n>>>>           cameraManager_->stop();\n>>>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n>>>>   {\n>>>>       cameraManager_ = new CameraManager();\n>>>>   +    /* Support camera hotplug. */\n>>>> +    cameraManager_->cameraAdded.connect(this,\n>>>> &CameraHalManager::cameraAdded);\n>>>> +    cameraManager_->cameraRemoved.connect(this,\n>>>> &CameraHalManager::cameraRemoved);\n>>>> +\n>>>> +    cameraCounter_ = 0;\n>>>> +    externalCameraCounter_ = 1000;\n>>>> +\n>>>>       int ret = cameraManager_->start();\n>>>>       if (ret) {\n>>>>           LOG(HAL, Error) << \"Failed to start camera manager: \"\n>>>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n>>>>           return ret;\n>>>>       }\n>>>>   -    /*\n>>>> -     * For each Camera registered in the system, a CameraDevice\n>>>> -     * gets created here to wraps a libcamera Camera instance.\n>>>> -     *\n>>>> -     * \\todo Support camera hotplug.\n>>>> -     */\n>>>> -    unsigned int index = 0;\n>>>> -    for (auto &cam : cameraManager_->cameras()) {\n>>>> -        CameraDevice *camera = new CameraDevice(index, cam);\n>>>> -        ret = camera->initialize();\n>>>> -        if (ret)\n>>>> -            continue;\n>>>> -\n>>>> -        cameras_.emplace_back(camera);\n>>>> -        ++index;\n>>>> -    }\n>>>> -\n>>>>       return 0;\n>>>>   }\n>>>>     CameraDevice *CameraHalManager::open(unsigned int id,\n>>>>                        const hw_module_t *hardwareModule)\n>>>>   {\n>>>> -    if (id >= numCameras()) {\n>>>> +    MutexLocker locker(mutex_);\n>>>> +\n>>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>>> +                 [id](std::shared_ptr &cam) {\n>>>> +                    return cam->id() == id;\n>>>> +                });\n>>>> +    if (iter == cameras_.end()) {\n>>> I think you should break this (and the similar ones below) into private\n>>> helpers instead if implementing the logic in-place\n>>>\n>>>      CameraHalManager::cameraFromAndroidId(..);\n>>>      CameraHalManager::androidIdFromCamera(..);\n>>>>           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>>>>           return nullptr;\n>>>>       }\n>>>>   -    CameraDevice *camera = cameras_[id].get();\n>>>> +    CameraDevice *camera = iter->get();\n>>>> +\n>>>>       if (camera->open(hardwareModule))\n>>>>           return nullptr;\n>>>>   @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned\n>>>> int id,\n>>>>       return camera;\n>>>>   }\n>>>>   +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n>>>> +{\n>>>> +    unsigned int id;\n>>>> +    bool isCameraExternal = false;\n>>>> +    bool isCameraNew = false;\n>>>> +\n>>>> +    MutexLocker locker(mutex_);\n>>>> +\n>>>> +    /* Each camera is assigned a unique integer id when it is seen\n>>>> for the\n>>>> +     * first time. If the camera has been seen before, the id is\n>>>> reused and\n>>>> +     * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT\n>>>> subsequently.\n>>>> +     *\n>>>> +     * ID starts from '0' for internal cameras and '1000' for\n>>>> external cameras.\n>>>> +     */\n>>>> +    auto iter = camerasMap_.find(cam->id());\n>>>> +    if (iter != camerasMap_.end()) {\n>>>> +        id = iter->second;\n>>>> +    } else {\n>>>> +        isCameraNew = true;\n>>>> +\n>>>> +        /*\n>>>> +         *  Now check if this is an external camera and assign\n>>>> +         *  its id accordingly.\n\nOnly for consistency, there's a double space indent on the text here,\nrather than single used everywhere else.\n\n\n>>>> +         */\n>>>> +        const ControlList &properties = cam->properties();\n>>>> +        if (properties.contains(properties::Location) &&\n>>>> +            properties.get(properties::Location) &\n>>>> +            properties::CameraLocationExternal) {\n>>>> +            isCameraExternal = true;\n>>>> +            id = externalCameraCounter_;\n>>>> +        } else {\n>>>> +            id = cameraCounter_;\n>>>> +        }\n>>> As I understand it the information the HAL wants is whether or not the\n>>> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n>> Again, the spec is not clear on the hot-plugged / removable camera IDs\n>> front.\n>> It is though clear on the internal cameras IDs - should start from 0,\n>> hence,\n>> we thought 1000 is a good number for assigning IDs for external cameras.\n>>>\n>>> If so I wonder if using the camera location as a judgment of the camera\n>>> is hot-plugged or not is the way to go here? Imagine a device where the\n>>> camera is permanently attached (not hot-unpluggable) but not fixated in\n>>> a location. I'm thinking cameras mounted at the end of instruments\n>>> (medical instruments, hand held tools) or robotics (mounted at the arm\n>>> of a welding robot). I would imagine those cameras would be marked as\n>>> located externally but they would not really be hot-pluggable.\n>> Yes, I get your point. Their location is external, but from the\n>> point-of-view\n>> of HAL, they are still needed to be treated as \"internal\" cameras, no?\n>> I guess, in HAL, internal cameras are the ones, which cannot be detached\n>> from the device, whereas external cameras are the one, which can,\n>> (Just a guess, I might be entirely wrong)\n\n\nI think given that the android layer only makes the distinction of\n'internal' or 'external', that's all we need to follow currently.\n\nAs seen in other mails, I too have thought they are two distinct states,\nbut alas ... I guess we just work to 'internal' and 'external' for now.\n\n\n>>>\n>>> I understand we have no other way to report or detect this at the moment\n>>> and I'm not pushing hard for this to be solved as part of this series if\n>>> it's not easy. But I think a bigger comment here is needed explaining\n>>> that the HAL wants to know if a camera is hot-pluggable or not and does\n>>> not really care if it's located internally or externally. I also think a\n>>> \\todo should be added so it's not forgotten.\n>>>\n>>>> +    }\n>>>> +\n>>>> +    /*\n>>>> +     * For each Camera registered in the system, a CameraDevice\n>>>> +     * gets created here to wraps a libcamera Camera instance.\n>>>> +     */\n>>>> +    std::shared_ptr camera = CameraDevice::create(id,\n>>>> std::move(cam));\n>>>> +    int ret = camera->initialize();\n>>>> +    if (ret) {\n>>>> +        LOG(HAL, Error) << \"Failed to initialize camera: \" <<\n>>>> cam->id();\n>>>> +        return;\n>>>> +    }\n>>>> +\n>>>> +    if (isCameraNew) {\n>>>> +        camerasMap_.emplace(cam->id(), id);\n>>>> +\n>>>> +        if (isCameraExternal)\n>>>> +            externalCameraCounter_++;\n>>>> +        else\n>>>> +            cameraCounter_++;\n>>>> +    }\n>>>> +\n>>>> +    cameras_.emplace_back(std::move(camera));\n>>>> +\n>>>> +    if (callbacks_)\n>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n>>>> +                            CAMERA_DEVICE_STATUS_PRESENT);\n>>>> +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n>>>> +}\n>>>> +\n>>>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n>>>> +{\n>>>> +    MutexLocker locker(mutex_);\n>>>> +\n>>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>>> +                 [cam](std::shared_ptr &camera) {\n>>>> +                    return cam.get() == camera->getCamera();\n>>>> +                });\n>>>> +    if (iter == cameras_.end())\n>>>> +        return;\n>>>> +\n>>>> +    unsigned int id = (*iter)->id();\n>>>> +    callbacks_->camera_device_status_change(callbacks_, id,\n>>>> +                        CAMERA_DEVICE_STATUS_NOT_PRESENT);\n>>>> +    cameras_.erase(iter);\n>>>> +\n>>>> +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed\n>>>> successfully.\";\n>>>> +}\n>>>> +\n>>>>   unsigned int CameraHalManager::numCameras() const\n>>>>   {\n>>>> -    return cameraManager_->cameras().size();\n>>>> +    return cameraCounter_;\n>>>>   }\n>>>>     int CameraHalManager::getCameraInfo(unsigned int id, struct\n>>>> camera_info *info)\n>>>> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned\n>>>> int id, struct camera_info *info)\n>>>>       if (!info)\n>>>>           return -EINVAL;\n>>>>   -    if (id >= numCameras()) {\n>>>> +    MutexLocker locker(mutex_);\n>>>> +\n>>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>>> +                 [id](std::shared_ptr &cam) {\n>>>> +                    return cam->id() == id;\n>>>> +                });\n>>>> +    if (iter == cameras_.end()) {\n>>>>           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>>>>           return -EINVAL;\n>>>>       }\n>>>>   -    CameraDevice *camera = cameras_[id].get();\n>>>> +    CameraDevice *camera = iter->get();\n>>>>         info->facing = camera->facing();\n>>>>       info->orientation = camera->orientation();\n>>>> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned\n>>>> int id, struct camera_info *info)\n>>>>   void CameraHalManager::setCallbacks(const\n>>>> camera_module_callbacks_t *callbacks)\n>>>>   {\n>>>>       callbacks_ = callbacks;\n>>>> +\n>>>> +    MutexLocker locker(mutex_);\n>>>> +    /*\n>>>> +     * Few cameras might have been hotplugged before setting\n>>>> callbacks_ here.\n>>>> +     * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them\n>>>> explicitly.\n>>>> +     * This hold only for external cameras, as internal cameras are\n>>>> assumed to\n>>>> +     * be present at module load time, by the framework.\n>>>> +     */\n>>>> +    for (auto &cam : cameraManager_->cameras()) {\n>>>> +        auto iter = camerasMap_.find(cam->id());\n>>>> +        if (iter == camerasMap_.end())\n>>>> +            continue;\n>>>> +\n>>>> +        unsigned int id = iter->second;\n>>>> +        const ControlList &properties = cam->properties();\n>>>> +        if (properties.contains(properties::Location) &&\n>>>> +            properties.get(properties::Location) &\n>>>> +            properties::CameraLocationExternal) {\n>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n>>>> +        }\n>>>> +    }\n>>>>   }\n>>>> diff --git a/src/android/camera_hal_manager.h\n>>>> b/src/android/camera_hal_manager.h\n>>>> index a582f04..7c481d4 100644\n>>>> --- a/src/android/camera_hal_manager.h\n>>>> +++ b/src/android/camera_hal_manager.h\n>>>> @@ -7,6 +7,8 @@\n>>>>   #ifndef __ANDROID_CAMERA_MANAGER_H__\n>>>>   #define __ANDROID_CAMERA_MANAGER_H__\n>>>>   +#include \n>>>> +#include \n>>>>   #include \n>>>>   #include \n>>>>   @@ -18,6 +20,9 @@\n>>>>     class CameraDevice;\n>>>>   +using Mutex = std::mutex;\n>>>> +using MutexLocker = std::unique_lock;\n>>>> +\n>>>>   class CameraHalManager\n>>>>   {\n>>>>   public:\n>>>> @@ -33,10 +38,18 @@ public:\n>>>>       void setCallbacks(const camera_module_callbacks_t *callbacks);\n>>>>     private:\n>>>> +    void cameraAdded(std::shared_ptr cam);\n>>>> +    void cameraRemoved(std::shared_ptr cam);\n>>>> +\n>>>>       libcamera::CameraManager *cameraManager_;\n>>>>         const camera_module_callbacks_t *callbacks_;\n>>>> -    std::vector> cameras_;\n>>>> +    std::vector> cameras_;\n>>>> +    std::map camerasMap_;\n>>> If each hot-plugged camera where treated as a new camera cameras_ and\n>>> camerasMap_ could be merged to a\n>>>\n>>>      std::map> cameras_;\n>>>\n>>> Which would eliminate the possibility of them going out-of-sync.\n>> Great point. I hope it's not too cubersome to iterate over all the\n>> entires,\n>> to find the exact CameraDevice we want for various plug events handler.\n>> I will look into it.\n> I found this to be a bit cubersome to implement. The foundation issue\n> here is:\n> we do need to cache the android HAL's ID<->libcamera::Camera::id mapping\n> to identify if we are replugging an already-seen camera (I am assuming that\n> libcamera::Camera::id is a unique ID generated by libcamera for each camera\n> and is constant whenever the camera is replugged).\n> \n> If I just maintain a single map std::map std::shared_ptr>\n> the CameraDevice will be deleted on hot-unplug. Then I do not have a\n> comparator to\n> compare against, to know if the ID key should be reused or not, when\n> hotplug event\n> handlers.\n\n\nIndeed, we can't map to something that would disappear, and we shouldn't\nkeep the CameraDevice around longer than necessary after it has been\nunplugged.\n\nI think a map of (android int) id to (unique string libcamera) id is\nsufficient.\n\n\n\n>>>\n>>>> +    Mutex mutex_;\n>>>> +\n>>>> +    unsigned int externalCameraCounter_;\n>>>> +    unsigned int cameraCounter_;\n>>>>   };\n>>>>     #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>>>> -- \n>>>> 2.26.2\n>>>>\n>>>> _______________________________________________\n>>>> libcamera-devel mailing list\n>>>> libcamera-devel@lists.libcamera.org\n>>>> https://lists.libcamera.org/listinfo/libcamera-devel\n>> [1] :\n>> https://lists.libcamera.org/pipermail/libcamera-devel/2020-August/011857.html\n>>\n>> _______________________________________________\n>> libcamera-devel mailing list\n>> libcamera-devel@lists.libcamera.org\n>> https://lists.libcamera.org/listinfo/libcamera-devel\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D7A07BD87D\n\tfor ;\n\tFri, 14 Aug 2020 08:46:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 584C0613D6;\n\tFri, 14 Aug 2020 10:46:04 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id EA09360386\n\tfor ;\n\tFri, 14 Aug 2020 10:46:02 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D9022331;\n\tFri, 14 Aug 2020 10:46:01 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"GhHMHm0Q\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597394762;\n\tbh=3yWil+8J/hqN862HXRk8HshM+fzP2AOtuvK6uEK7rP0=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=GhHMHm0QcL94VTu14F1UQd1sEU6lJWMWQkn8aSh5DBm/zDao0O9oRcG7/vTmkdy33\n\tQb8VsvXZbZYzjwPuNC/bBvwYpczbm2I+PuU2EEE+BUApDYHO24nBeiQ4F8OjulwVWU\n\tXzkl5X/BOY9HBDR0ZMiwGW1Uz6kkyjhYpGnvVxEo=","To":"Umang Jain , =?utf-8?q?Niklas_S=C3=B6derlund?=\n\t","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>\n\t<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<2e465494-31e1-88d4-9752-6339dee02bea@ideasonboard.com>","Date":"Fri, 14 Aug 2020 09:45:58 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12009,"web_url":"https://patchwork.libcamera.org/comment/12009/","msgid":"<20200814092555.GA1995759@oden.dyn.berto.se>","date":"2020-08-14T09:25:55","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/people/5/","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"content":"Hi Umang,\n\nOn 2020-08-14 08:29:58 +0000, Umang Jain wrote:\n> Hi Niklas,\n> \n> On 8/13/20 5:52 PM, Umang Jain wrote:\n> > Hi Niklas\n> > \n> > On 8/13/20 5:14 PM, Niklas Söderlund wrote:\n> > > Hi Umang,\n> > > \n> > > Thanks for your work.\n> > > \n> > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > know about the hotplug events and change the status of cameras being\n> > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > \n> > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > current implementation, we assume all UVC cameras are external cameras.\n> > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > benefit of preserving the same numerical ID between two plug events,\n> > > while I can think of quiet a few cons.\n> > > \n> > > - It's confusing when debugging as un-plugging and then replugging the\n> > >    same camera will result in logs where the numerical ID is the\n> > > same for\n> > >    both. This may even result in things working by \"chance\" is it reuses\n> > >    an already known numerical ID.\n> > > \n> > > - Looking at the code plugging a UVC camera in a different USB port will\n> > >    generate a different Camera::id() result and then be treated as a new\n> > >    camera vs plugging it in the same port and it then being treatad as a\n> > >    new camera.\n> > Ah, I just noticed this point. I assumed each camera will generate it's\n> > own unique ID, plugged in for any of the ports. I am not sure, what\n> > actions/\n> > measure we can take here to address this.\n> > > \n> > > - The logic in this patch is more complex due to it both having to deal\n> > >    with known and new cameras.\n> > > \n> > > What is the benefit of this cache that I'm missing?\n> > Laurent is following up too here, so I'll wait.\n> > I basically took his suggestion from the v1 review [1] and steered the\n> > patch into that\n> > direction.\n> > \n> > > \n> > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > \n> > > > Signed-off-by: Umang Jain \n> > > > ---\n> > > >   src/android/camera_device.cpp      |  15 +++\n> > > >   src/android/camera_device.h        |   8 +-\n> > > >   src/android/camera_hal_manager.cpp | 153\n> > > > ++++++++++++++++++++++++-----\n> > > >   src/android/camera_hal_manager.h   |  15 ++-\n> > > >   4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > \n> > > > diff --git a/src/android/camera_device.cpp\n> > > > b/src/android/camera_device.cpp\n> > > > index d918350..a79bb69 100644\n> > > > --- a/src/android/camera_device.cpp\n> > > > +++ b/src/android/camera_device.cpp\n> > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > >           delete it.second;\n> > > >   }\n> > > >   +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > +                            const std::shared_ptr &cam)\n> > > > +{\n> > > > +    struct Deleter : std::default_delete {\n> > > > +        void operator()(CameraDevice *camera)\n> > > > +        {\n> > > > +            delete camera;\n> > > > +        }\n> > > > +    };\n> > > > +\n> > > > +    CameraDevice *camera = new CameraDevice(id, cam);\n> > > > +\n> > > > +    return std::shared_ptr(camera, Deleter());\n> > > > +}\n> > > As Kieran points out I think this should be added in a separate as this\n> > > one is quiet large and therefore hard to review.\n> > > \n> > > > +\n> > > >   /*\n> > > >    * Initialize the camera static information.\n> > > >    * This method is called before the camera device is opened.\n> > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > index 7be9e11..7f9e010 100644\n> > > > --- a/src/android/camera_device.h\n> > > > +++ b/src/android/camera_device.h\n> > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > >   class CameraDevice : protected libcamera::Loggable\n> > > >   {\n> > > >   public:\n> > > > -    CameraDevice(unsigned int id, const\n> > > > std::shared_ptr &camera);\n> > > > -    ~CameraDevice();\n> > > > +    static std::shared_ptr create(unsigned int id,\n> > > > +                            const\n> > > > std::shared_ptr &cam);\n> > > >         int initialize();\n> > > >   @@ -57,6 +57,7 @@ public:\n> > > >         unsigned int id() const { return id_; }\n> > > >       camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > +    const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > Same here this can be done in a separate commit. Also I think this could\n> > > be named camera() instead of getCamera()\n> > Noted. Well, this would then be a single-line patch and we won't be able\n> > to see where\n> > it's used. I am under the impression that API introduction and it's\n> > relevant usage should\n> > be done in a single commit(this patch uses the ->getCamera()) so ....\n> > one can the entire\n> > picture of the diff change.\n> > > \n> > > >         int facing() const { return facing_; }\n> > > >       int orientation() const { return orientation_; }\n> > > > @@ -72,6 +73,9 @@ protected:\n> > > >       std::string logPrefix() const override;\n> > > >     private:\n> > > > +    CameraDevice(unsigned int id, const\n> > > > std::shared_ptr &camera);\n> > > > +    ~CameraDevice();\n> > > > +\n> > > >       struct Camera3RequestDescriptor {\n> > > >           Camera3RequestDescriptor(unsigned int frameNumber,\n> > > >                        unsigned int numBuffers);\n> > > > diff --git a/src/android/camera_hal_manager.cpp\n> > > > b/src/android/camera_hal_manager.cpp\n> > > > index 3d6d2b4..fdde2c0 100644\n> > > > --- a/src/android/camera_hal_manager.cpp\n> > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > @@ -8,6 +8,7 @@\n> > > >   #include \"camera_hal_manager.h\"\n> > > >     #include \n> > > > +#include \n> > > >     #include \"libcamera/internal/log.h\"\n> > > >   @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > >   CameraHalManager::~CameraHalManager()\n> > > >   {\n> > > >       cameras_.clear();\n> > > > +    camerasMap_.clear();\n> > > >         if (cameraManager_) {\n> > > >           cameraManager_->stop();\n> > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > >   {\n> > > >       cameraManager_ = new CameraManager();\n> > > >   +    /* Support camera hotplug. */\n> > > > +    cameraManager_->cameraAdded.connect(this,\n> > > > &CameraHalManager::cameraAdded);\n> > > > +    cameraManager_->cameraRemoved.connect(this,\n> > > > &CameraHalManager::cameraRemoved);\n> > > > +\n> > > > +    cameraCounter_ = 0;\n> > > > +    externalCameraCounter_ = 1000;\n> > > > +\n> > > >       int ret = cameraManager_->start();\n> > > >       if (ret) {\n> > > >           LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > >           return ret;\n> > > >       }\n> > > >   -    /*\n> > > > -     * For each Camera registered in the system, a CameraDevice\n> > > > -     * gets created here to wraps a libcamera Camera instance.\n> > > > -     *\n> > > > -     * \\todo Support camera hotplug.\n> > > > -     */\n> > > > -    unsigned int index = 0;\n> > > > -    for (auto &cam : cameraManager_->cameras()) {\n> > > > -        CameraDevice *camera = new CameraDevice(index, cam);\n> > > > -        ret = camera->initialize();\n> > > > -        if (ret)\n> > > > -            continue;\n> > > > -\n> > > > -        cameras_.emplace_back(camera);\n> > > > -        ++index;\n> > > > -    }\n> > > > -\n> > > >       return 0;\n> > > >   }\n> > > >     CameraDevice *CameraHalManager::open(unsigned int id,\n> > > >                        const hw_module_t *hardwareModule)\n> > > >   {\n> > > > -    if (id >= numCameras()) {\n> > > > +    MutexLocker locker(mutex_);\n> > > > +\n> > > > +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > +                 [id](std::shared_ptr &cam) {\n> > > > +                    return cam->id() == id;\n> > > > +                });\n> > > > +    if (iter == cameras_.end()) {\n> > > I think you should break this (and the similar ones below) into private\n> > > helpers instead if implementing the logic in-place\n> > > \n> > >      CameraHalManager::cameraFromAndroidId(..);\n> > >      CameraHalManager::androidIdFromCamera(..);\n> > > >           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > >           return nullptr;\n> > > >       }\n> > > >   -    CameraDevice *camera = cameras_[id].get();\n> > > > +    CameraDevice *camera = iter->get();\n> > > > +\n> > > >       if (camera->open(hardwareModule))\n> > > >           return nullptr;\n> > > >   @@ -93,9 +92,91 @@ CameraDevice\n> > > > *CameraHalManager::open(unsigned int id,\n> > > >       return camera;\n> > > >   }\n> > > >   +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > +{\n> > > > +    unsigned int id;\n> > > > +    bool isCameraExternal = false;\n> > > > +    bool isCameraNew = false;\n> > > > +\n> > > > +    MutexLocker locker(mutex_);\n> > > > +\n> > > > +    /* Each camera is assigned a unique integer id when it is\n> > > > seen for the\n> > > > +     * first time. If the camera has been seen before, the id\n> > > > is reused and\n> > > > +     * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT\n> > > > subsequently.\n> > > > +     *\n> > > > +     * ID starts from '0' for internal cameras and '1000' for\n> > > > external cameras.\n> > > > +     */\n> > > > +    auto iter = camerasMap_.find(cam->id());\n> > > > +    if (iter != camerasMap_.end()) {\n> > > > +        id = iter->second;\n> > > > +    } else {\n> > > > +        isCameraNew = true;\n> > > > +\n> > > > +        /*\n> > > > +         *  Now check if this is an external camera and assign\n> > > > +         *  its id accordingly.\n> > > > +         */\n> > > > +        const ControlList &properties = cam->properties();\n> > > > +        if (properties.contains(properties::Location) &&\n> > > > +            properties.get(properties::Location) &\n> > > > +            properties::CameraLocationExternal) {\n> > > > +            isCameraExternal = true;\n> > > > +            id = externalCameraCounter_;\n> > > > +        } else {\n> > > > +            id = cameraCounter_;\n> > > > +        }\n> > > As I understand it the information the HAL wants is whether or not the\n> > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > Again, the spec is not clear on the hot-plugged / removable camera IDs\n> > front.\n> > It is though clear on the internal cameras IDs - should start from 0,\n> > hence,\n> > we thought 1000 is a good number for assigning IDs for external cameras.\n> > > \n> > > If so I wonder if using the camera location as a judgment of the camera\n> > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > a location. I'm thinking cameras mounted at the end of instruments\n> > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > of a welding robot). I would imagine those cameras would be marked as\n> > > located externally but they would not really be hot-pluggable.\n> > Yes, I get your point. Their location is external, but from the\n> > point-of-view\n> > of HAL, they are still needed to be treated as \"internal\" cameras, no?\n> > I guess, in HAL, internal cameras are the ones, which cannot be detached\n> > from the device, whereas external cameras are the one, which can,\n> > (Just a guess, I might be entirely wrong)\n> > > \n> > > I understand we have no other way to report or detect this at the moment\n> > > and I'm not pushing hard for this to be solved as part of this series if\n> > > it's not easy. But I think a bigger comment here is needed explaining\n> > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > not really care if it's located internally or externally. I also think a\n> > > \\todo should be added so it's not forgotten.\n> > > \n> > > > +    }\n> > > > +\n> > > > +    /*\n> > > > +     * For each Camera registered in the system, a CameraDevice\n> > > > +     * gets created here to wraps a libcamera Camera instance.\n> > > > +     */\n> > > > +    std::shared_ptr camera =\n> > > > CameraDevice::create(id, std::move(cam));\n> > > > +    int ret = camera->initialize();\n> > > > +    if (ret) {\n> > > > +        LOG(HAL, Error) << \"Failed to initialize camera: \" <<\n> > > > cam->id();\n> > > > +        return;\n> > > > +    }\n> > > > +\n> > > > +    if (isCameraNew) {\n> > > > +        camerasMap_.emplace(cam->id(), id);\n> > > > +\n> > > > +        if (isCameraExternal)\n> > > > +            externalCameraCounter_++;\n> > > > +        else\n> > > > +            cameraCounter_++;\n> > > > +    }\n> > > > +\n> > > > +    cameras_.emplace_back(std::move(camera));\n> > > > +\n> > > > +    if (callbacks_)\n> > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > +                            CAMERA_DEVICE_STATUS_PRESENT);\n> > > > +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > +}\n> > > > +\n> > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > +{\n> > > > +    MutexLocker locker(mutex_);\n> > > > +\n> > > > +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > +                 [cam](std::shared_ptr &camera) {\n> > > > +                    return cam.get() == camera->getCamera();\n> > > > +                });\n> > > > +    if (iter == cameras_.end())\n> > > > +        return;\n> > > > +\n> > > > +    unsigned int id = (*iter)->id();\n> > > > +    callbacks_->camera_device_status_change(callbacks_, id,\n> > > > +                        CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > +    cameras_.erase(iter);\n> > > > +\n> > > > +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed\n> > > > successfully.\";\n> > > > +}\n> > > > +\n> > > >   unsigned int CameraHalManager::numCameras() const\n> > > >   {\n> > > > -    return cameraManager_->cameras().size();\n> > > > +    return cameraCounter_;\n> > > >   }\n> > > >     int CameraHalManager::getCameraInfo(unsigned int id, struct\n> > > > camera_info *info)\n> > > > @@ -103,12 +184,18 @@ int\n> > > > CameraHalManager::getCameraInfo(unsigned int id, struct\n> > > > camera_info *info)\n> > > >       if (!info)\n> > > >           return -EINVAL;\n> > > >   -    if (id >= numCameras()) {\n> > > > +    MutexLocker locker(mutex_);\n> > > > +\n> > > > +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > +                 [id](std::shared_ptr &cam) {\n> > > > +                    return cam->id() == id;\n> > > > +                });\n> > > > +    if (iter == cameras_.end()) {\n> > > >           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > >           return -EINVAL;\n> > > >       }\n> > > >   -    CameraDevice *camera = cameras_[id].get();\n> > > > +    CameraDevice *camera = iter->get();\n> > > >         info->facing = camera->facing();\n> > > >       info->orientation = camera->orientation();\n> > > > @@ -124,4 +211,26 @@ int\n> > > > CameraHalManager::getCameraInfo(unsigned int id, struct\n> > > > camera_info *info)\n> > > >   void CameraHalManager::setCallbacks(const\n> > > > camera_module_callbacks_t *callbacks)\n> > > >   {\n> > > >       callbacks_ = callbacks;\n> > > > +\n> > > > +    MutexLocker locker(mutex_);\n> > > > +    /*\n> > > > +     * Few cameras might have been hotplugged before setting\n> > > > callbacks_ here.\n> > > > +     * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them\n> > > > explicitly.\n> > > > +     * This hold only for external cameras, as internal cameras\n> > > > are assumed to\n> > > > +     * be present at module load time, by the framework.\n> > > > +     */\n> > > > +    for (auto &cam : cameraManager_->cameras()) {\n> > > > +        auto iter = camerasMap_.find(cam->id());\n> > > > +        if (iter == camerasMap_.end())\n> > > > +            continue;\n> > > > +\n> > > > +        unsigned int id = iter->second;\n> > > > +        const ControlList &properties = cam->properties();\n> > > > +        if (properties.contains(properties::Location) &&\n> > > > +            properties.get(properties::Location) &\n> > > > +            properties::CameraLocationExternal) {\n> > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > +        }\n> > > > +    }\n> > > >   }\n> > > > diff --git a/src/android/camera_hal_manager.h\n> > > > b/src/android/camera_hal_manager.h\n> > > > index a582f04..7c481d4 100644\n> > > > --- a/src/android/camera_hal_manager.h\n> > > > +++ b/src/android/camera_hal_manager.h\n> > > > @@ -7,6 +7,8 @@\n> > > >   #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > >   #define __ANDROID_CAMERA_MANAGER_H__\n> > > >   +#include \n> > > > +#include \n> > > >   #include \n> > > >   #include \n> > > >   @@ -18,6 +20,9 @@\n> > > >     class CameraDevice;\n> > > >   +using Mutex = std::mutex;\n> > > > +using MutexLocker = std::unique_lock;\n> > > > +\n> > > >   class CameraHalManager\n> > > >   {\n> > > >   public:\n> > > > @@ -33,10 +38,18 @@ public:\n> > > >       void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > >     private:\n> > > > +    void cameraAdded(std::shared_ptr cam);\n> > > > +    void cameraRemoved(std::shared_ptr cam);\n> > > > +\n> > > >       libcamera::CameraManager *cameraManager_;\n> > > >         const camera_module_callbacks_t *callbacks_;\n> > > > -    std::vector> cameras_;\n> > > > +    std::vector> cameras_;\n> > > > +    std::map camerasMap_;\n> > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > camerasMap_ could be merged to a\n> > > \n> > >      std::map> cameras_;\n> > > \n> > > Which would eliminate the possibility of them going out-of-sync.\n> > Great point. I hope it's not too cubersome to iterate over all the\n> > entires,\n> > to find the exact CameraDevice we want for various plug events handler.\n> > I will look into it.\n> I found this to be a bit cubersome to implement. The foundation issue here\n> is:\n> we do need to cache the android HAL's ID<->libcamera::Camera::id mapping\n> to identify if we are replugging an already-seen camera (I am assuming that\n> libcamera::Camera::id is a unique ID generated by libcamera for each camera\n> and is constant whenever the camera is replugged).\n> \n> If I just maintain a single map std::map std::shared_ptr>\n> the CameraDevice will be deleted on hot-unplug. Then I do not have a\n> comparator to\n> compare against, to know if the ID key should be reused or not, when hotplug\n> event\n> handlers.\n\nAs I stated above, having a single map is only a good simplification \n_if_ we treat each camera as a new camera. That is that we do not cache \nthe cameras in order to reuse the numerical IDs.\n\nIf we need to cache the ID information to fulfill some requirement of \nthe HAL design then my suggestion of a single map is unsuitable, as you \npoint.\n\n> > > \n> > > > +    Mutex mutex_;\n> > > > +\n> > > > +    unsigned int externalCameraCounter_;\n> > > > +    unsigned int cameraCounter_;\n> > > >   };\n> > > >     #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n> > > > -- \n> > > > 2.26.2\n> > > > \n> > > > _______________________________________________\n> > > > libcamera-devel mailing list\n> > > > libcamera-devel@lists.libcamera.org\n> > > > https://lists.libcamera.org/listinfo/libcamera-devel\n> > [1] : https://lists.libcamera.org/pipermail/libcamera-devel/2020-August/011857.html\n> > \n> > _______________________________________________\n> > libcamera-devel mailing list\n> > libcamera-devel@lists.libcamera.org\n> > https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 349FABD87C\n\tfor ;\n\tFri, 14 Aug 2020 09:26:01 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B60FF613B4;\n\tFri, 14 Aug 2020 11:26:00 +0200 (CEST)","from mail-lj1-x241.google.com (mail-lj1-x241.google.com\n\t[IPv6:2a00:1450:4864:20::241])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C94060386\n\tfor ;\n\tFri, 14 Aug 2020 11:25:58 +0200 (CEST)","by mail-lj1-x241.google.com with SMTP id t6so9195564ljk.9\n\tfor ;\n\tFri, 14 Aug 2020 02:25:58 -0700 (PDT)","from localhost (h-209-203.A463.priv.bahnhof.se. [155.4.209.203])\n\tby smtp.gmail.com with ESMTPSA id\n\tq22sm1757503lfc.33.2020.08.14.02.25.56\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tFri, 14 Aug 2020 02:25:56 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ragnatech-se.20150623.gappssmtp.com\n\theader.i=@ragnatech-se.20150623.gappssmtp.com\n\theader.b=\"s8WxjyXG\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ragnatech-se.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to;\n\tbh=z1yn5wtMZ+xZ9OfYR1EHj7YBGlBYXeR/SH05+GFXAbw=;\n\tb=s8WxjyXGqFR7aAnmHK+l3KMsKCgATWODZ25kB7NiDq/mey9Vm3mpTBChamSjWYTIG4\n\t9FPrihOPt9i1DJAThOmPaSzbsOUCRWDNypfr5NUgO32vYrjW6bN8uX7XQAS/CBrTmrHo\n\tNGaoujwUrH2BSnreIRu9f5wWVdwlW/vnhqpxge5criymrvSPw1HrERPjYhO38j1EQ8Si\n\tDBo4ygmpHAQ9T3nHaTlrTRCeRDdM3Y79p7K6CXzKN2g6s+EO73T2gOOvWjZVbC3uoaP1\n\t7cP8JrIzph/ARYPPW/kwJITZuI8FYxNlJP8MMkQ0DdDLmbl2y0jtlmiuA2tZPuU8vL4M\n\t3L+Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to;\n\tbh=z1yn5wtMZ+xZ9OfYR1EHj7YBGlBYXeR/SH05+GFXAbw=;\n\tb=f30AzCHLgC945bU5fhtFDBapXHrJmz+CfzZY0yUn2OWyriKV3nQb5MKRVQkjCdOV3r\n\t/pfyZO3ZsX3PBPELtYHBx0/zcsE8kGeOVBmJIssoUvlLryvf8+HXjSVuUAYouMMIlzBA\n\t3Mk1CpXz8/WeB6MfXU92NeYFnK/Fo2cDtXjjchqdCOHKSJAmC739xss98fPEHPGganGi\n\tV7b8rwjb21ewOavx6Y+IXfdJ+Wk1ybWZrMG0uhPgD38z+d66cwYbnVqs/89IjPtQiVXx\n\tiv6tpO2ttevdO1lQq/AOiyEO5fIxSuSkB562olZLmcmkWbIWquCmiaYQzYycq42rOZKz\n\tbGYg==","X-Gm-Message-State":"AOAM530MdJl2s2wgYbN+mdq9x73q/LkjTyVN6nmFLug25CDN7kF40vQY\n\tG3O3m4r6gNcpui4ZrdGZS/zqlavgG4w9pQ==","X-Google-Smtp-Source":"ABdhPJx8bAB00I8luKXbdf2mmTeZzntqEKw0BubOhB00tiW1/BzPfQ1IHeFHvLyI3oRoVmBOPvw8lw==","X-Received":"by 2002:a2e:9a91:: with SMTP id p17mr972524lji.378.1597397157171;\n\tFri, 14 Aug 2020 02:25:57 -0700 (PDT)","Date":"Fri, 14 Aug 2020 11:25:55 +0200","From":"Niklas =?iso-8859-1?q?S=F6derlund?= ","To":"Umang Jain ","Message-ID":"<20200814092555.GA1995759@oden.dyn.berto.se>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>\n\t<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"iso-8859-1\"","Content-Transfer-Encoding":"quoted-printable","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12018,"web_url":"https://patchwork.libcamera.org/comment/12018/","msgid":"<18d184eb-e8a1-aa96-7e7b-59c452782492@uajain.com>","date":"2020-08-14T11:53:07","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":1,"url":"https://patchwork.libcamera.org/api/people/1/","name":"Umang Jain","email":"email@uajain.com"},"content":"Hi Niklas,\n\nOn 8/14/20 2:55 PM, Niklas Söderlund wrote:\n> Hi Umang,\n>\n> On 2020-08-14 08:29:58 +0000, Umang Jain wrote:\n>> Hi Niklas,\n>>\n>> On 8/13/20 5:52 PM, Umang Jain wrote:\n>>> Hi Niklas\n>>>\n>>> On 8/13/20 5:14 PM, Niklas Söderlund wrote:\n>>>> Hi Umang,\n>>>>\n>>>> Thanks for your work.\n>>>>\n>>>> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n>>>>> Extend the support for camera hotplug from libcamera's CameraManager\n>>>>> to CameraHalManager. Use camera module callbacks to let the framework\n>>>>> know about the hotplug events and change the status of cameras being\n>>>>> being hotplugged or unplugged via camera_device_status_change().\n>>>>>\n>>>>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n>>>>> past by the CameraHalManager. If the camera is seen for the first time,\n>>>>> a new id is assigned to it. If the camera has been seen before by the\n>>>>> manager, it's old id is reused. IDs for internal cameras start with\n>>>>> '0' and for external cameras, they start with '1000'. Note, for the\n>>>>> current implementation, we assume all UVC cameras are external cameras.\n>>>> I wonder if keeping the cache of previously seen cameras or if we should\n>>>> not just treat any hot-plugged camera as a new one? I can't think of any\n>>>> benefit of preserving the same numerical ID between two plug events,\n>>>> while I can think of quiet a few cons.\n>>>>\n>>>> - It's confusing when debugging as un-plugging and then replugging the\n>>>>    same camera will result in logs where the numerical ID is the\n>>>> same for\n>>>>    both. This may even result in things working by \"chance\" is it reuses\n>>>>    an already known numerical ID.\n>>>>\n>>>> - Looking at the code plugging a UVC camera in a different USB port will\n>>>>    generate a different Camera::id() result and then be treated as a new\n>>>>    camera vs plugging it in the same port and it then being treatad as a\n>>>>    new camera.\n>>> Ah, I just noticed this point. I assumed each camera will generate it's\n>>> own unique ID, plugged in for any of the ports. I am not sure, what\n>>> actions/\n>>> measure we can take here to address this.\n>>>> - The logic in this patch is more complex due to it both having to deal\n>>>>    with known and new cameras.\n>>>>\n>>>> What is the benefit of this cache that I'm missing?\n>>> Laurent is following up too here, so I'll wait.\n>>> I basically took his suggestion from the v1 review [1] and steered the\n>>> patch into that\n>>> direction.\n>>>\n>>>>> CameraDevice is now a shared object and cameras_ vector stores shared\n>>>>> pointers to CameraDevice. This is done in order to introduce reference\n>>>>> counting for CameraDevice objects - especially to handle hot-unplug\n>>>>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n>>>>>\n>>>>> Signed-off-by: Umang Jain \n>>>>> ---\n>>>>>   src/android/camera_device.cpp      |  15 +++\n>>>>>   src/android/camera_device.h        |   8 +-\n>>>>>   src/android/camera_hal_manager.cpp | 153\n>>>>> ++++++++++++++++++++++++-----\n>>>>>   src/android/camera_hal_manager.h   |  15 ++-\n>>>>>   4 files changed, 166 insertions(+), 25 deletions(-)\n>>>>>\n>>>>> diff --git a/src/android/camera_device.cpp\n>>>>> b/src/android/camera_device.cpp\n>>>>> index d918350..a79bb69 100644\n>>>>> --- a/src/android/camera_device.cpp\n>>>>> +++ b/src/android/camera_device.cpp\n>>>>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n>>>>>           delete it.second;\n>>>>>   }\n>>>>>   +std::shared_ptr CameraDevice::create(unsigned int id,\n>>>>> +                            const std::shared_ptr &cam)\n>>>>> +{\n>>>>> +    struct Deleter : std::default_delete {\n>>>>> +        void operator()(CameraDevice *camera)\n>>>>> +        {\n>>>>> +            delete camera;\n>>>>> +        }\n>>>>> +    };\n>>>>> +\n>>>>> +    CameraDevice *camera = new CameraDevice(id, cam);\n>>>>> +\n>>>>> +    return std::shared_ptr(camera, Deleter());\n>>>>> +}\n>>>> As Kieran points out I think this should be added in a separate as this\n>>>> one is quiet large and therefore hard to review.\n>>>>\n>>>>> +\n>>>>>   /*\n>>>>>    * Initialize the camera static information.\n>>>>>    * This method is called before the camera device is opened.\n>>>>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n>>>>> index 7be9e11..7f9e010 100644\n>>>>> --- a/src/android/camera_device.h\n>>>>> +++ b/src/android/camera_device.h\n>>>>> @@ -47,8 +47,8 @@ struct CameraStream {\n>>>>>   class CameraDevice : protected libcamera::Loggable\n>>>>>   {\n>>>>>   public:\n>>>>> -    CameraDevice(unsigned int id, const\n>>>>> std::shared_ptr &camera);\n>>>>> -    ~CameraDevice();\n>>>>> +    static std::shared_ptr create(unsigned int id,\n>>>>> +                            const\n>>>>> std::shared_ptr &cam);\n>>>>>         int initialize();\n>>>>>   @@ -57,6 +57,7 @@ public:\n>>>>>         unsigned int id() const { return id_; }\n>>>>>       camera3_device_t *camera3Device() { return &camera3Device_; }\n>>>>> +    const libcamera::Camera *getCamera() { return camera_.get(); };\n>>>> Same here this can be done in a separate commit. Also I think this could\n>>>> be named camera() instead of getCamera()\n>>> Noted. Well, this would then be a single-line patch and we won't be able\n>>> to see where\n>>> it's used. I am under the impression that API introduction and it's\n>>> relevant usage should\n>>> be done in a single commit(this patch uses the ->getCamera()) so ....\n>>> one can the entire\n>>> picture of the diff change.\n>>>>>         int facing() const { return facing_; }\n>>>>>       int orientation() const { return orientation_; }\n>>>>> @@ -72,6 +73,9 @@ protected:\n>>>>>       std::string logPrefix() const override;\n>>>>>     private:\n>>>>> +    CameraDevice(unsigned int id, const\n>>>>> std::shared_ptr &camera);\n>>>>> +    ~CameraDevice();\n>>>>> +\n>>>>>       struct Camera3RequestDescriptor {\n>>>>>           Camera3RequestDescriptor(unsigned int frameNumber,\n>>>>>                        unsigned int numBuffers);\n>>>>> diff --git a/src/android/camera_hal_manager.cpp\n>>>>> b/src/android/camera_hal_manager.cpp\n>>>>> index 3d6d2b4..fdde2c0 100644\n>>>>> --- a/src/android/camera_hal_manager.cpp\n>>>>> +++ b/src/android/camera_hal_manager.cpp\n>>>>> @@ -8,6 +8,7 @@\n>>>>>   #include \"camera_hal_manager.h\"\n>>>>>     #include \n>>>>> +#include \n>>>>>     #include \"libcamera/internal/log.h\"\n>>>>>   @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n>>>>>   CameraHalManager::~CameraHalManager()\n>>>>>   {\n>>>>>       cameras_.clear();\n>>>>> +    camerasMap_.clear();\n>>>>>         if (cameraManager_) {\n>>>>>           cameraManager_->stop();\n>>>>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n>>>>>   {\n>>>>>       cameraManager_ = new CameraManager();\n>>>>>   +    /* Support camera hotplug. */\n>>>>> +    cameraManager_->cameraAdded.connect(this,\n>>>>> &CameraHalManager::cameraAdded);\n>>>>> +    cameraManager_->cameraRemoved.connect(this,\n>>>>> &CameraHalManager::cameraRemoved);\n>>>>> +\n>>>>> +    cameraCounter_ = 0;\n>>>>> +    externalCameraCounter_ = 1000;\n>>>>> +\n>>>>>       int ret = cameraManager_->start();\n>>>>>       if (ret) {\n>>>>>           LOG(HAL, Error) << \"Failed to start camera manager: \"\n>>>>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n>>>>>           return ret;\n>>>>>       }\n>>>>>   -    /*\n>>>>> -     * For each Camera registered in the system, a CameraDevice\n>>>>> -     * gets created here to wraps a libcamera Camera instance.\n>>>>> -     *\n>>>>> -     * \\todo Support camera hotplug.\n>>>>> -     */\n>>>>> -    unsigned int index = 0;\n>>>>> -    for (auto &cam : cameraManager_->cameras()) {\n>>>>> -        CameraDevice *camera = new CameraDevice(index, cam);\n>>>>> -        ret = camera->initialize();\n>>>>> -        if (ret)\n>>>>> -            continue;\n>>>>> -\n>>>>> -        cameras_.emplace_back(camera);\n>>>>> -        ++index;\n>>>>> -    }\n>>>>> -\n>>>>>       return 0;\n>>>>>   }\n>>>>>     CameraDevice *CameraHalManager::open(unsigned int id,\n>>>>>                        const hw_module_t *hardwareModule)\n>>>>>   {\n>>>>> -    if (id >= numCameras()) {\n>>>>> +    MutexLocker locker(mutex_);\n>>>>> +\n>>>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>>>> +                 [id](std::shared_ptr &cam) {\n>>>>> +                    return cam->id() == id;\n>>>>> +                });\n>>>>> +    if (iter == cameras_.end()) {\n>>>> I think you should break this (and the similar ones below) into private\n>>>> helpers instead if implementing the logic in-place\n>>>>\n>>>>      CameraHalManager::cameraFromAndroidId(..);\n>>>>      CameraHalManager::androidIdFromCamera(..);\n>>>>>           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>>>>>           return nullptr;\n>>>>>       }\n>>>>>   -    CameraDevice *camera = cameras_[id].get();\n>>>>> +    CameraDevice *camera = iter->get();\n>>>>> +\n>>>>>       if (camera->open(hardwareModule))\n>>>>>           return nullptr;\n>>>>>   @@ -93,9 +92,91 @@ CameraDevice\n>>>>> *CameraHalManager::open(unsigned int id,\n>>>>>       return camera;\n>>>>>   }\n>>>>>   +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n>>>>> +{\n>>>>> +    unsigned int id;\n>>>>> +    bool isCameraExternal = false;\n>>>>> +    bool isCameraNew = false;\n>>>>> +\n>>>>> +    MutexLocker locker(mutex_);\n>>>>> +\n>>>>> +    /* Each camera is assigned a unique integer id when it is\n>>>>> seen for the\n>>>>> +     * first time. If the camera has been seen before, the id\n>>>>> is reused and\n>>>>> +     * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT\n>>>>> subsequently.\n>>>>> +     *\n>>>>> +     * ID starts from '0' for internal cameras and '1000' for\n>>>>> external cameras.\n>>>>> +     */\n>>>>> +    auto iter = camerasMap_.find(cam->id());\n>>>>> +    if (iter != camerasMap_.end()) {\n>>>>> +        id = iter->second;\n>>>>> +    } else {\n>>>>> +        isCameraNew = true;\n>>>>> +\n>>>>> +        /*\n>>>>> +         *  Now check if this is an external camera and assign\n>>>>> +         *  its id accordingly.\n>>>>> +         */\n>>>>> +        const ControlList &properties = cam->properties();\n>>>>> +        if (properties.contains(properties::Location) &&\n>>>>> +            properties.get(properties::Location) &\n>>>>> +            properties::CameraLocationExternal) {\n>>>>> +            isCameraExternal = true;\n>>>>> +            id = externalCameraCounter_;\n>>>>> +        } else {\n>>>>> +            id = cameraCounter_;\n>>>>> +        }\n>>>> As I understand it the information the HAL wants is whether or not the\n>>>> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n>>> Again, the spec is not clear on the hot-plugged / removable camera IDs\n>>> front.\n>>> It is though clear on the internal cameras IDs - should start from 0,\n>>> hence,\n>>> we thought 1000 is a good number for assigning IDs for external cameras.\n>>>> If so I wonder if using the camera location as a judgment of the camera\n>>>> is hot-plugged or not is the way to go here? Imagine a device where the\n>>>> camera is permanently attached (not hot-unpluggable) but not fixated in\n>>>> a location. I'm thinking cameras mounted at the end of instruments\n>>>> (medical instruments, hand held tools) or robotics (mounted at the arm\n>>>> of a welding robot). I would imagine those cameras would be marked as\n>>>> located externally but they would not really be hot-pluggable.\n>>> Yes, I get your point. Their location is external, but from the\n>>> point-of-view\n>>> of HAL, they are still needed to be treated as \"internal\" cameras, no?\n>>> I guess, in HAL, internal cameras are the ones, which cannot be detached\n>>> from the device, whereas external cameras are the one, which can,\n>>> (Just a guess, I might be entirely wrong)\n>>>> I understand we have no other way to report or detect this at the moment\n>>>> and I'm not pushing hard for this to be solved as part of this series if\n>>>> it's not easy. But I think a bigger comment here is needed explaining\n>>>> that the HAL wants to know if a camera is hot-pluggable or not and does\n>>>> not really care if it's located internally or externally. I also think a\n>>>> \\todo should be added so it's not forgotten.\n>>>>\n>>>>> +    }\n>>>>> +\n>>>>> +    /*\n>>>>> +     * For each Camera registered in the system, a CameraDevice\n>>>>> +     * gets created here to wraps a libcamera Camera instance.\n>>>>> +     */\n>>>>> +    std::shared_ptr camera =\n>>>>> CameraDevice::create(id, std::move(cam));\n>>>>> +    int ret = camera->initialize();\n>>>>> +    if (ret) {\n>>>>> +        LOG(HAL, Error) << \"Failed to initialize camera: \" <<\n>>>>> cam->id();\n>>>>> +        return;\n>>>>> +    }\n>>>>> +\n>>>>> +    if (isCameraNew) {\n>>>>> +        camerasMap_.emplace(cam->id(), id);\n>>>>> +\n>>>>> +        if (isCameraExternal)\n>>>>> +            externalCameraCounter_++;\n>>>>> +        else\n>>>>> +            cameraCounter_++;\n>>>>> +    }\n>>>>> +\n>>>>> +    cameras_.emplace_back(std::move(camera));\n>>>>> +\n>>>>> +    if (callbacks_)\n>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n>>>>> +                            CAMERA_DEVICE_STATUS_PRESENT);\n>>>>> +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n>>>>> +}\n>>>>> +\n>>>>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n>>>>> +{\n>>>>> +    MutexLocker locker(mutex_);\n>>>>> +\n>>>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>>>> +                 [cam](std::shared_ptr &camera) {\n>>>>> +                    return cam.get() == camera->getCamera();\n>>>>> +                });\n>>>>> +    if (iter == cameras_.end())\n>>>>> +        return;\n>>>>> +\n>>>>> +    unsigned int id = (*iter)->id();\n>>>>> +    callbacks_->camera_device_status_change(callbacks_, id,\n>>>>> +                        CAMERA_DEVICE_STATUS_NOT_PRESENT);\n>>>>> +    cameras_.erase(iter);\n>>>>> +\n>>>>> +    LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed\n>>>>> successfully.\";\n>>>>> +}\n>>>>> +\n>>>>>   unsigned int CameraHalManager::numCameras() const\n>>>>>   {\n>>>>> -    return cameraManager_->cameras().size();\n>>>>> +    return cameraCounter_;\n>>>>>   }\n>>>>>     int CameraHalManager::getCameraInfo(unsigned int id, struct\n>>>>> camera_info *info)\n>>>>> @@ -103,12 +184,18 @@ int\n>>>>> CameraHalManager::getCameraInfo(unsigned int id, struct\n>>>>> camera_info *info)\n>>>>>       if (!info)\n>>>>>           return -EINVAL;\n>>>>>   -    if (id >= numCameras()) {\n>>>>> +    MutexLocker locker(mutex_);\n>>>>> +\n>>>>> +    auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n>>>>> +                 [id](std::shared_ptr &cam) {\n>>>>> +                    return cam->id() == id;\n>>>>> +                });\n>>>>> +    if (iter == cameras_.end()) {\n>>>>>           LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n>>>>>           return -EINVAL;\n>>>>>       }\n>>>>>   -    CameraDevice *camera = cameras_[id].get();\n>>>>> +    CameraDevice *camera = iter->get();\n>>>>>         info->facing = camera->facing();\n>>>>>       info->orientation = camera->orientation();\n>>>>> @@ -124,4 +211,26 @@ int\n>>>>> CameraHalManager::getCameraInfo(unsigned int id, struct\n>>>>> camera_info *info)\n>>>>>   void CameraHalManager::setCallbacks(const\n>>>>> camera_module_callbacks_t *callbacks)\n>>>>>   {\n>>>>>       callbacks_ = callbacks;\n>>>>> +\n>>>>> +    MutexLocker locker(mutex_);\n>>>>> +    /*\n>>>>> +     * Few cameras might have been hotplugged before setting\n>>>>> callbacks_ here.\n>>>>> +     * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them\n>>>>> explicitly.\n>>>>> +     * This hold only for external cameras, as internal cameras\n>>>>> are assumed to\n>>>>> +     * be present at module load time, by the framework.\n>>>>> +     */\n>>>>> +    for (auto &cam : cameraManager_->cameras()) {\n>>>>> +        auto iter = camerasMap_.find(cam->id());\n>>>>> +        if (iter == camerasMap_.end())\n>>>>> +            continue;\n>>>>> +\n>>>>> +        unsigned int id = iter->second;\n>>>>> +        const ControlList &properties = cam->properties();\n>>>>> +        if (properties.contains(properties::Location) &&\n>>>>> +            properties.get(properties::Location) &\n>>>>> +            properties::CameraLocationExternal) {\n>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n>>>>> +        }\n>>>>> +    }\n>>>>>   }\n>>>>> diff --git a/src/android/camera_hal_manager.h\n>>>>> b/src/android/camera_hal_manager.h\n>>>>> index a582f04..7c481d4 100644\n>>>>> --- a/src/android/camera_hal_manager.h\n>>>>> +++ b/src/android/camera_hal_manager.h\n>>>>> @@ -7,6 +7,8 @@\n>>>>>   #ifndef __ANDROID_CAMERA_MANAGER_H__\n>>>>>   #define __ANDROID_CAMERA_MANAGER_H__\n>>>>>   +#include \n>>>>> +#include \n>>>>>   #include \n>>>>>   #include \n>>>>>   @@ -18,6 +20,9 @@\n>>>>>     class CameraDevice;\n>>>>>   +using Mutex = std::mutex;\n>>>>> +using MutexLocker = std::unique_lock;\n>>>>> +\n>>>>>   class CameraHalManager\n>>>>>   {\n>>>>>   public:\n>>>>> @@ -33,10 +38,18 @@ public:\n>>>>>       void setCallbacks(const camera_module_callbacks_t *callbacks);\n>>>>>     private:\n>>>>> +    void cameraAdded(std::shared_ptr cam);\n>>>>> +    void cameraRemoved(std::shared_ptr cam);\n>>>>> +\n>>>>>       libcamera::CameraManager *cameraManager_;\n>>>>>         const camera_module_callbacks_t *callbacks_;\n>>>>> -    std::vector> cameras_;\n>>>>> +    std::vector> cameras_;\n>>>>> +    std::map camerasMap_;\n>>>> If each hot-plugged camera where treated as a new camera cameras_ and\n>>>> camerasMap_ could be merged to a\n>>>>\n>>>>      std::map> cameras_;\n>>>>\n>>>> Which would eliminate the possibility of them going out-of-sync.\n>>> Great point. I hope it's not too cubersome to iterate over all the\n>>> entires,\n>>> to find the exact CameraDevice we want for various plug events handler.\n>>> I will look into it.\n>> I found this to be a bit cubersome to implement. The foundation issue here\n>> is:\n>> we do need to cache the android HAL's ID<->libcamera::Camera::id mapping\n>> to identify if we are replugging an already-seen camera (I am assuming that\n>> libcamera::Camera::id is a unique ID generated by libcamera for each camera\n>> and is constant whenever the camera is replugged).\n>>\n>> If I just maintain a single map std::map> std::shared_ptr>\n>> the CameraDevice will be deleted on hot-unplug. Then I do not have a\n>> comparator to\n>> compare against, to know if the ID key should be reused or not, when hotplug\n>> event\n>> handlers.\n> As I stated above, having a single map is only a good simplification\n> _if_ we treat each camera as a new camera. That is that we do not cache\n> the cameras in order to reuse the numerical IDs.\nOk. I think I was confused as we do treat each camera as a new camera here\n(context: we create new pointers, initialize them *again* etc.)  and that we\ndon't cache the `CameraDevice`s at all, apart from their `id` part in \nthe map.\nAnyway, the confusion is now cleared up.\n> If we need to cache the ID information to fulfill some requirement of\n> the HAL design then my suggestion of a single map is unsuitable, as you\n> point.\n>\n>>>>> +    Mutex mutex_;\n>>>>> +\n>>>>> +    unsigned int externalCameraCounter_;\n>>>>> +    unsigned int cameraCounter_;\n>>>>>   };\n>>>>>     #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>>>>> -- \n>>>>> 2.26.2\n>>>>>\n>>>>> _______________________________________________\n>>>>> libcamera-devel mailing list\n>>>>> libcamera-devel@lists.libcamera.org\n>>>>> https://lists.libcamera.org/listinfo/libcamera-devel\n>>> [1] : https://lists.libcamera.org/pipermail/libcamera-devel/2020-August/011857.html\n>>>\n>>> _______________________________________________\n>>> libcamera-devel mailing list\n>>> libcamera-devel@lists.libcamera.org\n>>> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 980A2BD87D\n\tfor ;\n\tFri, 14 Aug 2020 11:53:10 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 19AF96038B;\n\tFri, 14 Aug 2020 13:53:10 +0200 (CEST)","from o1.f.az.sendgrid.net (o1.f.az.sendgrid.net [208.117.55.132])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B34A6038B\n\tfor ;\n\tFri, 14 Aug 2020 13:53:08 +0200 (CEST)","by filterdrecv-p3iad2-5c98798b7-85j65 with SMTP id\n\tfilterdrecv-p3iad2-5c98798b7-85j65-19-5F367B22-89\n\t2020-08-14 11:53:07.097440433 +0000 UTC m=+69818.766997595","from mail.uajain.com (unknown)\n\tby ismtpd0007p1hnd1.sendgrid.net (SG) with ESMTP\n\tid a2o2YK5WTqqRZAX12_P-xA Fri, 14 Aug 2020 11:53:06.588 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=uajain.com header.i=@uajain.com\n\theader.b=\"QgWuI/gZ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=uajain.com;\n\th=subject:references:from:mime-version:in-reply-to:to:cc:content-type:\n\tcontent-transfer-encoding;\n\ts=s1; bh=0W9NEogIL8QzCBc/iFZL/tQzR1w3yK6RqVnv3Vwcsbk=;\n\tb=QgWuI/gZv91GO9wlKzIxPFa00ihlfpMZTUK4BX0q5/OnB//fHGZaHyAj0Enp0S3Y4H5r\n\tkQN9gjTYayFFnf5O5G9bAR7DdNmzlqL8n3v0shajLBIOoUatEQ1zKgV1anO5+aztVo4JL+\n\twPo04uhchZQlTmdZYr9ih8BnsmHzdGLew=","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<07ba3d57-05c6-c21d-8250-069281064999@uajain.com>\n\t<10da26da-55f9-0c99-e763-74cb4270ef2f@uajain.com>\n\t<20200814092555.GA1995759@oden.dyn.berto.se>","From":"Umang Jain ","Message-ID":"<18d184eb-e8a1-aa96-7e7b-59c452782492@uajain.com>","Date":"Fri, 14 Aug 2020 11:53:07 +0000 (UTC)","Mime-Version":"1.0","In-Reply-To":"<20200814092555.GA1995759@oden.dyn.berto.se>","X-SG-EID":"1Q40EQ7YGir8a9gjSIAdTjhngY657NMk9ckeo4dbHZDiOpywc/L3L9rFqlwE4KPc1X6pmd7do7wYP56agy+aLkxi4ZlIVwiykAYGvn9/uioBgA4S++LfjFyamtrMVdtoVgU3O/b5MYQDjmsFhqQ4hPRziHGBWgMf7SGjVzJt2bNEaZ32hIEGuVRCCJA4yEDLwtl9Rs28Gi8gk/dO5IUji4fVXgpTWdcfl16myRp5thN2RMDSC3LH7SM5fuPjT4X5pmB2GjBYNZAsGpbrtJtwxQ==","To":"Niklas =?iso-8859-1?q?S=F6derlund?= ","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Transfer-Encoding":"quoted-printable","Content-Type":"text/plain; charset=\"iso-8859-1\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12032,"web_url":"https://patchwork.libcamera.org/comment/12032/","msgid":"<20200818083615.GA6029@pendragon.ideasonboard.com>","date":"2020-08-18T08:36:15","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Shik,\n\nWould it be possible for you to have a look at the question below ?\n\nOn Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> Hi Niklas,\n> \n> (CC'ing Shik)\n> \n> On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > know about the hotplug events and change the status of cameras being\n> > > being hotplugged or unplugged via camera_device_status_change().\n> > > \n> > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > a new id is assigned to it. If the camera has been seen before by the\n> > > manager, it's old id is reused. IDs for internal cameras start with\n> > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > current implementation, we assume all UVC cameras are external cameras.\n> > \n> > I wonder if keeping the cache of previously seen cameras or if we should \n> > not just treat any hot-plugged camera as a new one? I can't think of any \n> > benefit of preserving the same numerical ID between two plug events, \n> > while I can think of quiet a few cons.\n> > \n> > - It's confusing when debugging as un-plugging and then replugging the \n> > same camera will result in logs where the numerical ID is the same for \n> > both. This may even result in things working by \"chance\" is it reuses \n> > an already known numerical ID.\n> > \n> > - Looking at the code plugging a UVC camera in a different USB port will \n> > generate a different Camera::id() result and then be treated as a new \n> > camera vs plugging it in the same port and it then being treatad as a \n> > new camera.\n> > \n> > - The logic in this patch is more complex due to it both having to deal \n> > with known and new cameras.\n> > \n> > What is the benefit of this cache that I'm missing?\n> \n> It's not clear whether Android requires it. It requires different\n> cameras to have different IDs, but I couldn't find a mention of how\n> identical cameras should be treated. The Chrome OS UVC HAL caches the\n> ID, I don't know if it's mandatory though.\n> \n> Shik, as you've worked on the UVC HAL, would you happen to know if there\n> are requirements to keep the same ID when a camera is unplugged and\n> replugged ?\n> \n> > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > pointers to CameraDevice. This is done in order to introduce reference\n> > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > \n> > > Signed-off-by: Umang Jain \n> > > ---\n> > > src/android/camera_device.cpp | 15 +++\n> > > src/android/camera_device.h | 8 +-\n> > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > src/android/camera_hal_manager.h | 15 ++-\n> > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > \n> > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > index d918350..a79bb69 100644\n> > > --- a/src/android/camera_device.cpp\n> > > +++ b/src/android/camera_device.cpp\n> > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > \t\tdelete it.second;\n> > > }\n> > > \n> > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > +\t\t\t\t\t\t const std::shared_ptr &cam)\n> > > +{\n> > > +\tstruct Deleter : std::default_delete {\n> > > +\t\tvoid operator()(CameraDevice *camera)\n> > > +\t\t{\n> > > +\t\t\tdelete camera;\n> > > +\t\t}\n> > > +\t};\n> > > +\n> > > +\tCameraDevice *camera = new CameraDevice(id, cam);\n> > > +\n> > > +\treturn std::shared_ptr(camera, Deleter());\n> > > +}\n> > \n> > As Kieran points out I think this should be added in a separate as this \n> > one is quiet large and therefore hard to review.\n> > \n> > > +\n> > > /*\n> > > * Initialize the camera static information.\n> > > * This method is called before the camera device is opened.\n> > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > index 7be9e11..7f9e010 100644\n> > > --- a/src/android/camera_device.h\n> > > +++ b/src/android/camera_device.h\n> > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > class CameraDevice : protected libcamera::Loggable\n> > > {\n> > > public:\n> > > -\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > -\t~CameraDevice();\n> > > +\tstatic std::shared_ptr create(unsigned int id,\n> > > +\t\t\t\t\t\t const std::shared_ptr &cam);\n> > > \n> > > \tint initialize();\n> > > \n> > > @@ -57,6 +57,7 @@ public:\n> > > \n> > > \tunsigned int id() const { return id_; }\n> > > \tcamera3_device_t *camera3Device() { return &camera3Device_; }\n> > > +\tconst libcamera::Camera *getCamera() { return camera_.get(); };\n> > \n> > Same here this can be done in a separate commit. Also I think this could \n> > be named camera() instead of getCamera()\n> > \n> > > \n> > > \tint facing() const { return facing_; }\n> > > \tint orientation() const { return orientation_; }\n> > > @@ -72,6 +73,9 @@ protected:\n> > > \tstd::string logPrefix() const override;\n> > > \n> > > private:\n> > > +\tCameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > +\t~CameraDevice();\n> > > +\n> > > \tstruct Camera3RequestDescriptor {\n> > > \t\tCamera3RequestDescriptor(unsigned int frameNumber,\n> > > \t\t\t\t\t unsigned int numBuffers);\n> > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > index 3d6d2b4..fdde2c0 100644\n> > > --- a/src/android/camera_hal_manager.cpp\n> > > +++ b/src/android/camera_hal_manager.cpp\n> > > @@ -8,6 +8,7 @@\n> > > #include \"camera_hal_manager.h\"\n> > > \n> > > #include \n> > > +#include \n> > > \n> > > #include \"libcamera/internal/log.h\"\n> > > \n> > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > CameraHalManager::~CameraHalManager()\n> > > {\n> > > \tcameras_.clear();\n> > > +\tcamerasMap_.clear();\n> > > \n> > > \tif (cameraManager_) {\n> > > \t\tcameraManager_->stop();\n> > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > {\n> > > \tcameraManager_ = new CameraManager();\n> > > \n> > > +\t/* Support camera hotplug. */\n> > > +\tcameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > +\tcameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > +\n> > > +\tcameraCounter_ = 0;\n> > > +\texternalCameraCounter_ = 1000;\n> > > +\n> > > \tint ret = cameraManager_->start();\n> > > \tif (ret) {\n> > > \t\tLOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > \t\treturn ret;\n> > > \t}\n> > > \n> > > -\t/*\n> > > -\t * For each Camera registered in the system, a CameraDevice\n> > > -\t * gets created here to wraps a libcamera Camera instance.\n> > > -\t *\n> > > -\t * \\todo Support camera hotplug.\n> > > -\t */\n> > > -\tunsigned int index = 0;\n> > > -\tfor (auto &cam : cameraManager_->cameras()) {\n> > > -\t\tCameraDevice *camera = new CameraDevice(index, cam);\n> > > -\t\tret = camera->initialize();\n> > > -\t\tif (ret)\n> > > -\t\t\tcontinue;\n> > > -\n> > > -\t\tcameras_.emplace_back(camera);\n> > > -\t\t++index;\n> > > -\t}\n> > > -\n> > > \treturn 0;\n> > > }\n> > > \n> > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > \t\t\t\t const hw_module_t *hardwareModule)\n> > > {\n> > > -\tif (id >= numCameras()) {\n> > > +\tMutexLocker locker(mutex_);\n> > > +\n> > > +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > +\t\t\t\t [id](std::shared_ptr &cam) {\n> > > +\t\t\t\t\treturn cam->id() == id;\n> > > +\t\t\t\t});\n> > > +\tif (iter == cameras_.end()) {\n> > \n> > I think you should break this (and the similar ones below) into private \n> > helpers instead if implementing the logic in-place\n> > \n> > CameraHalManager::cameraFromAndroidId(..);\n> > CameraHalManager::androidIdFromCamera(..);\n> > \n> > > \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > \t\treturn nullptr;\n> > > \t}\n> > > \n> > > -\tCameraDevice *camera = cameras_[id].get();\n> > > +\tCameraDevice *camera = iter->get();\n> > > +\n> > > \tif (camera->open(hardwareModule))\n> > > \t\treturn nullptr;\n> > > \n> > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > \treturn camera;\n> > > }\n> > > \n> > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > +{\n> > > +\tunsigned int id;\n> > > +\tbool isCameraExternal = false;\n> > > +\tbool isCameraNew = false;\n> > > +\n> > > +\tMutexLocker locker(mutex_);\n> > > +\n> > > +\t/* Each camera is assigned a unique integer id when it is seen for the\n> > > +\t * first time. If the camera has been seen before, the id is reused and\n> > > +\t * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > +\t *\n> > > +\t * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > +\t */\n> > > +\tauto iter = camerasMap_.find(cam->id());\n> > > +\tif (iter != camerasMap_.end()) {\n> > > +\t\tid = iter->second;\n> > > +\t} else {\n> > > +\t\tisCameraNew = true;\n> > > +\n> > > +\t\t/*\n> > > +\t\t * Now check if this is an external camera and assign\n> > > +\t\t * its id accordingly.\n> > > +\t\t */\n> > > +\t\tconst ControlList &properties = cam->properties();\n> > > +\t\tif (properties.contains(properties::Location) &&\n> > > +\t\t properties.get(properties::Location) &\n> > > +\t\t properties::CameraLocationExternal) {\n> > > +\t\t\tisCameraExternal = true;\n> > > +\t\t\tid = externalCameraCounter_;\n> > > +\t\t} else {\n> > > +\t\t\tid = cameraCounter_;\n> > > +\t\t}\n> > \n> > As I understand it the information the HAL wants is whether or not the \n> > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > \n> > If so I wonder if using the camera location as a judgment of the camera \n> > is hot-plugged or not is the way to go here? Imagine a device where the \n> > camera is permanently attached (not hot-unpluggable) but not fixated in \n> > a location. I'm thinking cameras mounted at the end of instruments \n> > (medical instruments, hand held tools) or robotics (mounted at the arm \n> > of a welding robot). I would imagine those cameras would be marked as \n> > located externally but they would not really be hot-pluggable.\n> > \n> > I understand we have no other way to report or detect this at the moment \n> > and I'm not pushing hard for this to be solved as part of this series if \n> > it's not easy. But I think a bigger comment here is needed explaining \n> > that the HAL wants to know if a camera is hot-pluggable or not and does \n> > not really care if it's located internally or externally. I also think a \n> > \\todo should be added so it's not forgotten.\n> > \n> > > +\t}\n> > > +\n> > > +\t/*\n> > > +\t * For each Camera registered in the system, a CameraDevice\n> > > +\t * gets created here to wraps a libcamera Camera instance.\n> > > +\t */\n> > > +\tstd::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > +\tint ret = camera->initialize();\n> > > +\tif (ret) {\n> > > +\t\tLOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > +\t\treturn;\n> > > +\t}\n> > > +\n> > > +\tif (isCameraNew) {\n> > > +\t\tcamerasMap_.emplace(cam->id(), id);\n> > > +\n> > > +\t\tif (isCameraExternal)\n> > > +\t\t\texternalCameraCounter_++;\n> > > +\t\telse\n> > > +\t\t\tcameraCounter_++;\n> > > +\t}\n> > > +\n> > > +\tcameras_.emplace_back(std::move(camera));\n> > > +\n> > > +\tif (callbacks_)\n> > > +\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> > > +\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> > > +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > +}\n> > > +\n> > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > +{\n> > > +\tMutexLocker locker(mutex_);\n> > > +\n> > > +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > +\t\t\t\t [cam](std::shared_ptr &camera) {\n> > > +\t\t\t\t\treturn cam.get() == camera->getCamera();\n> > > +\t\t\t\t});\n> > > +\tif (iter == cameras_.end())\n> > > +\t\treturn;\n> > > +\n> > > +\tunsigned int id = (*iter)->id();\n> > > +\tcallbacks_->camera_device_status_change(callbacks_, id,\n> > > +\t\t\t\t\t\tCAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > +\tcameras_.erase(iter);\n> > > +\n> > > +\tLOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > +}\n> > > +\n> > > unsigned int CameraHalManager::numCameras() const\n> > > {\n> > > -\treturn cameraManager_->cameras().size();\n> > > +\treturn cameraCounter_;\n> > > }\n> > > \n> > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > \tif (!info)\n> > > \t\treturn -EINVAL;\n> > > \n> > > -\tif (id >= numCameras()) {\n> > > +\tMutexLocker locker(mutex_);\n> > > +\n> > > +\tauto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > +\t\t\t\t [id](std::shared_ptr &cam) {\n> > > +\t\t\t\t\treturn cam->id() == id;\n> > > +\t\t\t\t});\n> > > +\tif (iter == cameras_.end()) {\n> > > \t\tLOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > \t\treturn -EINVAL;\n> > > \t}\n> > > \n> > > -\tCameraDevice *camera = cameras_[id].get();\n> > > +\tCameraDevice *camera = iter->get();\n> > > \n> > > \tinfo->facing = camera->facing();\n> > > \tinfo->orientation = camera->orientation();\n> > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > {\n> > > \tcallbacks_ = callbacks;\n> > > +\n> > > +\tMutexLocker locker(mutex_);\n> > > +\t/*\n> > > +\t * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > +\t * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > +\t * This hold only for external cameras, as internal cameras are assumed to\n> > > +\t * be present at module load time, by the framework.\n> > > +\t */\n> > > +\tfor (auto &cam : cameraManager_->cameras()) {\n> > > +\t\tauto iter = camerasMap_.find(cam->id());\n> > > +\t\tif (iter == camerasMap_.end())\n> > > +\t\t\tcontinue;\n> > > +\n> > > +\t\tunsigned int id = iter->second;\n> > > +\t\tconst ControlList &properties = cam->properties();\n> > > +\t\tif (properties.contains(properties::Location) &&\n> > > +\t\t properties.get(properties::Location) &\n> > > +\t\t properties::CameraLocationExternal) {\n> > > +\t\t\tcallbacks_->camera_device_status_change(callbacks_, id,\n> > > +\t\t\t\t\t\t\t\tCAMERA_DEVICE_STATUS_PRESENT);\n> > > +\t\t}\n> > > +\t}\n> > > }\n> > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > index a582f04..7c481d4 100644\n> > > --- a/src/android/camera_hal_manager.h\n> > > +++ b/src/android/camera_hal_manager.h\n> > > @@ -7,6 +7,8 @@\n> > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > \n> > > +#include \n> > > +#include \n> > > #include \n> > > #include \n> > > \n> > > @@ -18,6 +20,9 @@\n> > > \n> > > class CameraDevice;\n> > > \n> > > +using Mutex = std::mutex;\n> > > +using MutexLocker = std::unique_lock;\n> > > +\n> > > class CameraHalManager\n> > > {\n> > > public:\n> > > @@ -33,10 +38,18 @@ public:\n> > > \tvoid setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > \n> > > private:\n> > > +\tvoid cameraAdded(std::shared_ptr cam);\n> > > +\tvoid cameraRemoved(std::shared_ptr cam);\n> > > +\n> > > \tlibcamera::CameraManager *cameraManager_;\n> > > \n> > > \tconst camera_module_callbacks_t *callbacks_;\n> > > -\tstd::vector> cameras_;\n> > > +\tstd::vector> cameras_;\n> > > +\tstd::map camerasMap_;\n> > \n> > If each hot-plugged camera where treated as a new camera cameras_ and \n> > camerasMap_ could be merged to a\n> > \n> > std::map> cameras_;\n> > \n> > Which would eliminate the possibility of them going out-of-sync.\n> > \n> > > +\tMutex mutex_;\n> > > +\n> > > +\tunsigned int externalCameraCounter_;\n> > > +\tunsigned int cameraCounter_;\n> > > };\n> > > \n> > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EF9DFBD87C\n\tfor ;\n\tTue, 18 Aug 2020 08:36:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 760C361930;\n\tTue, 18 Aug 2020 10:36:35 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B3B7360382\n\tfor ;\n\tTue, 18 Aug 2020 10:36:33 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2DBA7279;\n\tTue, 18 Aug 2020 10:36:33 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"JiM9KjaZ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597739793;\n\tbh=+Z28Ux2NtZMNCiw2RPooHF9PwEgQN/cDLeJR9e12Cxk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=JiM9KjaZXDDPJWff5b4CiPpqZha9/8DKjyTGaFZSAFSES5WrRLNyoaeAdDADc8d1B\n\tWKWhwTqzVrq/OEIgA0Dmd4BDxCSjdtfw4AorF1xdNnrffG2oL8ZfLPOBjP2aHaUFj8\n\tCsNAsu/8/q6pkXkioImipkGTl27iRPC+Ab57bhdI=","Date":"Tue, 18 Aug 2020 11:36:15 +0300","From":"Laurent Pinchart ","To":"Shik Chen ","Message-ID":"<20200818083615.GA6029@pendragon.ideasonboard.com>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200813120529.GE6057@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12049,"web_url":"https://patchwork.libcamera.org/comment/12049/","msgid":"","date":"2020-08-19T15:41:45","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":8,"url":"https://patchwork.libcamera.org/api/people/8/","name":"Shik Chen","email":"shik@google.com"},"content":"Hi Laurent,\n\nOn Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart\n wrote:\n>\n> Hi Shik,\n>\n> Would it be possible for you to have a look at the question below ?\n>\n> On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > Hi Niklas,\n> >\n> > (CC'ing Shik)\n> >\n> > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > know about the hotplug events and change the status of cameras being\n> > > > being hotplugged or unplugged via camera_device_status_change().\n> > > >\n> > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > current implementation, we assume all UVC cameras are external cameras.\n> > >\n> > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > benefit of preserving the same numerical ID between two plug events,\n> > > while I can think of quiet a few cons.\n> > >\n> > > - It's confusing when debugging as un-plugging and then replugging the\n> > > same camera will result in logs where the numerical ID is the same for\n> > > both. This may even result in things working by \"chance\" is it reuses\n> > > an already known numerical ID.\n> > >\n> > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > generate a different Camera::id() result and then be treated as a new\n> > > camera vs plugging it in the same port and it then being treatad as a\n> > > new camera.\n> > >\n> > > - The logic in this patch is more complex due to it both having to deal\n> > > with known and new cameras.\n> > >\n> > > What is the benefit of this cache that I'm missing?\n> >\n> > It's not clear whether Android requires it. It requires different\n> > cameras to have different IDs, but I couldn't find a mention of how\n> > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > ID, I don't know if it's mandatory though.\n> >\n> > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > are requirements to keep the same ID when a camera is unplugged and\n> > replugged ?\n\nIt's for persistency in settings UI. Chrome uses the camera id as the key of\nthe default camera in chrome://settings/content/camera. Reuse the same id for\nthe same camera so the default camera would be changed back after replug.\n\nIt's introduced in this CL: https://crrev.com/c/1608985\n\n> >\n> > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > >\n> > > > Signed-off-by: Umang Jain \n> > > > ---\n> > > > src/android/camera_device.cpp | 15 +++\n> > > > src/android/camera_device.h | 8 +-\n> > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > >\n> > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > index d918350..a79bb69 100644\n> > > > --- a/src/android/camera_device.cpp\n> > > > +++ b/src/android/camera_device.cpp\n> > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > delete it.second;\n> > > > }\n> > > >\n> > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > + const std::shared_ptr &cam)\n> > > > +{\n> > > > + struct Deleter : std::default_delete {\n> > > > + void operator()(CameraDevice *camera)\n> > > > + {\n> > > > + delete camera;\n> > > > + }\n> > > > + };\n> > > > +\n> > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > +\n> > > > + return std::shared_ptr(camera, Deleter());\n> > > > +}\n> > >\n> > > As Kieran points out I think this should be added in a separate as this\n> > > one is quiet large and therefore hard to review.\n> > >\n> > > > +\n> > > > /*\n> > > > * Initialize the camera static information.\n> > > > * This method is called before the camera device is opened.\n> > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > index 7be9e11..7f9e010 100644\n> > > > --- a/src/android/camera_device.h\n> > > > +++ b/src/android/camera_device.h\n> > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > class CameraDevice : protected libcamera::Loggable\n> > > > {\n> > > > public:\n> > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > - ~CameraDevice();\n> > > > + static std::shared_ptr create(unsigned int id,\n> > > > + const std::shared_ptr &cam);\n> > > >\n> > > > int initialize();\n> > > >\n> > > > @@ -57,6 +57,7 @@ public:\n> > > >\n> > > > unsigned int id() const { return id_; }\n> > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > >\n> > > Same here this can be done in a separate commit. Also I think this could\n> > > be named camera() instead of getCamera()\n> > >\n> > > >\n> > > > int facing() const { return facing_; }\n> > > > int orientation() const { return orientation_; }\n> > > > @@ -72,6 +73,9 @@ protected:\n> > > > std::string logPrefix() const override;\n> > > >\n> > > > private:\n> > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > + ~CameraDevice();\n> > > > +\n> > > > struct Camera3RequestDescriptor {\n> > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > unsigned int numBuffers);\n> > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > index 3d6d2b4..fdde2c0 100644\n> > > > --- a/src/android/camera_hal_manager.cpp\n> > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > @@ -8,6 +8,7 @@\n> > > > #include \"camera_hal_manager.h\"\n> > > >\n> > > > #include \n> > > > +#include \n> > > >\n> > > > #include \"libcamera/internal/log.h\"\n> > > >\n> > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > CameraHalManager::~CameraHalManager()\n> > > > {\n> > > > cameras_.clear();\n> > > > + camerasMap_.clear();\n> > > >\n> > > > if (cameraManager_) {\n> > > > cameraManager_->stop();\n> > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > {\n> > > > cameraManager_ = new CameraManager();\n> > > >\n> > > > + /* Support camera hotplug. */\n> > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > +\n> > > > + cameraCounter_ = 0;\n> > > > + externalCameraCounter_ = 1000;\n> > > > +\n> > > > int ret = cameraManager_->start();\n> > > > if (ret) {\n> > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > return ret;\n> > > > }\n> > > >\n> > > > - /*\n> > > > - * For each Camera registered in the system, a CameraDevice\n> > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > - *\n> > > > - * \\todo Support camera hotplug.\n> > > > - */\n> > > > - unsigned int index = 0;\n> > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > - ret = camera->initialize();\n> > > > - if (ret)\n> > > > - continue;\n> > > > -\n> > > > - cameras_.emplace_back(camera);\n> > > > - ++index;\n> > > > - }\n> > > > -\n> > > > return 0;\n> > > > }\n> > > >\n> > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > const hw_module_t *hardwareModule)\n> > > > {\n> > > > - if (id >= numCameras()) {\n> > > > + MutexLocker locker(mutex_);\n> > > > +\n> > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > + [id](std::shared_ptr &cam) {\n> > > > + return cam->id() == id;\n> > > > + });\n> > > > + if (iter == cameras_.end()) {\n> > >\n> > > I think you should break this (and the similar ones below) into private\n> > > helpers instead if implementing the logic in-place\n> > >\n> > > CameraHalManager::cameraFromAndroidId(..);\n> > > CameraHalManager::androidIdFromCamera(..);\n> > >\n> > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > return nullptr;\n> > > > }\n> > > >\n> > > > - CameraDevice *camera = cameras_[id].get();\n> > > > + CameraDevice *camera = iter->get();\n> > > > +\n> > > > if (camera->open(hardwareModule))\n> > > > return nullptr;\n> > > >\n> > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > return camera;\n> > > > }\n> > > >\n> > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > +{\n> > > > + unsigned int id;\n> > > > + bool isCameraExternal = false;\n> > > > + bool isCameraNew = false;\n> > > > +\n> > > > + MutexLocker locker(mutex_);\n> > > > +\n> > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > + *\n> > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > + */\n> > > > + auto iter = camerasMap_.find(cam->id());\n> > > > + if (iter != camerasMap_.end()) {\n> > > > + id = iter->second;\n> > > > + } else {\n> > > > + isCameraNew = true;\n> > > > +\n> > > > + /*\n> > > > + * Now check if this is an external camera and assign\n> > > > + * its id accordingly.\n> > > > + */\n> > > > + const ControlList &properties = cam->properties();\n> > > > + if (properties.contains(properties::Location) &&\n> > > > + properties.get(properties::Location) &\n> > > > + properties::CameraLocationExternal) {\n> > > > + isCameraExternal = true;\n> > > > + id = externalCameraCounter_;\n> > > > + } else {\n> > > > + id = cameraCounter_;\n> > > > + }\n> > >\n> > > As I understand it the information the HAL wants is whether or not the\n> > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > >\n> > > If so I wonder if using the camera location as a judgment of the camera\n> > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > a location. I'm thinking cameras mounted at the end of instruments\n> > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > of a welding robot). I would imagine those cameras would be marked as\n> > > located externally but they would not really be hot-pluggable.\n> > >\n> > > I understand we have no other way to report or detect this at the moment\n> > > and I'm not pushing hard for this to be solved as part of this series if\n> > > it's not easy. But I think a bigger comment here is needed explaining\n> > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > not really care if it's located internally or externally. I also think a\n> > > \\todo should be added so it's not forgotten.\n> > >\n> > > > + }\n> > > > +\n> > > > + /*\n> > > > + * For each Camera registered in the system, a CameraDevice\n> > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > + */\n> > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > + int ret = camera->initialize();\n> > > > + if (ret) {\n> > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > + return;\n> > > > + }\n> > > > +\n> > > > + if (isCameraNew) {\n> > > > + camerasMap_.emplace(cam->id(), id);\n> > > > +\n> > > > + if (isCameraExternal)\n> > > > + externalCameraCounter_++;\n> > > > + else\n> > > > + cameraCounter_++;\n> > > > + }\n> > > > +\n> > > > + cameras_.emplace_back(std::move(camera));\n> > > > +\n> > > > + if (callbacks_)\n> > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > +}\n> > > > +\n> > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > +{\n> > > > + MutexLocker locker(mutex_);\n> > > > +\n> > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > + [cam](std::shared_ptr &camera) {\n> > > > + return cam.get() == camera->getCamera();\n> > > > + });\n> > > > + if (iter == cameras_.end())\n> > > > + return;\n> > > > +\n> > > > + unsigned int id = (*iter)->id();\n> > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > + cameras_.erase(iter);\n> > > > +\n> > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > +}\n> > > > +\n> > > > unsigned int CameraHalManager::numCameras() const\n> > > > {\n> > > > - return cameraManager_->cameras().size();\n> > > > + return cameraCounter_;\n> > > > }\n> > > >\n> > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > if (!info)\n> > > > return -EINVAL;\n> > > >\n> > > > - if (id >= numCameras()) {\n> > > > + MutexLocker locker(mutex_);\n> > > > +\n> > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > + [id](std::shared_ptr &cam) {\n> > > > + return cam->id() == id;\n> > > > + });\n> > > > + if (iter == cameras_.end()) {\n> > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > return -EINVAL;\n> > > > }\n> > > >\n> > > > - CameraDevice *camera = cameras_[id].get();\n> > > > + CameraDevice *camera = iter->get();\n> > > >\n> > > > info->facing = camera->facing();\n> > > > info->orientation = camera->orientation();\n> > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > {\n> > > > callbacks_ = callbacks;\n> > > > +\n> > > > + MutexLocker locker(mutex_);\n> > > > + /*\n> > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > + * be present at module load time, by the framework.\n> > > > + */\n> > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > + auto iter = camerasMap_.find(cam->id());\n> > > > + if (iter == camerasMap_.end())\n> > > > + continue;\n> > > > +\n> > > > + unsigned int id = iter->second;\n> > > > + const ControlList &properties = cam->properties();\n> > > > + if (properties.contains(properties::Location) &&\n> > > > + properties.get(properties::Location) &\n> > > > + properties::CameraLocationExternal) {\n> > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > + }\n> > > > + }\n> > > > }\n> > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > index a582f04..7c481d4 100644\n> > > > --- a/src/android/camera_hal_manager.h\n> > > > +++ b/src/android/camera_hal_manager.h\n> > > > @@ -7,6 +7,8 @@\n> > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > >\n> > > > +#include \n> > > > +#include \n> > > > #include \n> > > > #include \n> > > >\n> > > > @@ -18,6 +20,9 @@\n> > > >\n> > > > class CameraDevice;\n> > > >\n> > > > +using Mutex = std::mutex;\n> > > > +using MutexLocker = std::unique_lock;\n> > > > +\n> > > > class CameraHalManager\n> > > > {\n> > > > public:\n> > > > @@ -33,10 +38,18 @@ public:\n> > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > >\n> > > > private:\n> > > > + void cameraAdded(std::shared_ptr cam);\n> > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > +\n> > > > libcamera::CameraManager *cameraManager_;\n> > > >\n> > > > const camera_module_callbacks_t *callbacks_;\n> > > > - std::vector> cameras_;\n> > > > + std::vector> cameras_;\n> > > > + std::map camerasMap_;\n> > >\n> > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > camerasMap_ could be merged to a\n> > >\n> > > std::map> cameras_;\n> > >\n> > > Which would eliminate the possibility of them going out-of-sync.\n> > >\n> > > > + Mutex mutex_;\n> > > > +\n> > > > + unsigned int externalCameraCounter_;\n> > > > + unsigned int cameraCounter_;\n> > > > };\n> > > >\n> > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n\n- shik","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F1F23BE173\n\tfor ;\n\tWed, 19 Aug 2020 15:42:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 815B861ED9;\n\tWed, 19 Aug 2020 17:42:06 +0200 (CEST)","from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com\n\t[IPv6:2607:f8b0:4864:20::82a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4208F60382\n\tfor ;\n\tWed, 19 Aug 2020 17:42:04 +0200 (CEST)","by mail-qt1-x82a.google.com with SMTP id o22so18067261qtt.13\n\tfor ;\n\tWed, 19 Aug 2020 08:42:04 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"aEFXN2XL\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=20161025; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=2V+QizGMhNxTXqNKsYtUuQwsfc+98NIl6FdUAWY77zY=;\n\tb=aEFXN2XLOzg2nPGP9wxNxNua93ViZGKfexRUKFKgE09evVGB+FSdLpNN1gj3FQQH/Q\n\tBy+2Pb2jLdU0dKALMbcQq2FAwwg071FTGH0SQ+gn1VYnKB7Splm+yFuVkQjA0k49WZ9V\n\tMVjVXIv7hoH+ZgRgo70DzlIMEBg1WJyiWzKlniBxOFHUEc3y3SlKv5BtaFTzxNC5mDRQ\n\tGE+Q243JoqXEyVknKEZcrN/53IhvyJs0CFsPNNbr4Q3dyDTOW/7uBIhM3yuvDd4kYJZ6\n\tqptsq94apHa3LLfgIM5KrEDaRPbKu05moViCessBx4fM36H7HvOLCZKQOMpdCT+iug2t\n\twM+Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=2V+QizGMhNxTXqNKsYtUuQwsfc+98NIl6FdUAWY77zY=;\n\tb=F0gB1JsFMtjeTdMjqH2BLZG9dwFf6bwCQnnitbat6jQQ0MtE0EiVeQTGg+v+01vQ3l\n\t5zAOeTW114fec6Epe8M29tkT2o9WDwYfUj073ZUz0gDVAZZx5GNKA20cWqmuXeMZ8jai\n\tdZM+7uZ3CpVH0Iq6uoY9AM8ok6qdc42ZwqVjyhiBfISSG7io0BgCXHzEXnGuxe4g7bjP\n\tPL4WEge/9hYszW7aOoJRUrg4awdtV1sMWL6EwFxuRojRga1yNylzOuJGVeBClpzvZ3E7\n\tylVBpcSJYInkQestUrtJV86IENsuXsHXvPBFsQ1lFC3r04b3J2vzVeOiz6OtJuaueF92\n\ty+FQ==","X-Gm-Message-State":"AOAM531oMMzynsBO1Ex1vShhpGzDciASVWSLZjEEdPjLxHqNaC8vHh8L\n\tTj0GKN2BWGNiaZ1KcMHQIik3DqlpVs88HUcNZB4PZA==","X-Google-Smtp-Source":"ABdhPJz7+kkBSq+EpPyPJwpzpL8VqMwXhIaMkTTBt2hyB7f3iHUqSfyR4an8sPGLVUuBTBmIO0oPJeSV7iZS1SfAbuE=","X-Received":"by 2002:aed:27c8:: with SMTP id\n\tm8mr23118605qtg.302.1597851722227; \n\tWed, 19 Aug 2020 08:42:02 -0700 (PDT)","MIME-Version":"1.0","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>","In-Reply-To":"<20200818083615.GA6029@pendragon.ideasonboard.com>","From":"Shik Chen ","Date":"Wed, 19 Aug 2020 23:41:45 +0800","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12050,"web_url":"https://patchwork.libcamera.org/comment/12050/","msgid":"<20200819155221.GP6049@pendragon.ideasonboard.com>","date":"2020-08-19T15:52:21","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Shik,\n\nOn Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> >\n> > Hi Shik,\n> >\n> > Would it be possible for you to have a look at the question below ?\n> >\n> > On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > > Hi Niklas,\n> > >\n> > > (CC'ing Shik)\n> > >\n> > > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > > know about the hotplug events and change the status of cameras being\n> > > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > >\n> > > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > > current implementation, we assume all UVC cameras are external cameras.\n> > > >\n> > > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > > benefit of preserving the same numerical ID between two plug events,\n> > > > while I can think of quiet a few cons.\n> > > >\n> > > > - It's confusing when debugging as un-plugging and then replugging the\n> > > > same camera will result in logs where the numerical ID is the same for\n> > > > both. This may even result in things working by \"chance\" is it reuses\n> > > > an already known numerical ID.\n> > > >\n> > > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > > generate a different Camera::id() result and then be treated as a new\n> > > > camera vs plugging it in the same port and it then being treatad as a\n> > > > new camera.\n> > > >\n> > > > - The logic in this patch is more complex due to it both having to deal\n> > > > with known and new cameras.\n> > > >\n> > > > What is the benefit of this cache that I'm missing?\n> > >\n> > > It's not clear whether Android requires it. It requires different\n> > > cameras to have different IDs, but I couldn't find a mention of how\n> > > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > > ID, I don't know if it's mandatory though.\n> > >\n> > > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > > are requirements to keep the same ID when a camera is unplugged and\n> > > replugged ?\n> \n> It's for persistency in settings UI. Chrome uses the camera id as the key of\n> the default camera in chrome://settings/content/camera. Reuse the same id for\n> the same camera so the default camera would be changed back after replug.\n> \n> It's introduced in this CL: https://crrev.com/c/1608985\n\nThank you for the information. We have a similar caching system, which\nhowever suffers from two limitations:\n\n- If a camera is unplugged, and a different camera with the same VID:PID\n is plugged into the same USB port, the new camera be given the same ID\n as the previous camera.\n\n- If a camera is unplugged and replugged in a different USB port, it\n will be given a different ID.\n\nIs that acceptable from a Chrome OS point of view ?\n\nWould you happen to know how Android uses camera IDs, and whether it\nrequires or could benefit from ID caching ?\n\n> > > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > >\n> > > > > Signed-off-by: Umang Jain \n> > > > > ---\n> > > > > src/android/camera_device.cpp | 15 +++\n> > > > > src/android/camera_device.h | 8 +-\n> > > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > >\n> > > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > > index d918350..a79bb69 100644\n> > > > > --- a/src/android/camera_device.cpp\n> > > > > +++ b/src/android/camera_device.cpp\n> > > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > > delete it.second;\n> > > > > }\n> > > > >\n> > > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > > + const std::shared_ptr &cam)\n> > > > > +{\n> > > > > + struct Deleter : std::default_delete {\n> > > > > + void operator()(CameraDevice *camera)\n> > > > > + {\n> > > > > + delete camera;\n> > > > > + }\n> > > > > + };\n> > > > > +\n> > > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > > +\n> > > > > + return std::shared_ptr(camera, Deleter());\n> > > > > +}\n> > > >\n> > > > As Kieran points out I think this should be added in a separate as this\n> > > > one is quiet large and therefore hard to review.\n> > > >\n> > > > > +\n> > > > > /*\n> > > > > * Initialize the camera static information.\n> > > > > * This method is called before the camera device is opened.\n> > > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > > index 7be9e11..7f9e010 100644\n> > > > > --- a/src/android/camera_device.h\n> > > > > +++ b/src/android/camera_device.h\n> > > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > > class CameraDevice : protected libcamera::Loggable\n> > > > > {\n> > > > > public:\n> > > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > - ~CameraDevice();\n> > > > > + static std::shared_ptr create(unsigned int id,\n> > > > > + const std::shared_ptr &cam);\n> > > > >\n> > > > > int initialize();\n> > > > >\n> > > > > @@ -57,6 +57,7 @@ public:\n> > > > >\n> > > > > unsigned int id() const { return id_; }\n> > > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > >\n> > > > Same here this can be done in a separate commit. Also I think this could\n> > > > be named camera() instead of getCamera()\n> > > >\n> > > > >\n> > > > > int facing() const { return facing_; }\n> > > > > int orientation() const { return orientation_; }\n> > > > > @@ -72,6 +73,9 @@ protected:\n> > > > > std::string logPrefix() const override;\n> > > > >\n> > > > > private:\n> > > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > + ~CameraDevice();\n> > > > > +\n> > > > > struct Camera3RequestDescriptor {\n> > > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > > unsigned int numBuffers);\n> > > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > > index 3d6d2b4..fdde2c0 100644\n> > > > > --- a/src/android/camera_hal_manager.cpp\n> > > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > > @@ -8,6 +8,7 @@\n> > > > > #include \"camera_hal_manager.h\"\n> > > > >\n> > > > > #include \n> > > > > +#include \n> > > > >\n> > > > > #include \"libcamera/internal/log.h\"\n> > > > >\n> > > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > > CameraHalManager::~CameraHalManager()\n> > > > > {\n> > > > > cameras_.clear();\n> > > > > + camerasMap_.clear();\n> > > > >\n> > > > > if (cameraManager_) {\n> > > > > cameraManager_->stop();\n> > > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > > {\n> > > > > cameraManager_ = new CameraManager();\n> > > > >\n> > > > > + /* Support camera hotplug. */\n> > > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > > +\n> > > > > + cameraCounter_ = 0;\n> > > > > + externalCameraCounter_ = 1000;\n> > > > > +\n> > > > > int ret = cameraManager_->start();\n> > > > > if (ret) {\n> > > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > > return ret;\n> > > > > }\n> > > > >\n> > > > > - /*\n> > > > > - * For each Camera registered in the system, a CameraDevice\n> > > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > > - *\n> > > > > - * \\todo Support camera hotplug.\n> > > > > - */\n> > > > > - unsigned int index = 0;\n> > > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > > - ret = camera->initialize();\n> > > > > - if (ret)\n> > > > > - continue;\n> > > > > -\n> > > > > - cameras_.emplace_back(camera);\n> > > > > - ++index;\n> > > > > - }\n> > > > > -\n> > > > > return 0;\n> > > > > }\n> > > > >\n> > > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > const hw_module_t *hardwareModule)\n> > > > > {\n> > > > > - if (id >= numCameras()) {\n> > > > > + MutexLocker locker(mutex_);\n> > > > > +\n> > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > + [id](std::shared_ptr &cam) {\n> > > > > + return cam->id() == id;\n> > > > > + });\n> > > > > + if (iter == cameras_.end()) {\n> > > >\n> > > > I think you should break this (and the similar ones below) into private\n> > > > helpers instead if implementing the logic in-place\n> > > >\n> > > > CameraHalManager::cameraFromAndroidId(..);\n> > > > CameraHalManager::androidIdFromCamera(..);\n> > > >\n> > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > return nullptr;\n> > > > > }\n> > > > >\n> > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > + CameraDevice *camera = iter->get();\n> > > > > +\n> > > > > if (camera->open(hardwareModule))\n> > > > > return nullptr;\n> > > > >\n> > > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > return camera;\n> > > > > }\n> > > > >\n> > > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > > +{\n> > > > > + unsigned int id;\n> > > > > + bool isCameraExternal = false;\n> > > > > + bool isCameraNew = false;\n> > > > > +\n> > > > > + MutexLocker locker(mutex_);\n> > > > > +\n> > > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > > + *\n> > > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > > + */\n> > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > + if (iter != camerasMap_.end()) {\n> > > > > + id = iter->second;\n> > > > > + } else {\n> > > > > + isCameraNew = true;\n> > > > > +\n> > > > > + /*\n> > > > > + * Now check if this is an external camera and assign\n> > > > > + * its id accordingly.\n> > > > > + */\n> > > > > + const ControlList &properties = cam->properties();\n> > > > > + if (properties.contains(properties::Location) &&\n> > > > > + properties.get(properties::Location) &\n> > > > > + properties::CameraLocationExternal) {\n> > > > > + isCameraExternal = true;\n> > > > > + id = externalCameraCounter_;\n> > > > > + } else {\n> > > > > + id = cameraCounter_;\n> > > > > + }\n> > > >\n> > > > As I understand it the information the HAL wants is whether or not the\n> > > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > > >\n> > > > If so I wonder if using the camera location as a judgment of the camera\n> > > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > > a location. I'm thinking cameras mounted at the end of instruments\n> > > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > > of a welding robot). I would imagine those cameras would be marked as\n> > > > located externally but they would not really be hot-pluggable.\n> > > >\n> > > > I understand we have no other way to report or detect this at the moment\n> > > > and I'm not pushing hard for this to be solved as part of this series if\n> > > > it's not easy. But I think a bigger comment here is needed explaining\n> > > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > > not really care if it's located internally or externally. I also think a\n> > > > \\todo should be added so it's not forgotten.\n> > > >\n> > > > > + }\n> > > > > +\n> > > > > + /*\n> > > > > + * For each Camera registered in the system, a CameraDevice\n> > > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > > + */\n> > > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > > + int ret = camera->initialize();\n> > > > > + if (ret) {\n> > > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > > + return;\n> > > > > + }\n> > > > > +\n> > > > > + if (isCameraNew) {\n> > > > > + camerasMap_.emplace(cam->id(), id);\n> > > > > +\n> > > > > + if (isCameraExternal)\n> > > > > + externalCameraCounter_++;\n> > > > > + else\n> > > > > + cameraCounter_++;\n> > > > > + }\n> > > > > +\n> > > > > + cameras_.emplace_back(std::move(camera));\n> > > > > +\n> > > > > + if (callbacks_)\n> > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > > +}\n> > > > > +\n> > > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > > +{\n> > > > > + MutexLocker locker(mutex_);\n> > > > > +\n> > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > + [cam](std::shared_ptr &camera) {\n> > > > > + return cam.get() == camera->getCamera();\n> > > > > + });\n> > > > > + if (iter == cameras_.end())\n> > > > > + return;\n> > > > > +\n> > > > > + unsigned int id = (*iter)->id();\n> > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > > + cameras_.erase(iter);\n> > > > > +\n> > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > > +}\n> > > > > +\n> > > > > unsigned int CameraHalManager::numCameras() const\n> > > > > {\n> > > > > - return cameraManager_->cameras().size();\n> > > > > + return cameraCounter_;\n> > > > > }\n> > > > >\n> > > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > if (!info)\n> > > > > return -EINVAL;\n> > > > >\n> > > > > - if (id >= numCameras()) {\n> > > > > + MutexLocker locker(mutex_);\n> > > > > +\n> > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > + [id](std::shared_ptr &cam) {\n> > > > > + return cam->id() == id;\n> > > > > + });\n> > > > > + if (iter == cameras_.end()) {\n> > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > return -EINVAL;\n> > > > > }\n> > > > >\n> > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > + CameraDevice *camera = iter->get();\n> > > > >\n> > > > > info->facing = camera->facing();\n> > > > > info->orientation = camera->orientation();\n> > > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > > {\n> > > > > callbacks_ = callbacks;\n> > > > > +\n> > > > > + MutexLocker locker(mutex_);\n> > > > > + /*\n> > > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > > + * be present at module load time, by the framework.\n> > > > > + */\n> > > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > + if (iter == camerasMap_.end())\n> > > > > + continue;\n> > > > > +\n> > > > > + unsigned int id = iter->second;\n> > > > > + const ControlList &properties = cam->properties();\n> > > > > + if (properties.contains(properties::Location) &&\n> > > > > + properties.get(properties::Location) &\n> > > > > + properties::CameraLocationExternal) {\n> > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > + }\n> > > > > + }\n> > > > > }\n> > > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > > index a582f04..7c481d4 100644\n> > > > > --- a/src/android/camera_hal_manager.h\n> > > > > +++ b/src/android/camera_hal_manager.h\n> > > > > @@ -7,6 +7,8 @@\n> > > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > > >\n> > > > > +#include \n> > > > > +#include \n> > > > > #include \n> > > > > #include \n> > > > >\n> > > > > @@ -18,6 +20,9 @@\n> > > > >\n> > > > > class CameraDevice;\n> > > > >\n> > > > > +using Mutex = std::mutex;\n> > > > > +using MutexLocker = std::unique_lock;\n> > > > > +\n> > > > > class CameraHalManager\n> > > > > {\n> > > > > public:\n> > > > > @@ -33,10 +38,18 @@ public:\n> > > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > > >\n> > > > > private:\n> > > > > + void cameraAdded(std::shared_ptr cam);\n> > > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > > +\n> > > > > libcamera::CameraManager *cameraManager_;\n> > > > >\n> > > > > const camera_module_callbacks_t *callbacks_;\n> > > > > - std::vector> cameras_;\n> > > > > + std::vector> cameras_;\n> > > > > + std::map camerasMap_;\n> > > >\n> > > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > > camerasMap_ could be merged to a\n> > > >\n> > > > std::map> cameras_;\n> > > >\n> > > > Which would eliminate the possibility of them going out-of-sync.\n> > > >\n> > > > > + Mutex mutex_;\n> > > > > +\n> > > > > + unsigned int externalCameraCounter_;\n> > > > > + unsigned int cameraCounter_;\n> > > > > };\n> > > > >\n> > > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4FA9EBD87C\n\tfor ;\n\tWed, 19 Aug 2020 15:52:41 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B90BC61ED9;\n\tWed, 19 Aug 2020 17:52:40 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5FBF160382\n\tfor ;\n\tWed, 19 Aug 2020 17:52:39 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C8EF929E;\n\tWed, 19 Aug 2020 17:52:38 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"IpCdhRWB\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597852359;\n\tbh=P4ipW2lR4LMlYsLyNSC1UAwJD1X/Uba6Y3iHWeUCFCg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=IpCdhRWBM05xgnUx3lAEjoDJFAg8MpLE8FOpbmVtbywIPf8R4E/5EZBcCe6GTVmjR\n\tMmmdhFFULowNyVAoigtU4WxLAyUwBBsAo0OOa63RA/90rQbT2+r4BNKctNRtHx5HJc\n\tM6p2Mq9AiY2/+6OlvT5TxX+V75N+ynbsQfruGi2k=","Date":"Wed, 19 Aug 2020 18:52:21 +0300","From":"Laurent Pinchart ","To":"Shik Chen ","Message-ID":"<20200819155221.GP6049@pendragon.ideasonboard.com>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12051,"web_url":"https://patchwork.libcamera.org/comment/12051/","msgid":"","date":"2020-08-19T16:04:41","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":8,"url":"https://patchwork.libcamera.org/api/people/8/","name":"Shik Chen","email":"shik@google.com"},"content":"Hi Laurent,\n\nOn Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart\n wrote:\n>\n> Hi Shik,\n>\n> On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> > On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> > >\n> > > Hi Shik,\n> > >\n> > > Would it be possible for you to have a look at the question below ?\n> > >\n> > > On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > > > Hi Niklas,\n> > > >\n> > > > (CC'ing Shik)\n> > > >\n> > > > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > > > know about the hotplug events and change the status of cameras being\n> > > > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > > >\n> > > > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > > > current implementation, we assume all UVC cameras are external cameras.\n> > > > >\n> > > > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > > > benefit of preserving the same numerical ID between two plug events,\n> > > > > while I can think of quiet a few cons.\n> > > > >\n> > > > > - It's confusing when debugging as un-plugging and then replugging the\n> > > > > same camera will result in logs where the numerical ID is the same for\n> > > > > both. This may even result in things working by \"chance\" is it reuses\n> > > > > an already known numerical ID.\n> > > > >\n> > > > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > > > generate a different Camera::id() result and then be treated as a new\n> > > > > camera vs plugging it in the same port and it then being treatad as a\n> > > > > new camera.\n> > > > >\n> > > > > - The logic in this patch is more complex due to it both having to deal\n> > > > > with known and new cameras.\n> > > > >\n> > > > > What is the benefit of this cache that I'm missing?\n> > > >\n> > > > It's not clear whether Android requires it. It requires different\n> > > > cameras to have different IDs, but I couldn't find a mention of how\n> > > > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > > > ID, I don't know if it's mandatory though.\n> > > >\n> > > > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > > > are requirements to keep the same ID when a camera is unplugged and\n> > > > replugged ?\n> >\n> > It's for persistency in settings UI. Chrome uses the camera id as the key of\n> > the default camera in chrome://settings/content/camera. Reuse the same id for\n> > the same camera so the default camera would be changed back after replug.\n> >\n> > It's introduced in this CL: https://crrev.com/c/1608985\n>\n> Thank you for the information. We have a similar caching system, which\n> however suffers from two limitations:\n>\n> - If a camera is unplugged, and a different camera with the same VID:PID\n> is plugged into the same USB port, the new camera be given the same ID\n> as the previous camera.\n\nThe current CrOS implementation will also use the same id in this case, and I\nthink it should be fine.\n\n>\n> - If a camera is unplugged and replugged in a different USB port, it\n> will be given a different ID.\n\nHmm this looks non-ideal and may confuse the users.\n\n>\n> Is that acceptable from a Chrome OS point of view ?\n>\n> Would you happen to know how Android uses camera IDs, and whether it\n> requires or could benefit from ID caching ?\n\nEach camera app could have a similar preference caching system like the\nsettings UI, so it would be better to reuse the same id for the same camera.\nAFAIK there is no such requirement at the framework level.\n\n>\n> > > > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > > >\n> > > > > > Signed-off-by: Umang Jain \n> > > > > > ---\n> > > > > > src/android/camera_device.cpp | 15 +++\n> > > > > > src/android/camera_device.h | 8 +-\n> > > > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > > >\n> > > > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > > > index d918350..a79bb69 100644\n> > > > > > --- a/src/android/camera_device.cpp\n> > > > > > +++ b/src/android/camera_device.cpp\n> > > > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > > > delete it.second;\n> > > > > > }\n> > > > > >\n> > > > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > > > + const std::shared_ptr &cam)\n> > > > > > +{\n> > > > > > + struct Deleter : std::default_delete {\n> > > > > > + void operator()(CameraDevice *camera)\n> > > > > > + {\n> > > > > > + delete camera;\n> > > > > > + }\n> > > > > > + };\n> > > > > > +\n> > > > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > > > +\n> > > > > > + return std::shared_ptr(camera, Deleter());\n> > > > > > +}\n> > > > >\n> > > > > As Kieran points out I think this should be added in a separate as this\n> > > > > one is quiet large and therefore hard to review.\n> > > > >\n> > > > > > +\n> > > > > > /*\n> > > > > > * Initialize the camera static information.\n> > > > > > * This method is called before the camera device is opened.\n> > > > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > > > index 7be9e11..7f9e010 100644\n> > > > > > --- a/src/android/camera_device.h\n> > > > > > +++ b/src/android/camera_device.h\n> > > > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > > > class CameraDevice : protected libcamera::Loggable\n> > > > > > {\n> > > > > > public:\n> > > > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > - ~CameraDevice();\n> > > > > > + static std::shared_ptr create(unsigned int id,\n> > > > > > + const std::shared_ptr &cam);\n> > > > > >\n> > > > > > int initialize();\n> > > > > >\n> > > > > > @@ -57,6 +57,7 @@ public:\n> > > > > >\n> > > > > > unsigned int id() const { return id_; }\n> > > > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > > >\n> > > > > Same here this can be done in a separate commit. Also I think this could\n> > > > > be named camera() instead of getCamera()\n> > > > >\n> > > > > >\n> > > > > > int facing() const { return facing_; }\n> > > > > > int orientation() const { return orientation_; }\n> > > > > > @@ -72,6 +73,9 @@ protected:\n> > > > > > std::string logPrefix() const override;\n> > > > > >\n> > > > > > private:\n> > > > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > + ~CameraDevice();\n> > > > > > +\n> > > > > > struct Camera3RequestDescriptor {\n> > > > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > > > unsigned int numBuffers);\n> > > > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > > > index 3d6d2b4..fdde2c0 100644\n> > > > > > --- a/src/android/camera_hal_manager.cpp\n> > > > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > > > @@ -8,6 +8,7 @@\n> > > > > > #include \"camera_hal_manager.h\"\n> > > > > >\n> > > > > > #include \n> > > > > > +#include \n> > > > > >\n> > > > > > #include \"libcamera/internal/log.h\"\n> > > > > >\n> > > > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > > > CameraHalManager::~CameraHalManager()\n> > > > > > {\n> > > > > > cameras_.clear();\n> > > > > > + camerasMap_.clear();\n> > > > > >\n> > > > > > if (cameraManager_) {\n> > > > > > cameraManager_->stop();\n> > > > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > > > {\n> > > > > > cameraManager_ = new CameraManager();\n> > > > > >\n> > > > > > + /* Support camera hotplug. */\n> > > > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > > > +\n> > > > > > + cameraCounter_ = 0;\n> > > > > > + externalCameraCounter_ = 1000;\n> > > > > > +\n> > > > > > int ret = cameraManager_->start();\n> > > > > > if (ret) {\n> > > > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > > > return ret;\n> > > > > > }\n> > > > > >\n> > > > > > - /*\n> > > > > > - * For each Camera registered in the system, a CameraDevice\n> > > > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > > > - *\n> > > > > > - * \\todo Support camera hotplug.\n> > > > > > - */\n> > > > > > - unsigned int index = 0;\n> > > > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > > > - ret = camera->initialize();\n> > > > > > - if (ret)\n> > > > > > - continue;\n> > > > > > -\n> > > > > > - cameras_.emplace_back(camera);\n> > > > > > - ++index;\n> > > > > > - }\n> > > > > > -\n> > > > > > return 0;\n> > > > > > }\n> > > > > >\n> > > > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > const hw_module_t *hardwareModule)\n> > > > > > {\n> > > > > > - if (id >= numCameras()) {\n> > > > > > + MutexLocker locker(mutex_);\n> > > > > > +\n> > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > + return cam->id() == id;\n> > > > > > + });\n> > > > > > + if (iter == cameras_.end()) {\n> > > > >\n> > > > > I think you should break this (and the similar ones below) into private\n> > > > > helpers instead if implementing the logic in-place\n> > > > >\n> > > > > CameraHalManager::cameraFromAndroidId(..);\n> > > > > CameraHalManager::androidIdFromCamera(..);\n> > > > >\n> > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > return nullptr;\n> > > > > > }\n> > > > > >\n> > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > + CameraDevice *camera = iter->get();\n> > > > > > +\n> > > > > > if (camera->open(hardwareModule))\n> > > > > > return nullptr;\n> > > > > >\n> > > > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > return camera;\n> > > > > > }\n> > > > > >\n> > > > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > > > +{\n> > > > > > + unsigned int id;\n> > > > > > + bool isCameraExternal = false;\n> > > > > > + bool isCameraNew = false;\n> > > > > > +\n> > > > > > + MutexLocker locker(mutex_);\n> > > > > > +\n> > > > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > > > + *\n> > > > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > > > + */\n> > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > + if (iter != camerasMap_.end()) {\n> > > > > > + id = iter->second;\n> > > > > > + } else {\n> > > > > > + isCameraNew = true;\n> > > > > > +\n> > > > > > + /*\n> > > > > > + * Now check if this is an external camera and assign\n> > > > > > + * its id accordingly.\n> > > > > > + */\n> > > > > > + const ControlList &properties = cam->properties();\n> > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > + properties.get(properties::Location) &\n> > > > > > + properties::CameraLocationExternal) {\n> > > > > > + isCameraExternal = true;\n> > > > > > + id = externalCameraCounter_;\n> > > > > > + } else {\n> > > > > > + id = cameraCounter_;\n> > > > > > + }\n> > > > >\n> > > > > As I understand it the information the HAL wants is whether or not the\n> > > > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > > > >\n> > > > > If so I wonder if using the camera location as a judgment of the camera\n> > > > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > > > a location. I'm thinking cameras mounted at the end of instruments\n> > > > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > > > of a welding robot). I would imagine those cameras would be marked as\n> > > > > located externally but they would not really be hot-pluggable.\n> > > > >\n> > > > > I understand we have no other way to report or detect this at the moment\n> > > > > and I'm not pushing hard for this to be solved as part of this series if\n> > > > > it's not easy. But I think a bigger comment here is needed explaining\n> > > > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > > > not really care if it's located internally or externally. I also think a\n> > > > > \\todo should be added so it's not forgotten.\n> > > > >\n> > > > > > + }\n> > > > > > +\n> > > > > > + /*\n> > > > > > + * For each Camera registered in the system, a CameraDevice\n> > > > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > > > + */\n> > > > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > > > + int ret = camera->initialize();\n> > > > > > + if (ret) {\n> > > > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > > > + return;\n> > > > > > + }\n> > > > > > +\n> > > > > > + if (isCameraNew) {\n> > > > > > + camerasMap_.emplace(cam->id(), id);\n> > > > > > +\n> > > > > > + if (isCameraExternal)\n> > > > > > + externalCameraCounter_++;\n> > > > > > + else\n> > > > > > + cameraCounter_++;\n> > > > > > + }\n> > > > > > +\n> > > > > > + cameras_.emplace_back(std::move(camera));\n> > > > > > +\n> > > > > > + if (callbacks_)\n> > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > > > +}\n> > > > > > +\n> > > > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > > > +{\n> > > > > > + MutexLocker locker(mutex_);\n> > > > > > +\n> > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > + [cam](std::shared_ptr &camera) {\n> > > > > > + return cam.get() == camera->getCamera();\n> > > > > > + });\n> > > > > > + if (iter == cameras_.end())\n> > > > > > + return;\n> > > > > > +\n> > > > > > + unsigned int id = (*iter)->id();\n> > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > > > + cameras_.erase(iter);\n> > > > > > +\n> > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > > > +}\n> > > > > > +\n> > > > > > unsigned int CameraHalManager::numCameras() const\n> > > > > > {\n> > > > > > - return cameraManager_->cameras().size();\n> > > > > > + return cameraCounter_;\n> > > > > > }\n> > > > > >\n> > > > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > if (!info)\n> > > > > > return -EINVAL;\n> > > > > >\n> > > > > > - if (id >= numCameras()) {\n> > > > > > + MutexLocker locker(mutex_);\n> > > > > > +\n> > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > + return cam->id() == id;\n> > > > > > + });\n> > > > > > + if (iter == cameras_.end()) {\n> > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > return -EINVAL;\n> > > > > > }\n> > > > > >\n> > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > + CameraDevice *camera = iter->get();\n> > > > > >\n> > > > > > info->facing = camera->facing();\n> > > > > > info->orientation = camera->orientation();\n> > > > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > > > {\n> > > > > > callbacks_ = callbacks;\n> > > > > > +\n> > > > > > + MutexLocker locker(mutex_);\n> > > > > > + /*\n> > > > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > > > + * be present at module load time, by the framework.\n> > > > > > + */\n> > > > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > + if (iter == camerasMap_.end())\n> > > > > > + continue;\n> > > > > > +\n> > > > > > + unsigned int id = iter->second;\n> > > > > > + const ControlList &properties = cam->properties();\n> > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > + properties.get(properties::Location) &\n> > > > > > + properties::CameraLocationExternal) {\n> > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > + }\n> > > > > > + }\n> > > > > > }\n> > > > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > > > index a582f04..7c481d4 100644\n> > > > > > --- a/src/android/camera_hal_manager.h\n> > > > > > +++ b/src/android/camera_hal_manager.h\n> > > > > > @@ -7,6 +7,8 @@\n> > > > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > > > >\n> > > > > > +#include \n> > > > > > +#include \n> > > > > > #include \n> > > > > > #include \n> > > > > >\n> > > > > > @@ -18,6 +20,9 @@\n> > > > > >\n> > > > > > class CameraDevice;\n> > > > > >\n> > > > > > +using Mutex = std::mutex;\n> > > > > > +using MutexLocker = std::unique_lock;\n> > > > > > +\n> > > > > > class CameraHalManager\n> > > > > > {\n> > > > > > public:\n> > > > > > @@ -33,10 +38,18 @@ public:\n> > > > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > > > >\n> > > > > > private:\n> > > > > > + void cameraAdded(std::shared_ptr cam);\n> > > > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > > > +\n> > > > > > libcamera::CameraManager *cameraManager_;\n> > > > > >\n> > > > > > const camera_module_callbacks_t *callbacks_;\n> > > > > > - std::vector> cameras_;\n> > > > > > + std::vector> cameras_;\n> > > > > > + std::map camerasMap_;\n> > > > >\n> > > > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > > > camerasMap_ could be merged to a\n> > > > >\n> > > > > std::map> cameras_;\n> > > > >\n> > > > > Which would eliminate the possibility of them going out-of-sync.\n> > > > >\n> > > > > > + Mutex mutex_;\n> > > > > > +\n> > > > > > + unsigned int externalCameraCounter_;\n> > > > > > + unsigned int cameraCounter_;\n> > > > > > };\n> > > > > >\n> > > > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n\n- shik","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A044CBE173\n\tfor ;\n\tWed, 19 Aug 2020 16:05:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 302AC61EA0;\n\tWed, 19 Aug 2020 18:05:02 +0200 (CEST)","from mail-qt1-x843.google.com (mail-qt1-x843.google.com\n\t[IPv6:2607:f8b0:4864:20::843])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F3FEA60382\n\tfor ;\n\tWed, 19 Aug 2020 18:04:59 +0200 (CEST)","by mail-qt1-x843.google.com with SMTP id s23so18142289qtq.12\n\tfor ;\n\tWed, 19 Aug 2020 09:04:59 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"MbMBokpw\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=20161025; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=IBLZSkjxyi7OxWTX0qapBhhmKKp3weWKr+frp2ep/vE=;\n\tb=MbMBokpwGGUeIcBer3hv4wFDaYbOzrrhzo4TRO0XXlVKmGmq01wG1Zxrp6WMrgA6KA\n\torEbYC+VBXVcakls3Tj5jqAbRAXePL0cnHsnxMWQzfskqYS/M07PH5e5kqwMWSDa0dN2\n\tJ7enGEEc59v0bVaKryi4UTG0B7C9mKXiHOZqGRlADw/ljUfVgo23A/t9H+DXisJvfV/J\n\tMaN5DNz/XUjzQ6SQwS2euba7ZrpAS9OzZw0w+IRSq8d+eBhspDodXcFh9CxNz1UiPpZv\n\tf+KsPrOUbRsDoq4o+C88dyS/OAoLgO9HYSOhewTb7glaI25tA9tOyp1m1L6kjhfgbzbR\n\tBu/A==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=IBLZSkjxyi7OxWTX0qapBhhmKKp3weWKr+frp2ep/vE=;\n\tb=lQ8HkdjmPGaKezkq89pRc+kUaKwiZEiP+Ll1RJ6iwItl9BiFbrsTbNm0vXFOwe8V6p\n\ttm0ik+FwM0xQ0CA4L9wx16dEeWj2E8vuTqmODD0VIkjum7RfG2F666cY4qWXnuHMrD5S\n\tr3SWB3UG7tJBUBvHlmyOoLD512EPpmlpHahXUEN1IduHW/9IRNO6l4sApztfkpYOoq3n\n\tj8abzD3uD1KHquzJG/sXyK1cA3uzqTOJOCAVpByEkqCSsl3mAJqGMXRTH+bfdf/fO/vc\n\tAl2+PmIBPZwVn7KnJ7VsIFp3CQ5/uet1ksJApqF5rUlJHpFqTkl8iaCYC7etpg7U9w4P\n\tOhkg==","X-Gm-Message-State":"AOAM530vlRTbZzN8WoJ2zgBjHEZhF6DG9Z1Z9cxSL3hzIikI21gd0fcU\n\td4DkLJhWCJC490k80CIozoFwdM7SB48Z8ERbacFmZnmq+x67Ljvr","X-Google-Smtp-Source":"ABdhPJwtJgC1pmQ+dJ9sPNdW70xpNZrQjA79zr19yaFEHQ1aiDkBK8WEvBrE5JZNUAwDBzAVZXfFa+ohBNn68Elcc+0=","X-Received":"by 2002:aed:27c8:: with SMTP id\n\tm8mr23218071qtg.302.1597853098171; \n\tWed, 19 Aug 2020 09:04:58 -0700 (PDT)","MIME-Version":"1.0","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>","In-Reply-To":"<20200819155221.GP6049@pendragon.ideasonboard.com>","From":"Shik Chen ","Date":"Thu, 20 Aug 2020 00:04:41 +0800","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12052,"web_url":"https://patchwork.libcamera.org/comment/12052/","msgid":"<20200819162140.GQ6049@pendragon.ideasonboard.com>","date":"2020-08-19T16:21:40","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Shik,\n\nOn Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> > On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> > > On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> > > >\n> > > > Hi Shik,\n> > > >\n> > > > Would it be possible for you to have a look at the question below ?\n> > > >\n> > > > On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > > > > Hi Niklas,\n> > > > >\n> > > > > (CC'ing Shik)\n> > > > >\n> > > > > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > > > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > > > > know about the hotplug events and change the status of cameras being\n> > > > > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > > > >\n> > > > > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > > > > current implementation, we assume all UVC cameras are external cameras.\n> > > > > >\n> > > > > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > > > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > > > > benefit of preserving the same numerical ID between two plug events,\n> > > > > > while I can think of quiet a few cons.\n> > > > > >\n> > > > > > - It's confusing when debugging as un-plugging and then replugging the\n> > > > > > same camera will result in logs where the numerical ID is the same for\n> > > > > > both. This may even result in things working by \"chance\" is it reuses\n> > > > > > an already known numerical ID.\n> > > > > >\n> > > > > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > > > > generate a different Camera::id() result and then be treated as a new\n> > > > > > camera vs plugging it in the same port and it then being treatad as a\n> > > > > > new camera.\n> > > > > >\n> > > > > > - The logic in this patch is more complex due to it both having to deal\n> > > > > > with known and new cameras.\n> > > > > >\n> > > > > > What is the benefit of this cache that I'm missing?\n> > > > >\n> > > > > It's not clear whether Android requires it. It requires different\n> > > > > cameras to have different IDs, but I couldn't find a mention of how\n> > > > > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > > > > ID, I don't know if it's mandatory though.\n> > > > >\n> > > > > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > > > > are requirements to keep the same ID when a camera is unplugged and\n> > > > > replugged ?\n> > >\n> > > It's for persistency in settings UI. Chrome uses the camera id as the key of\n> > > the default camera in chrome://settings/content/camera. Reuse the same id for\n> > > the same camera so the default camera would be changed back after replug.\n> > >\n> > > It's introduced in this CL: https://crrev.com/c/1608985\n> >\n> > Thank you for the information. We have a similar caching system, which\n> > however suffers from two limitations:\n> >\n> > - If a camera is unplugged, and a different camera with the same VID:PID\n> > is plugged into the same USB port, the new camera be given the same ID\n> > as the previous camera.\n> \n> The current CrOS implementation will also use the same id in this case, and I\n> think it should be fine.\n> \n> > - If a camera is unplugged and replugged in a different USB port, it\n> > will be given a different ID.\n> \n> Hmm this looks non-ideal and may confuse the users.\n\nWe will try to fix that on top\n\nThis isn't trivial to handle, and there are corner cases. I'm thinking\nin particular about a system where two identical external USB cameras\nwould be set up to point at particular locations. How to handle cameras\nstable IDs in that case is debatable, and I don't think the problem can\nbe solved unless the cameras report distinct serial numbers (and most\nwebcams don't :-S). At the moment, libcamera identifies each camera with\na stable ID (guaranteed to be unique and stable across reboots, as long\nas the hardware and firmware are not modified) made of the concatenation\nof\n\n- The USB controller DT or ACPI path\n- The USB port(s) number(s) (starting at the root hub, going through\n every hub)\n- The USB device VID:PID\n\nThe Android ID is then assigned based on the libcamera ID, with a cache\nto ensure that the same HAL ID will be given to the same libcamera ID.\nIf we were to drop the USB port number, we'd have to find another way to\ncreate a stable and unique camera ID.\n\n> > Is that acceptable from a Chrome OS point of view ?\n> >\n> > Would you happen to know how Android uses camera IDs, and whether it\n> > requires or could benefit from ID caching ?\n> \n> Each camera app could have a similar preference caching system like the\n> settings UI, so it would be better to reuse the same id for the same camera.\n> AFAIK there is no such requirement at the framework level.\n\nThanks.\n\n> > > > > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > > > >\n> > > > > > > Signed-off-by: Umang Jain \n> > > > > > > ---\n> > > > > > > src/android/camera_device.cpp | 15 +++\n> > > > > > > src/android/camera_device.h | 8 +-\n> > > > > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > > > >\n> > > > > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > > > > index d918350..a79bb69 100644\n> > > > > > > --- a/src/android/camera_device.cpp\n> > > > > > > +++ b/src/android/camera_device.cpp\n> > > > > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > > > > delete it.second;\n> > > > > > > }\n> > > > > > >\n> > > > > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > > > > + const std::shared_ptr &cam)\n> > > > > > > +{\n> > > > > > > + struct Deleter : std::default_delete {\n> > > > > > > + void operator()(CameraDevice *camera)\n> > > > > > > + {\n> > > > > > > + delete camera;\n> > > > > > > + }\n> > > > > > > + };\n> > > > > > > +\n> > > > > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > > > > +\n> > > > > > > + return std::shared_ptr(camera, Deleter());\n> > > > > > > +}\n> > > > > >\n> > > > > > As Kieran points out I think this should be added in a separate as this\n> > > > > > one is quiet large and therefore hard to review.\n> > > > > >\n> > > > > > > +\n> > > > > > > /*\n> > > > > > > * Initialize the camera static information.\n> > > > > > > * This method is called before the camera device is opened.\n> > > > > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > > > > index 7be9e11..7f9e010 100644\n> > > > > > > --- a/src/android/camera_device.h\n> > > > > > > +++ b/src/android/camera_device.h\n> > > > > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > > > > class CameraDevice : protected libcamera::Loggable\n> > > > > > > {\n> > > > > > > public:\n> > > > > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > - ~CameraDevice();\n> > > > > > > + static std::shared_ptr create(unsigned int id,\n> > > > > > > + const std::shared_ptr &cam);\n> > > > > > >\n> > > > > > > int initialize();\n> > > > > > >\n> > > > > > > @@ -57,6 +57,7 @@ public:\n> > > > > > >\n> > > > > > > unsigned int id() const { return id_; }\n> > > > > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > > > >\n> > > > > > Same here this can be done in a separate commit. Also I think this could\n> > > > > > be named camera() instead of getCamera()\n> > > > > >\n> > > > > > >\n> > > > > > > int facing() const { return facing_; }\n> > > > > > > int orientation() const { return orientation_; }\n> > > > > > > @@ -72,6 +73,9 @@ protected:\n> > > > > > > std::string logPrefix() const override;\n> > > > > > >\n> > > > > > > private:\n> > > > > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > + ~CameraDevice();\n> > > > > > > +\n> > > > > > > struct Camera3RequestDescriptor {\n> > > > > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > > > > unsigned int numBuffers);\n> > > > > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > > > > index 3d6d2b4..fdde2c0 100644\n> > > > > > > --- a/src/android/camera_hal_manager.cpp\n> > > > > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > > > > @@ -8,6 +8,7 @@\n> > > > > > > #include \"camera_hal_manager.h\"\n> > > > > > >\n> > > > > > > #include \n> > > > > > > +#include \n> > > > > > >\n> > > > > > > #include \"libcamera/internal/log.h\"\n> > > > > > >\n> > > > > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > > > > CameraHalManager::~CameraHalManager()\n> > > > > > > {\n> > > > > > > cameras_.clear();\n> > > > > > > + camerasMap_.clear();\n> > > > > > >\n> > > > > > > if (cameraManager_) {\n> > > > > > > cameraManager_->stop();\n> > > > > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > > > > {\n> > > > > > > cameraManager_ = new CameraManager();\n> > > > > > >\n> > > > > > > + /* Support camera hotplug. */\n> > > > > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > > > > +\n> > > > > > > + cameraCounter_ = 0;\n> > > > > > > + externalCameraCounter_ = 1000;\n> > > > > > > +\n> > > > > > > int ret = cameraManager_->start();\n> > > > > > > if (ret) {\n> > > > > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > > > > return ret;\n> > > > > > > }\n> > > > > > >\n> > > > > > > - /*\n> > > > > > > - * For each Camera registered in the system, a CameraDevice\n> > > > > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > > > > - *\n> > > > > > > - * \\todo Support camera hotplug.\n> > > > > > > - */\n> > > > > > > - unsigned int index = 0;\n> > > > > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > > > > - ret = camera->initialize();\n> > > > > > > - if (ret)\n> > > > > > > - continue;\n> > > > > > > -\n> > > > > > > - cameras_.emplace_back(camera);\n> > > > > > > - ++index;\n> > > > > > > - }\n> > > > > > > -\n> > > > > > > return 0;\n> > > > > > > }\n> > > > > > >\n> > > > > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > const hw_module_t *hardwareModule)\n> > > > > > > {\n> > > > > > > - if (id >= numCameras()) {\n> > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > +\n> > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > + return cam->id() == id;\n> > > > > > > + });\n> > > > > > > + if (iter == cameras_.end()) {\n> > > > > >\n> > > > > > I think you should break this (and the similar ones below) into private\n> > > > > > helpers instead if implementing the logic in-place\n> > > > > >\n> > > > > > CameraHalManager::cameraFromAndroidId(..);\n> > > > > > CameraHalManager::androidIdFromCamera(..);\n> > > > > >\n> > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > return nullptr;\n> > > > > > > }\n> > > > > > >\n> > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > +\n> > > > > > > if (camera->open(hardwareModule))\n> > > > > > > return nullptr;\n> > > > > > >\n> > > > > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > return camera;\n> > > > > > > }\n> > > > > > >\n> > > > > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > > > > +{\n> > > > > > > + unsigned int id;\n> > > > > > > + bool isCameraExternal = false;\n> > > > > > > + bool isCameraNew = false;\n> > > > > > > +\n> > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > +\n> > > > > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > > > > + *\n> > > > > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > > > > + */\n> > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > + if (iter != camerasMap_.end()) {\n> > > > > > > + id = iter->second;\n> > > > > > > + } else {\n> > > > > > > + isCameraNew = true;\n> > > > > > > +\n> > > > > > > + /*\n> > > > > > > + * Now check if this is an external camera and assign\n> > > > > > > + * its id accordingly.\n> > > > > > > + */\n> > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > + properties.get(properties::Location) &\n> > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > + isCameraExternal = true;\n> > > > > > > + id = externalCameraCounter_;\n> > > > > > > + } else {\n> > > > > > > + id = cameraCounter_;\n> > > > > > > + }\n> > > > > >\n> > > > > > As I understand it the information the HAL wants is whether or not the\n> > > > > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > > > > >\n> > > > > > If so I wonder if using the camera location as a judgment of the camera\n> > > > > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > > > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > > > > a location. I'm thinking cameras mounted at the end of instruments\n> > > > > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > > > > of a welding robot). I would imagine those cameras would be marked as\n> > > > > > located externally but they would not really be hot-pluggable.\n> > > > > >\n> > > > > > I understand we have no other way to report or detect this at the moment\n> > > > > > and I'm not pushing hard for this to be solved as part of this series if\n> > > > > > it's not easy. But I think a bigger comment here is needed explaining\n> > > > > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > > > > not really care if it's located internally or externally. I also think a\n> > > > > > \\todo should be added so it's not forgotten.\n> > > > > >\n> > > > > > > + }\n> > > > > > > +\n> > > > > > > + /*\n> > > > > > > + * For each Camera registered in the system, a CameraDevice\n> > > > > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > > > > + */\n> > > > > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > > > > + int ret = camera->initialize();\n> > > > > > > + if (ret) {\n> > > > > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > > > > + return;\n> > > > > > > + }\n> > > > > > > +\n> > > > > > > + if (isCameraNew) {\n> > > > > > > + camerasMap_.emplace(cam->id(), id);\n> > > > > > > +\n> > > > > > > + if (isCameraExternal)\n> > > > > > > + externalCameraCounter_++;\n> > > > > > > + else\n> > > > > > > + cameraCounter_++;\n> > > > > > > + }\n> > > > > > > +\n> > > > > > > + cameras_.emplace_back(std::move(camera));\n> > > > > > > +\n> > > > > > > + if (callbacks_)\n> > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > > > > +}\n> > > > > > > +\n> > > > > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > > > > +{\n> > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > +\n> > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > + [cam](std::shared_ptr &camera) {\n> > > > > > > + return cam.get() == camera->getCamera();\n> > > > > > > + });\n> > > > > > > + if (iter == cameras_.end())\n> > > > > > > + return;\n> > > > > > > +\n> > > > > > > + unsigned int id = (*iter)->id();\n> > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > > > > + cameras_.erase(iter);\n> > > > > > > +\n> > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > > > > +}\n> > > > > > > +\n> > > > > > > unsigned int CameraHalManager::numCameras() const\n> > > > > > > {\n> > > > > > > - return cameraManager_->cameras().size();\n> > > > > > > + return cameraCounter_;\n> > > > > > > }\n> > > > > > >\n> > > > > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > if (!info)\n> > > > > > > return -EINVAL;\n> > > > > > >\n> > > > > > > - if (id >= numCameras()) {\n> > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > +\n> > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > + return cam->id() == id;\n> > > > > > > + });\n> > > > > > > + if (iter == cameras_.end()) {\n> > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > return -EINVAL;\n> > > > > > > }\n> > > > > > >\n> > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > >\n> > > > > > > info->facing = camera->facing();\n> > > > > > > info->orientation = camera->orientation();\n> > > > > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > > > > {\n> > > > > > > callbacks_ = callbacks;\n> > > > > > > +\n> > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > + /*\n> > > > > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > > > > + * be present at module load time, by the framework.\n> > > > > > > + */\n> > > > > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > + if (iter == camerasMap_.end())\n> > > > > > > + continue;\n> > > > > > > +\n> > > > > > > + unsigned int id = iter->second;\n> > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > + properties.get(properties::Location) &\n> > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > + }\n> > > > > > > + }\n> > > > > > > }\n> > > > > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > > > > index a582f04..7c481d4 100644\n> > > > > > > --- a/src/android/camera_hal_manager.h\n> > > > > > > +++ b/src/android/camera_hal_manager.h\n> > > > > > > @@ -7,6 +7,8 @@\n> > > > > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > > > > >\n> > > > > > > +#include \n> > > > > > > +#include \n> > > > > > > #include \n> > > > > > > #include \n> > > > > > >\n> > > > > > > @@ -18,6 +20,9 @@\n> > > > > > >\n> > > > > > > class CameraDevice;\n> > > > > > >\n> > > > > > > +using Mutex = std::mutex;\n> > > > > > > +using MutexLocker = std::unique_lock;\n> > > > > > > +\n> > > > > > > class CameraHalManager\n> > > > > > > {\n> > > > > > > public:\n> > > > > > > @@ -33,10 +38,18 @@ public:\n> > > > > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > > > > >\n> > > > > > > private:\n> > > > > > > + void cameraAdded(std::shared_ptr cam);\n> > > > > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > > > > +\n> > > > > > > libcamera::CameraManager *cameraManager_;\n> > > > > > >\n> > > > > > > const camera_module_callbacks_t *callbacks_;\n> > > > > > > - std::vector> cameras_;\n> > > > > > > + std::vector> cameras_;\n> > > > > > > + std::map camerasMap_;\n> > > > > >\n> > > > > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > > > > camerasMap_ could be merged to a\n> > > > > >\n> > > > > > std::map> cameras_;\n> > > > > >\n> > > > > > Which would eliminate the possibility of them going out-of-sync.\n> > > > > >\n> > > > > > > + Mutex mutex_;\n> > > > > > > +\n> > > > > > > + unsigned int externalCameraCounter_;\n> > > > > > > + unsigned int cameraCounter_;\n> > > > > > > };\n> > > > > > >\n> > > > > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 773ECBD87C\n\tfor ;\n\tWed, 19 Aug 2020 16:22:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DFBD261ED9;\n\tWed, 19 Aug 2020 18:21:59 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C71660382\n\tfor ;\n\tWed, 19 Aug 2020 18:21:58 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B311229E;\n\tWed, 19 Aug 2020 18:21:57 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"J5y4uFfn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597854117;\n\tbh=LLqiH42fCFXm4Nz54yr3VtCIqhVoCDnzvq12iat0yUg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=J5y4uFfnN8WJWntIjtsNQw8XkeXKwNYsgMPukqB+3VLvNHVTmpiMUf5soX9SxB9XZ\n\tp3xzWXT9ma3nK0UsUFGJw1VdpoIfK3V2lsf7u7ZUNjj1KQ3XA6owwdQdflDOAIPvBu\n\tFxQxxYmyB64LHVfRnBv/I2TJ3X6z6AnvcS/CNyMY=","Date":"Wed, 19 Aug 2020 19:21:40 +0300","From":"Laurent Pinchart ","To":"Shik Chen ","Message-ID":"<20200819162140.GQ6049@pendragon.ideasonboard.com>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12053,"web_url":"https://patchwork.libcamera.org/comment/12053/","msgid":"<20200820071050.GB39265@wyvern>","date":"2020-08-20T07:12:10","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/people/5/","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"content":"Hi Laurent, Shik\n\nOn 2020-08-19 19:21:40 +0300, Laurent Pinchart wrote:\n> Hi Shik,\n> \n> On Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> > On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> > > On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> > > > On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> > > > >\n> > > > > Hi Shik,\n> > > > >\n> > > > > Would it be possible for you to have a look at the question below ?\n> > > > >\n> > > > > On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > > > > > Hi Niklas,\n> > > > > >\n> > > > > > (CC'ing Shik)\n> > > > > >\n> > > > > > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > > > > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > > > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > > > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > > > > > know about the hotplug events and change the status of cameras being\n> > > > > > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > > > > >\n> > > > > > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > > > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > > > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > > > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > > > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > > > > > current implementation, we assume all UVC cameras are external cameras.\n> > > > > > >\n> > > > > > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > > > > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > > > > > benefit of preserving the same numerical ID between two plug events,\n> > > > > > > while I can think of quiet a few cons.\n> > > > > > >\n> > > > > > > - It's confusing when debugging as un-plugging and then replugging the\n> > > > > > > same camera will result in logs where the numerical ID is the same for\n> > > > > > > both. This may even result in things working by \"chance\" is it reuses\n> > > > > > > an already known numerical ID.\n> > > > > > >\n> > > > > > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > > > > > generate a different Camera::id() result and then be treated as a new\n> > > > > > > camera vs plugging it in the same port and it then being treatad as a\n> > > > > > > new camera.\n> > > > > > >\n> > > > > > > - The logic in this patch is more complex due to it both having to deal\n> > > > > > > with known and new cameras.\n> > > > > > >\n> > > > > > > What is the benefit of this cache that I'm missing?\n> > > > > >\n> > > > > > It's not clear whether Android requires it. It requires different\n> > > > > > cameras to have different IDs, but I couldn't find a mention of how\n> > > > > > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > > > > > ID, I don't know if it's mandatory though.\n> > > > > >\n> > > > > > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > > > > > are requirements to keep the same ID when a camera is unplugged and\n> > > > > > replugged ?\n> > > >\n> > > > It's for persistency in settings UI. Chrome uses the camera id as the key of\n> > > > the default camera in chrome://settings/content/camera. Reuse the same id for\n> > > > the same camera so the default camera would be changed back after replug.\n> > > >\n> > > > It's introduced in this CL: https://crrev.com/c/1608985\n> > >\n> > > Thank you for the information. We have a similar caching system, which\n> > > however suffers from two limitations:\n\nWe have more limitations I'm afraid :-(\n\nIf I understand Shik correctly the numerical camera ID is stored \npersistently in the camera settings UI. Our code does not guarantee that \nthe numerical ID in the libcamera HAL is persistent. The cameras in the \nCameraManager is stored in a vector and is therefor susceptible to be \nenumerated in different order each time the HAL is started.\n\nWe could of course remedy this by sorting the camera vecotr before \nassigning numerical IDs. But this would still not be guaranteed to be \npersistent as more/less cameras registered in the HAL could alter the \nread out order. If we really wish to do this should we save/load the \ncamera ID to numerical ID mapping to file (not nice)?\n\n> > >\n> > > - If a camera is unplugged, and a different camera with the same VID:PID\n> > > is plugged into the same USB port, the new camera be given the same ID\n> > > as the previous camera.\n> > \n> > The current CrOS implementation will also use the same id in this case, and I\n> > think it should be fine.\n> > \n> > > - If a camera is unplugged and replugged in a different USB port, it\n> > > will be given a different ID.\n> > \n> > Hmm this looks non-ideal and may confuse the users.\n> \n> We will try to fix that on top\n> \n> This isn't trivial to handle, and there are corner cases. I'm thinking\n> in particular about a system where two identical external USB cameras\n> would be set up to point at particular locations. How to handle cameras\n> stable IDs in that case is debatable, and I don't think the problem can\n> be solved unless the cameras report distinct serial numbers (and most\n> webcams don't :-S). At the moment, libcamera identifies each camera with\n> a stable ID (guaranteed to be unique and stable across reboots, as long\n> as the hardware and firmware are not modified) made of the concatenation\n> of\n> \n> - The USB controller DT or ACPI path\n> - The USB port(s) number(s) (starting at the root hub, going through\n> every hub)\n> - The USB device VID:PID\n> \n> The Android ID is then assigned based on the libcamera ID, with a cache\n> to ensure that the same HAL ID will be given to the same libcamera ID.\n> If we were to drop the USB port number, we'd have to find another way to\n> create a stable and unique camera ID.\n> \n> > > Is that acceptable from a Chrome OS point of view ?\n> > >\n> > > Would you happen to know how Android uses camera IDs, and whether it\n> > > requires or could benefit from ID caching ?\n> > \n> > Each camera app could have a similar preference caching system like the\n> > settings UI, so it would be better to reuse the same id for the same camera.\n> > AFAIK there is no such requirement at the framework level.\n\nThis worries me a bit. Are we not altering the libcamera HAL to fit a \nsingle use-case, all be it an important user. But if the framework does \nnot document an expected behavior is it not tricky to implement \ncorrectly? What if we later find a different usage pattern in another \napplication which conflicts with what we are trying to do here?\n\n> \n> Thanks.\n> \n> > > > > > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > > > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > > > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > > > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > > > > >\n> > > > > > > > Signed-off-by: Umang Jain \n> > > > > > > > ---\n> > > > > > > > src/android/camera_device.cpp | 15 +++\n> > > > > > > > src/android/camera_device.h | 8 +-\n> > > > > > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > > > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > > > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > > > > >\n> > > > > > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > > > > > index d918350..a79bb69 100644\n> > > > > > > > --- a/src/android/camera_device.cpp\n> > > > > > > > +++ b/src/android/camera_device.cpp\n> > > > > > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > > > > > delete it.second;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > > > > > + const std::shared_ptr &cam)\n> > > > > > > > +{\n> > > > > > > > + struct Deleter : std::default_delete {\n> > > > > > > > + void operator()(CameraDevice *camera)\n> > > > > > > > + {\n> > > > > > > > + delete camera;\n> > > > > > > > + }\n> > > > > > > > + };\n> > > > > > > > +\n> > > > > > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > > > > > +\n> > > > > > > > + return std::shared_ptr(camera, Deleter());\n> > > > > > > > +}\n> > > > > > >\n> > > > > > > As Kieran points out I think this should be added in a separate as this\n> > > > > > > one is quiet large and therefore hard to review.\n> > > > > > >\n> > > > > > > > +\n> > > > > > > > /*\n> > > > > > > > * Initialize the camera static information.\n> > > > > > > > * This method is called before the camera device is opened.\n> > > > > > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > > > > > index 7be9e11..7f9e010 100644\n> > > > > > > > --- a/src/android/camera_device.h\n> > > > > > > > +++ b/src/android/camera_device.h\n> > > > > > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > > > > > class CameraDevice : protected libcamera::Loggable\n> > > > > > > > {\n> > > > > > > > public:\n> > > > > > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > > - ~CameraDevice();\n> > > > > > > > + static std::shared_ptr create(unsigned int id,\n> > > > > > > > + const std::shared_ptr &cam);\n> > > > > > > >\n> > > > > > > > int initialize();\n> > > > > > > >\n> > > > > > > > @@ -57,6 +57,7 @@ public:\n> > > > > > > >\n> > > > > > > > unsigned int id() const { return id_; }\n> > > > > > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > > > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > > > > >\n> > > > > > > Same here this can be done in a separate commit. Also I think this could\n> > > > > > > be named camera() instead of getCamera()\n> > > > > > >\n> > > > > > > >\n> > > > > > > > int facing() const { return facing_; }\n> > > > > > > > int orientation() const { return orientation_; }\n> > > > > > > > @@ -72,6 +73,9 @@ protected:\n> > > > > > > > std::string logPrefix() const override;\n> > > > > > > >\n> > > > > > > > private:\n> > > > > > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > > + ~CameraDevice();\n> > > > > > > > +\n> > > > > > > > struct Camera3RequestDescriptor {\n> > > > > > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > > > > > unsigned int numBuffers);\n> > > > > > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > > > > > index 3d6d2b4..fdde2c0 100644\n> > > > > > > > --- a/src/android/camera_hal_manager.cpp\n> > > > > > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > > > > > @@ -8,6 +8,7 @@\n> > > > > > > > #include \"camera_hal_manager.h\"\n> > > > > > > >\n> > > > > > > > #include \n> > > > > > > > +#include \n> > > > > > > >\n> > > > > > > > #include \"libcamera/internal/log.h\"\n> > > > > > > >\n> > > > > > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > > > > > CameraHalManager::~CameraHalManager()\n> > > > > > > > {\n> > > > > > > > cameras_.clear();\n> > > > > > > > + camerasMap_.clear();\n> > > > > > > >\n> > > > > > > > if (cameraManager_) {\n> > > > > > > > cameraManager_->stop();\n> > > > > > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > > > > > {\n> > > > > > > > cameraManager_ = new CameraManager();\n> > > > > > > >\n> > > > > > > > + /* Support camera hotplug. */\n> > > > > > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > > > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > > > > > +\n> > > > > > > > + cameraCounter_ = 0;\n> > > > > > > > + externalCameraCounter_ = 1000;\n> > > > > > > > +\n> > > > > > > > int ret = cameraManager_->start();\n> > > > > > > > if (ret) {\n> > > > > > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > > > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > > > > > return ret;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > - /*\n> > > > > > > > - * For each Camera registered in the system, a CameraDevice\n> > > > > > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > > > > > - *\n> > > > > > > > - * \\todo Support camera hotplug.\n> > > > > > > > - */\n> > > > > > > > - unsigned int index = 0;\n> > > > > > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > > > > > - ret = camera->initialize();\n> > > > > > > > - if (ret)\n> > > > > > > > - continue;\n> > > > > > > > -\n> > > > > > > > - cameras_.emplace_back(camera);\n> > > > > > > > - ++index;\n> > > > > > > > - }\n> > > > > > > > -\n> > > > > > > > return 0;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > > const hw_module_t *hardwareModule)\n> > > > > > > > {\n> > > > > > > > - if (id >= numCameras()) {\n> > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > +\n> > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > > + return cam->id() == id;\n> > > > > > > > + });\n> > > > > > > > + if (iter == cameras_.end()) {\n> > > > > > >\n> > > > > > > I think you should break this (and the similar ones below) into private\n> > > > > > > helpers instead if implementing the logic in-place\n> > > > > > >\n> > > > > > > CameraHalManager::cameraFromAndroidId(..);\n> > > > > > > CameraHalManager::androidIdFromCamera(..);\n> > > > > > >\n> > > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > > return nullptr;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > > +\n> > > > > > > > if (camera->open(hardwareModule))\n> > > > > > > > return nullptr;\n> > > > > > > >\n> > > > > > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > > return camera;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > > > > > +{\n> > > > > > > > + unsigned int id;\n> > > > > > > > + bool isCameraExternal = false;\n> > > > > > > > + bool isCameraNew = false;\n> > > > > > > > +\n> > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > +\n> > > > > > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > > > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > > > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > > > > > + *\n> > > > > > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > > > > > + */\n> > > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > > + if (iter != camerasMap_.end()) {\n> > > > > > > > + id = iter->second;\n> > > > > > > > + } else {\n> > > > > > > > + isCameraNew = true;\n> > > > > > > > +\n> > > > > > > > + /*\n> > > > > > > > + * Now check if this is an external camera and assign\n> > > > > > > > + * its id accordingly.\n> > > > > > > > + */\n> > > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > > + properties.get(properties::Location) &\n> > > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > > + isCameraExternal = true;\n> > > > > > > > + id = externalCameraCounter_;\n> > > > > > > > + } else {\n> > > > > > > > + id = cameraCounter_;\n> > > > > > > > + }\n> > > > > > >\n> > > > > > > As I understand it the information the HAL wants is whether or not the\n> > > > > > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > > > > > >\n> > > > > > > If so I wonder if using the camera location as a judgment of the camera\n> > > > > > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > > > > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > > > > > a location. I'm thinking cameras mounted at the end of instruments\n> > > > > > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > > > > > of a welding robot). I would imagine those cameras would be marked as\n> > > > > > > located externally but they would not really be hot-pluggable.\n> > > > > > >\n> > > > > > > I understand we have no other way to report or detect this at the moment\n> > > > > > > and I'm not pushing hard for this to be solved as part of this series if\n> > > > > > > it's not easy. But I think a bigger comment here is needed explaining\n> > > > > > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > > > > > not really care if it's located internally or externally. I also think a\n> > > > > > > \\todo should be added so it's not forgotten.\n> > > > > > >\n> > > > > > > > + }\n> > > > > > > > +\n> > > > > > > > + /*\n> > > > > > > > + * For each Camera registered in the system, a CameraDevice\n> > > > > > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > > > > > + */\n> > > > > > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > > > > > + int ret = camera->initialize();\n> > > > > > > > + if (ret) {\n> > > > > > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > > > > > + return;\n> > > > > > > > + }\n> > > > > > > > +\n> > > > > > > > + if (isCameraNew) {\n> > > > > > > > + camerasMap_.emplace(cam->id(), id);\n> > > > > > > > +\n> > > > > > > > + if (isCameraExternal)\n> > > > > > > > + externalCameraCounter_++;\n> > > > > > > > + else\n> > > > > > > > + cameraCounter_++;\n> > > > > > > > + }\n> > > > > > > > +\n> > > > > > > > + cameras_.emplace_back(std::move(camera));\n> > > > > > > > +\n> > > > > > > > + if (callbacks_)\n> > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > > > > > +}\n> > > > > > > > +\n> > > > > > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > > > > > +{\n> > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > +\n> > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > + [cam](std::shared_ptr &camera) {\n> > > > > > > > + return cam.get() == camera->getCamera();\n> > > > > > > > + });\n> > > > > > > > + if (iter == cameras_.end())\n> > > > > > > > + return;\n> > > > > > > > +\n> > > > > > > > + unsigned int id = (*iter)->id();\n> > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > > > > > + cameras_.erase(iter);\n> > > > > > > > +\n> > > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > > > > > +}\n> > > > > > > > +\n> > > > > > > > unsigned int CameraHalManager::numCameras() const\n> > > > > > > > {\n> > > > > > > > - return cameraManager_->cameras().size();\n> > > > > > > > + return cameraCounter_;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > if (!info)\n> > > > > > > > return -EINVAL;\n> > > > > > > >\n> > > > > > > > - if (id >= numCameras()) {\n> > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > +\n> > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > > + return cam->id() == id;\n> > > > > > > > + });\n> > > > > > > > + if (iter == cameras_.end()) {\n> > > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > > return -EINVAL;\n> > > > > > > > }\n> > > > > > > >\n> > > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > >\n> > > > > > > > info->facing = camera->facing();\n> > > > > > > > info->orientation = camera->orientation();\n> > > > > > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > > > > > {\n> > > > > > > > callbacks_ = callbacks;\n> > > > > > > > +\n> > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > + /*\n> > > > > > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > > > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > > > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > > > > > + * be present at module load time, by the framework.\n> > > > > > > > + */\n> > > > > > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > > + if (iter == camerasMap_.end())\n> > > > > > > > + continue;\n> > > > > > > > +\n> > > > > > > > + unsigned int id = iter->second;\n> > > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > > + properties.get(properties::Location) &\n> > > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > > + }\n> > > > > > > > + }\n> > > > > > > > }\n> > > > > > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > > > > > index a582f04..7c481d4 100644\n> > > > > > > > --- a/src/android/camera_hal_manager.h\n> > > > > > > > +++ b/src/android/camera_hal_manager.h\n> > > > > > > > @@ -7,6 +7,8 @@\n> > > > > > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > >\n> > > > > > > > +#include \n> > > > > > > > +#include \n> > > > > > > > #include \n> > > > > > > > #include \n> > > > > > > >\n> > > > > > > > @@ -18,6 +20,9 @@\n> > > > > > > >\n> > > > > > > > class CameraDevice;\n> > > > > > > >\n> > > > > > > > +using Mutex = std::mutex;\n> > > > > > > > +using MutexLocker = std::unique_lock;\n> > > > > > > > +\n> > > > > > > > class CameraHalManager\n> > > > > > > > {\n> > > > > > > > public:\n> > > > > > > > @@ -33,10 +38,18 @@ public:\n> > > > > > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > > > > > >\n> > > > > > > > private:\n> > > > > > > > + void cameraAdded(std::shared_ptr cam);\n> > > > > > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > > > > > +\n> > > > > > > > libcamera::CameraManager *cameraManager_;\n> > > > > > > >\n> > > > > > > > const camera_module_callbacks_t *callbacks_;\n> > > > > > > > - std::vector> cameras_;\n> > > > > > > > + std::vector> cameras_;\n> > > > > > > > + std::map camerasMap_;\n> > > > > > >\n> > > > > > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > > > > > camerasMap_ could be merged to a\n> > > > > > >\n> > > > > > > std::map> cameras_;\n> > > > > > >\n> > > > > > > Which would eliminate the possibility of them going out-of-sync.\n> > > > > > >\n> > > > > > > > + Mutex mutex_;\n> > > > > > > > +\n> > > > > > > > + unsigned int externalCameraCounter_;\n> > > > > > > > + unsigned int cameraCounter_;\n> > > > > > > > };\n> > > > > > > >\n> > > > > > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 819C0BE173\n\tfor ;\n\tThu, 20 Aug 2020 07:12:19 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E84CD61F1C;\n\tThu, 20 Aug 2020 09:12:18 +0200 (CEST)","from mail-lj1-x244.google.com (mail-lj1-x244.google.com\n\t[IPv6:2a00:1450:4864:20::244])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6BEDC60380\n\tfor ;\n\tThu, 20 Aug 2020 09:12:17 +0200 (CEST)","by mail-lj1-x244.google.com with SMTP id y2so979228ljc.1\n\tfor ;\n\tThu, 20 Aug 2020 00:12:17 -0700 (PDT)","from localhost (host-78-79-224-122.mobileonline.telia.com.\n\t[78.79.224.122]) by smtp.gmail.com with ESMTPSA id\n\tv10sm300940lfo.11.2020.08.20.00.12.12\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 20 Aug 2020 00:12:15 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ragnatech-se.20150623.gappssmtp.com\n\theader.i=@ragnatech-se.20150623.gappssmtp.com\n\theader.b=\"18CALNNn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ragnatech-se.20150623.gappssmtp.com; s=20150623;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:content-transfer-encoding:in-reply-to;\n\tbh=vORvkRNkcZAhHNQprWVxcJbp8/2d4L3lzB8bFmCNANQ=;\n\tb=18CALNNn2gJfqQMH1O0sugrhpX4TutMBmTTRv++4AT4bus5y0oKmXcwI2aSBPm7lMo\n\toao801F3PESVjOC7jhv0GrviK94qvzxJmtsclct5GjdGTMq0dqr+RlQoTei7tGkLnvXg\n\tJAFuUtVWNVHGhuIRScUbSspsG4HR3hBpkv2F1pRCdyipQHXQpLM5RMWSrAiggzSaqfJV\n\tSS7JQZrru0wr5LLQLa8enexzB7QaJdlj67OZWwBJ6Z1QHvBEwDR2j6B/N7T/ZITsegRV\n\t5OXLATJgNZ4yXCzcUhnvKbt9IDQE4BnjwUfhqGqpyY1bOucooksQDeBfEqqHhoKlpNWF\n\tFRlg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:content-transfer-encoding\n\t:in-reply-to;\n\tbh=vORvkRNkcZAhHNQprWVxcJbp8/2d4L3lzB8bFmCNANQ=;\n\tb=BG4f7g7LHXf5DYtdZ9392pzU6iCLnuzzvkY2ZI3Yx6HjYK+nvz0jEk2J9G0R6cHgRw\n\tRPgGGwFTVfPSxnc8HEoECEmml86giI+joH4g5ehV/0Sex1Srbbv1aJUPk3qoyDE9FBLq\n\tkCtV+6+yf252OZau8Tn3iesheQ/1/lyRLdV16KI8ulreO7523RQxpVKFu7kIJMkz5/78\n\t9OuDLcRsrYX/RxdyxT+SXVAu7dI39SgdgWeFHo9Rfz2quuU2m/87KWQRXsoG9SiR60yP\n\tJN8QnVlCbIgLoHa7tNsfrCNsSBQbTnYKEY7qaNychN7o1Qa+Mkdqf3MPpTt0OQwB2Hbq\n\tXqeQ==","X-Gm-Message-State":"AOAM533vmCq1lPhXhjUOu2SKcg3WzFRiYVH2f1/OrjzDGs1eQnAh/mtx\n\tGeMBMpDlEOyTXR6SKrmiWY82/g==","X-Google-Smtp-Source":"ABdhPJyDoXzlUg839cXRxGBMoGWjqZmPi+kIgxzhhTDIrr2T6Ewxb/OtN76eBf8oDSmqz4r8XqNG9g==","X-Received":"by 2002:a2e:980c:: with SMTP id a12mr864364ljj.113.1597907536249;\n\tThu, 20 Aug 2020 00:12:16 -0700 (PDT)","Date":"Thu, 20 Aug 2020 09:12:10 +0200","From":"Niklas =?iso-8859-1?q?S=F6derlund?= ","To":"Laurent Pinchart ","Message-ID":"<20200820071050.GB39265@wyvern>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t\n\t<20200819162140.GQ6049@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200819162140.GQ6049@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"iso-8859-1\"","Content-Transfer-Encoding":"quoted-printable","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12067,"web_url":"https://patchwork.libcamera.org/comment/12067/","msgid":"<20200820101028.GA6593@pendragon.ideasonboard.com>","date":"2020-08-20T10:10:28","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Niklas,\n\nOn Thu, Aug 20, 2020 at 09:12:10AM +0200, Niklas Söderlund wrote:\n> On 2020-08-19 19:21:40 +0300, Laurent Pinchart wrote:\n> > On Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> > > On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> > > > On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> > > > > On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> > > > > >\n> > > > > > Hi Shik,\n> > > > > >\n> > > > > > Would it be possible for you to have a look at the question below ?\n> > > > > >\n> > > > > > On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > > > > > > Hi Niklas,\n> > > > > > >\n> > > > > > > (CC'ing Shik)\n> > > > > > >\n> > > > > > > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > > > > > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > > > > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > > > > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > > > > > > know about the hotplug events and change the status of cameras being\n> > > > > > > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > > > > > >\n> > > > > > > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > > > > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > > > > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > > > > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > > > > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > > > > > > current implementation, we assume all UVC cameras are external cameras.\n> > > > > > > >\n> > > > > > > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > > > > > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > > > > > > benefit of preserving the same numerical ID between two plug events,\n> > > > > > > > while I can think of quiet a few cons.\n> > > > > > > >\n> > > > > > > > - It's confusing when debugging as un-plugging and then replugging the\n> > > > > > > > same camera will result in logs where the numerical ID is the same for\n> > > > > > > > both. This may even result in things working by \"chance\" is it reuses\n> > > > > > > > an already known numerical ID.\n> > > > > > > >\n> > > > > > > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > > > > > > generate a different Camera::id() result and then be treated as a new\n> > > > > > > > camera vs plugging it in the same port and it then being treatad as a\n> > > > > > > > new camera.\n> > > > > > > >\n> > > > > > > > - The logic in this patch is more complex due to it both having to deal\n> > > > > > > > with known and new cameras.\n> > > > > > > >\n> > > > > > > > What is the benefit of this cache that I'm missing?\n> > > > > > >\n> > > > > > > It's not clear whether Android requires it. It requires different\n> > > > > > > cameras to have different IDs, but I couldn't find a mention of how\n> > > > > > > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > > > > > > ID, I don't know if it's mandatory though.\n> > > > > > >\n> > > > > > > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > > > > > > are requirements to keep the same ID when a camera is unplugged and\n> > > > > > > replugged ?\n> > > > >\n> > > > > It's for persistency in settings UI. Chrome uses the camera id as the key of\n> > > > > the default camera in chrome://settings/content/camera. Reuse the same id for\n> > > > > the same camera so the default camera would be changed back after replug.\n> > > > >\n> > > > > It's introduced in this CL: https://crrev.com/c/1608985\n> > > >\n> > > > Thank you for the information. We have a similar caching system, which\n> > > > however suffers from two limitations:\n> \n> We have more limitations I'm afraid :-(\n> \n> If I understand Shik correctly the numerical camera ID is stored \n> persistently in the camera settings UI. Our code does not guarantee that \n> the numerical ID in the libcamera HAL is persistent. The cameras in the \n> CameraManager is stored in a vector and is therefor susceptible to be \n> enumerated in different order each time the HAL is started.\n> \n> We could of course remedy this by sorting the camera vecotr before \n> assigning numerical IDs. But this would still not be guaranteed to be \n> persistent as more/less cameras registered in the HAL could alter the \n> read out order. If we really wish to do this should we save/load the \n> camera ID to numerical ID mapping to file (not nice)?\n\nShik, does the ID cache need to be persistent across reboots (or\nrestarts of the camera service) ?\n\n> > > > - If a camera is unplugged, and a different camera with the same VID:PID\n> > > > is plugged into the same USB port, the new camera be given the same ID\n> > > > as the previous camera.\n> > > \n> > > The current CrOS implementation will also use the same id in this case, and I\n> > > think it should be fine.\n> > > \n> > > > - If a camera is unplugged and replugged in a different USB port, it\n> > > > will be given a different ID.\n> > > \n> > > Hmm this looks non-ideal and may confuse the users.\n> > \n> > We will try to fix that on top\n> > \n> > This isn't trivial to handle, and there are corner cases. I'm thinking\n> > in particular about a system where two identical external USB cameras\n> > would be set up to point at particular locations. How to handle cameras\n> > stable IDs in that case is debatable, and I don't think the problem can\n> > be solved unless the cameras report distinct serial numbers (and most\n> > webcams don't :-S). At the moment, libcamera identifies each camera with\n> > a stable ID (guaranteed to be unique and stable across reboots, as long\n> > as the hardware and firmware are not modified) made of the concatenation\n> > of\n> > \n> > - The USB controller DT or ACPI path\n> > - The USB port(s) number(s) (starting at the root hub, going through\n> > every hub)\n> > - The USB device VID:PID\n> > \n> > The Android ID is then assigned based on the libcamera ID, with a cache\n> > to ensure that the same HAL ID will be given to the same libcamera ID.\n> > If we were to drop the USB port number, we'd have to find another way to\n> > create a stable and unique camera ID.\n> > \n> > > > Is that acceptable from a Chrome OS point of view ?\n> > > >\n> > > > Would you happen to know how Android uses camera IDs, and whether it\n> > > > requires or could benefit from ID caching ?\n> > > \n> > > Each camera app could have a similar preference caching system like the\n> > > settings UI, so it would be better to reuse the same id for the same camera.\n> > > AFAIK there is no such requirement at the framework level.\n> \n> This worries me a bit. Are we not altering the libcamera HAL to fit a \n> single use-case, all be it an important user. But if the framework does \n> not document an expected behavior is it not tricky to implement \n> correctly? What if we later find a different usage pattern in another \n> application which conflicts with what we are trying to do here?\n\nThis is an undocumented area, so HALs can do pretty much what they want.\nIf different platforms have different requirements, then we'll have to\ndeal with it, possibly working with the platform vendors to align the\nrequirements.\n\n> > > > > > > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > > > > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > > > > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > > > > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > > > > > >\n> > > > > > > > > Signed-off-by: Umang Jain \n> > > > > > > > > ---\n> > > > > > > > > src/android/camera_device.cpp | 15 +++\n> > > > > > > > > src/android/camera_device.h | 8 +-\n> > > > > > > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > > > > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > > > > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > > > > > >\n> > > > > > > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > > > > > > index d918350..a79bb69 100644\n> > > > > > > > > --- a/src/android/camera_device.cpp\n> > > > > > > > > +++ b/src/android/camera_device.cpp\n> > > > > > > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > > > > > > delete it.second;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > > > > > > + const std::shared_ptr &cam)\n> > > > > > > > > +{\n> > > > > > > > > + struct Deleter : std::default_delete {\n> > > > > > > > > + void operator()(CameraDevice *camera)\n> > > > > > > > > + {\n> > > > > > > > > + delete camera;\n> > > > > > > > > + }\n> > > > > > > > > + };\n> > > > > > > > > +\n> > > > > > > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > > > > > > +\n> > > > > > > > > + return std::shared_ptr(camera, Deleter());\n> > > > > > > > > +}\n> > > > > > > >\n> > > > > > > > As Kieran points out I think this should be added in a separate as this\n> > > > > > > > one is quiet large and therefore hard to review.\n> > > > > > > >\n> > > > > > > > > +\n> > > > > > > > > /*\n> > > > > > > > > * Initialize the camera static information.\n> > > > > > > > > * This method is called before the camera device is opened.\n> > > > > > > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > > > > > > index 7be9e11..7f9e010 100644\n> > > > > > > > > --- a/src/android/camera_device.h\n> > > > > > > > > +++ b/src/android/camera_device.h\n> > > > > > > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > > > > > > class CameraDevice : protected libcamera::Loggable\n> > > > > > > > > {\n> > > > > > > > > public:\n> > > > > > > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > > > - ~CameraDevice();\n> > > > > > > > > + static std::shared_ptr create(unsigned int id,\n> > > > > > > > > + const std::shared_ptr &cam);\n> > > > > > > > >\n> > > > > > > > > int initialize();\n> > > > > > > > >\n> > > > > > > > > @@ -57,6 +57,7 @@ public:\n> > > > > > > > >\n> > > > > > > > > unsigned int id() const { return id_; }\n> > > > > > > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > > > > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > > > > > >\n> > > > > > > > Same here this can be done in a separate commit. Also I think this could\n> > > > > > > > be named camera() instead of getCamera()\n> > > > > > > >\n> > > > > > > > >\n> > > > > > > > > int facing() const { return facing_; }\n> > > > > > > > > int orientation() const { return orientation_; }\n> > > > > > > > > @@ -72,6 +73,9 @@ protected:\n> > > > > > > > > std::string logPrefix() const override;\n> > > > > > > > >\n> > > > > > > > > private:\n> > > > > > > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > > > + ~CameraDevice();\n> > > > > > > > > +\n> > > > > > > > > struct Camera3RequestDescriptor {\n> > > > > > > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > > > > > > unsigned int numBuffers);\n> > > > > > > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > > > > > > index 3d6d2b4..fdde2c0 100644\n> > > > > > > > > --- a/src/android/camera_hal_manager.cpp\n> > > > > > > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > > > > > > @@ -8,6 +8,7 @@\n> > > > > > > > > #include \"camera_hal_manager.h\"\n> > > > > > > > >\n> > > > > > > > > #include \n> > > > > > > > > +#include \n> > > > > > > > >\n> > > > > > > > > #include \"libcamera/internal/log.h\"\n> > > > > > > > >\n> > > > > > > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > > > > > > CameraHalManager::~CameraHalManager()\n> > > > > > > > > {\n> > > > > > > > > cameras_.clear();\n> > > > > > > > > + camerasMap_.clear();\n> > > > > > > > >\n> > > > > > > > > if (cameraManager_) {\n> > > > > > > > > cameraManager_->stop();\n> > > > > > > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > > > > > > {\n> > > > > > > > > cameraManager_ = new CameraManager();\n> > > > > > > > >\n> > > > > > > > > + /* Support camera hotplug. */\n> > > > > > > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > > > > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > > > > > > +\n> > > > > > > > > + cameraCounter_ = 0;\n> > > > > > > > > + externalCameraCounter_ = 1000;\n> > > > > > > > > +\n> > > > > > > > > int ret = cameraManager_->start();\n> > > > > > > > > if (ret) {\n> > > > > > > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > > > > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > > > > > > return ret;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > - /*\n> > > > > > > > > - * For each Camera registered in the system, a CameraDevice\n> > > > > > > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > > > > > > - *\n> > > > > > > > > - * \\todo Support camera hotplug.\n> > > > > > > > > - */\n> > > > > > > > > - unsigned int index = 0;\n> > > > > > > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > > > > > > - ret = camera->initialize();\n> > > > > > > > > - if (ret)\n> > > > > > > > > - continue;\n> > > > > > > > > -\n> > > > > > > > > - cameras_.emplace_back(camera);\n> > > > > > > > > - ++index;\n> > > > > > > > > - }\n> > > > > > > > > -\n> > > > > > > > > return 0;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > > > const hw_module_t *hardwareModule)\n> > > > > > > > > {\n> > > > > > > > > - if (id >= numCameras()) {\n> > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > +\n> > > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > > > + return cam->id() == id;\n> > > > > > > > > + });\n> > > > > > > > > + if (iter == cameras_.end()) {\n> > > > > > > >\n> > > > > > > > I think you should break this (and the similar ones below) into private\n> > > > > > > > helpers instead if implementing the logic in-place\n> > > > > > > >\n> > > > > > > > CameraHalManager::cameraFromAndroidId(..);\n> > > > > > > > CameraHalManager::androidIdFromCamera(..);\n> > > > > > > >\n> > > > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > > > return nullptr;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > > > +\n> > > > > > > > > if (camera->open(hardwareModule))\n> > > > > > > > > return nullptr;\n> > > > > > > > >\n> > > > > > > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > > > return camera;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > > > > > > +{\n> > > > > > > > > + unsigned int id;\n> > > > > > > > > + bool isCameraExternal = false;\n> > > > > > > > > + bool isCameraNew = false;\n> > > > > > > > > +\n> > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > +\n> > > > > > > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > > > > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > > > > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > > > > > > + *\n> > > > > > > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > > > > > > + */\n> > > > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > > > + if (iter != camerasMap_.end()) {\n> > > > > > > > > + id = iter->second;\n> > > > > > > > > + } else {\n> > > > > > > > > + isCameraNew = true;\n> > > > > > > > > +\n> > > > > > > > > + /*\n> > > > > > > > > + * Now check if this is an external camera and assign\n> > > > > > > > > + * its id accordingly.\n> > > > > > > > > + */\n> > > > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > > > + properties.get(properties::Location) &\n> > > > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > > > + isCameraExternal = true;\n> > > > > > > > > + id = externalCameraCounter_;\n> > > > > > > > > + } else {\n> > > > > > > > > + id = cameraCounter_;\n> > > > > > > > > + }\n> > > > > > > >\n> > > > > > > > As I understand it the information the HAL wants is whether or not the\n> > > > > > > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > > > > > > >\n> > > > > > > > If so I wonder if using the camera location as a judgment of the camera\n> > > > > > > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > > > > > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > > > > > > a location. I'm thinking cameras mounted at the end of instruments\n> > > > > > > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > > > > > > of a welding robot). I would imagine those cameras would be marked as\n> > > > > > > > located externally but they would not really be hot-pluggable.\n> > > > > > > >\n> > > > > > > > I understand we have no other way to report or detect this at the moment\n> > > > > > > > and I'm not pushing hard for this to be solved as part of this series if\n> > > > > > > > it's not easy. But I think a bigger comment here is needed explaining\n> > > > > > > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > > > > > > not really care if it's located internally or externally. I also think a\n> > > > > > > > \\todo should be added so it's not forgotten.\n> > > > > > > >\n> > > > > > > > > + }\n> > > > > > > > > +\n> > > > > > > > > + /*\n> > > > > > > > > + * For each Camera registered in the system, a CameraDevice\n> > > > > > > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > > > > > > + */\n> > > > > > > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > > > > > > + int ret = camera->initialize();\n> > > > > > > > > + if (ret) {\n> > > > > > > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > > > > > > + return;\n> > > > > > > > > + }\n> > > > > > > > > +\n> > > > > > > > > + if (isCameraNew) {\n> > > > > > > > > + camerasMap_.emplace(cam->id(), id);\n> > > > > > > > > +\n> > > > > > > > > + if (isCameraExternal)\n> > > > > > > > > + externalCameraCounter_++;\n> > > > > > > > > + else\n> > > > > > > > > + cameraCounter_++;\n> > > > > > > > > + }\n> > > > > > > > > +\n> > > > > > > > > + cameras_.emplace_back(std::move(camera));\n> > > > > > > > > +\n> > > > > > > > > + if (callbacks_)\n> > > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > > > > > > +}\n> > > > > > > > > +\n> > > > > > > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > > > > > > +{\n> > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > +\n> > > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > > + [cam](std::shared_ptr &camera) {\n> > > > > > > > > + return cam.get() == camera->getCamera();\n> > > > > > > > > + });\n> > > > > > > > > + if (iter == cameras_.end())\n> > > > > > > > > + return;\n> > > > > > > > > +\n> > > > > > > > > + unsigned int id = (*iter)->id();\n> > > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > > > > > > + cameras_.erase(iter);\n> > > > > > > > > +\n> > > > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > > > > > > +}\n> > > > > > > > > +\n> > > > > > > > > unsigned int CameraHalManager::numCameras() const\n> > > > > > > > > {\n> > > > > > > > > - return cameraManager_->cameras().size();\n> > > > > > > > > + return cameraCounter_;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > > if (!info)\n> > > > > > > > > return -EINVAL;\n> > > > > > > > >\n> > > > > > > > > - if (id >= numCameras()) {\n> > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > +\n> > > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > > > + return cam->id() == id;\n> > > > > > > > > + });\n> > > > > > > > > + if (iter == cameras_.end()) {\n> > > > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > > > return -EINVAL;\n> > > > > > > > > }\n> > > > > > > > >\n> > > > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > > >\n> > > > > > > > > info->facing = camera->facing();\n> > > > > > > > > info->orientation = camera->orientation();\n> > > > > > > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > > > > > > {\n> > > > > > > > > callbacks_ = callbacks;\n> > > > > > > > > +\n> > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > + /*\n> > > > > > > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > > > > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > > > > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > > > > > > + * be present at module load time, by the framework.\n> > > > > > > > > + */\n> > > > > > > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > > > + if (iter == camerasMap_.end())\n> > > > > > > > > + continue;\n> > > > > > > > > +\n> > > > > > > > > + unsigned int id = iter->second;\n> > > > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > > > + properties.get(properties::Location) &\n> > > > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > > > + }\n> > > > > > > > > + }\n> > > > > > > > > }\n> > > > > > > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > > > > > > index a582f04..7c481d4 100644\n> > > > > > > > > --- a/src/android/camera_hal_manager.h\n> > > > > > > > > +++ b/src/android/camera_hal_manager.h\n> > > > > > > > > @@ -7,6 +7,8 @@\n> > > > > > > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > > >\n> > > > > > > > > +#include \n> > > > > > > > > +#include \n> > > > > > > > > #include \n> > > > > > > > > #include \n> > > > > > > > >\n> > > > > > > > > @@ -18,6 +20,9 @@\n> > > > > > > > >\n> > > > > > > > > class CameraDevice;\n> > > > > > > > >\n> > > > > > > > > +using Mutex = std::mutex;\n> > > > > > > > > +using MutexLocker = std::unique_lock;\n> > > > > > > > > +\n> > > > > > > > > class CameraHalManager\n> > > > > > > > > {\n> > > > > > > > > public:\n> > > > > > > > > @@ -33,10 +38,18 @@ public:\n> > > > > > > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > > > > > > >\n> > > > > > > > > private:\n> > > > > > > > > + void cameraAdded(std::shared_ptr cam);\n> > > > > > > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > > > > > > +\n> > > > > > > > > libcamera::CameraManager *cameraManager_;\n> > > > > > > > >\n> > > > > > > > > const camera_module_callbacks_t *callbacks_;\n> > > > > > > > > - std::vector> cameras_;\n> > > > > > > > > + std::vector> cameras_;\n> > > > > > > > > + std::map camerasMap_;\n> > > > > > > >\n> > > > > > > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > > > > > > camerasMap_ could be merged to a\n> > > > > > > >\n> > > > > > > > std::map> cameras_;\n> > > > > > > >\n> > > > > > > > Which would eliminate the possibility of them going out-of-sync.\n> > > > > > > >\n> > > > > > > > > + Mutex mutex_;\n> > > > > > > > > +\n> > > > > > > > > + unsigned int externalCameraCounter_;\n> > > > > > > > > + unsigned int cameraCounter_;\n> > > > > > > > > };\n> > > > > > > > >\n> > > > > > > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 380B3BD87C\n\tfor ;\n\tThu, 20 Aug 2020 10:10:48 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A97D161F61;\n\tThu, 20 Aug 2020 12:10:47 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C95E60381\n\tfor ;\n\tThu, 20 Aug 2020 12:10:46 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9120223D;\n\tThu, 20 Aug 2020 12:10:45 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Kdoox8/g\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597918245;\n\tbh=nEmw18cPjkads7PgtQG4zmfBcjqSI9kgVrKe5mCj/8s=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Kdoox8/gaDPTpic4HWhXBL1hRtDcwuvoSZk6qco+1miskNrPMnG4tAelPoUswt+va\n\tQntJ+iS9pU5GxqDOyN/1ux2cW//L4X9wigVy9A5zHIDG2wwZdyirjsQMyVueyr4eNh\n\tWCuTp0LrmQZiAcPLq9n5SFMdYfKZ4Za5igQXspLU=","Date":"Thu, 20 Aug 2020 13:10:28 +0300","From":"Laurent Pinchart ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Message-ID":"<20200820101028.GA6593@pendragon.ideasonboard.com>","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t\n\t<20200819162140.GQ6049@pendragon.ideasonboard.com>\n\t<20200820071050.GB39265@wyvern>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200820071050.GB39265@wyvern>","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12070,"web_url":"https://patchwork.libcamera.org/comment/12070/","msgid":"","date":"2020-08-20T12:44:31","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":8,"url":"https://patchwork.libcamera.org/api/people/8/","name":"Shik Chen","email":"shik@google.com"},"content":"Hi all,\n\nOn Thu, Aug 20, 2020 at 6:10 PM Laurent Pinchart\n wrote:\n>\n> Hi Niklas,\n>\n> On Thu, Aug 20, 2020 at 09:12:10AM +0200, Niklas Söderlund wrote:\n> > On 2020-08-19 19:21:40 +0300, Laurent Pinchart wrote:\n> > > On Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> > > > On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> > > > > On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> > > > > > On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> > > > > > >\n> > > > > > > Hi Shik,\n> > > > > > >\n> > > > > > > Would it be possible for you to have a look at the question below ?\n> > > > > > >\n> > > > > > > On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > > > > > > > Hi Niklas,\n> > > > > > > >\n> > > > > > > > (CC'ing Shik)\n> > > > > > > >\n> > > > > > > > On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > > > > > > > > On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > > > > > > > > > Extend the support for camera hotplug from libcamera's CameraManager\n> > > > > > > > > > to CameraHalManager. Use camera module callbacks to let the framework\n> > > > > > > > > > know about the hotplug events and change the status of cameras being\n> > > > > > > > > > being hotplugged or unplugged via camera_device_status_change().\n> > > > > > > > > >\n> > > > > > > > > > Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > > > > > > > > > past by the CameraHalManager. If the camera is seen for the first time,\n> > > > > > > > > > a new id is assigned to it. If the camera has been seen before by the\n> > > > > > > > > > manager, it's old id is reused. IDs for internal cameras start with\n> > > > > > > > > > '0' and for external cameras, they start with '1000'. Note, for the\n> > > > > > > > > > current implementation, we assume all UVC cameras are external cameras.\n> > > > > > > > >\n> > > > > > > > > I wonder if keeping the cache of previously seen cameras or if we should\n> > > > > > > > > not just treat any hot-plugged camera as a new one? I can't think of any\n> > > > > > > > > benefit of preserving the same numerical ID between two plug events,\n> > > > > > > > > while I can think of quiet a few cons.\n> > > > > > > > >\n> > > > > > > > > - It's confusing when debugging as un-plugging and then replugging the\n> > > > > > > > > same camera will result in logs where the numerical ID is the same for\n> > > > > > > > > both. This may even result in things working by \"chance\" is it reuses\n> > > > > > > > > an already known numerical ID.\n> > > > > > > > >\n> > > > > > > > > - Looking at the code plugging a UVC camera in a different USB port will\n> > > > > > > > > generate a different Camera::id() result and then be treated as a new\n> > > > > > > > > camera vs plugging it in the same port and it then being treatad as a\n> > > > > > > > > new camera.\n> > > > > > > > >\n> > > > > > > > > - The logic in this patch is more complex due to it both having to deal\n> > > > > > > > > with known and new cameras.\n> > > > > > > > >\n> > > > > > > > > What is the benefit of this cache that I'm missing?\n> > > > > > > >\n> > > > > > > > It's not clear whether Android requires it. It requires different\n> > > > > > > > cameras to have different IDs, but I couldn't find a mention of how\n> > > > > > > > identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > > > > > > > ID, I don't know if it's mandatory though.\n> > > > > > > >\n> > > > > > > > Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > > > > > > > are requirements to keep the same ID when a camera is unplugged and\n> > > > > > > > replugged ?\n> > > > > >\n> > > > > > It's for persistency in settings UI. Chrome uses the camera id as the key of\n> > > > > > the default camera in chrome://settings/content/camera. Reuse the same id for\n> > > > > > the same camera so the default camera would be changed back after replug.\n> > > > > >\n> > > > > > It's introduced in this CL: https://crrev.com/c/1608985\n> > > > >\n> > > > > Thank you for the information. We have a similar caching system, which\n> > > > > however suffers from two limitations:\n> >\n> > We have more limitations I'm afraid :-(\n> >\n> > If I understand Shik correctly the numerical camera ID is stored\n> > persistently in the camera settings UI. Our code does not guarantee that\n> > the numerical ID in the libcamera HAL is persistent. The cameras in the\n> > CameraManager is stored in a vector and is therefor susceptible to be\n> > enumerated in different order each time the HAL is started.\n> >\n> > We could of course remedy this by sorting the camera vecotr before\n> > assigning numerical IDs. But this would still not be guaranteed to be\n> > persistent as more/less cameras registered in the HAL could alter the\n> > read out order. If we really wish to do this should we save/load the\n> > camera ID to numerical ID mapping to file (not nice)?\n>\n> Shik, does the ID cache need to be persistent across reboots (or\n> restarts of the camera service) ?\n\nFor built-in cameras, yes. They need to be persistent across reboots and order\nthe camera ids by facing. There is a common implicit assumption in many\nAndroid camera apps that assume the front camera is 0 and back camera is 1.\n\nFor external cameras, ideally yes. But for most of the use cases currently,\nit's acceptable to use a solution that only persists the ids in a boot cycle,\nand always assign the same id for the first external camera after boots.\n\n>\n> > > > > - If a camera is unplugged, and a different camera with the same VID:PID\n> > > > > is plugged into the same USB port, the new camera be given the same ID\n> > > > > as the previous camera.\n> > > >\n> > > > The current CrOS implementation will also use the same id in this case, and I\n> > > > think it should be fine.\n> > > >\n> > > > > - If a camera is unplugged and replugged in a different USB port, it\n> > > > > will be given a different ID.\n> > > >\n> > > > Hmm this looks non-ideal and may confuse the users.\n> > >\n> > > We will try to fix that on top\n> > >\n> > > This isn't trivial to handle, and there are corner cases. I'm thinking\n> > > in particular about a system where two identical external USB cameras\n> > > would be set up to point at particular locations. How to handle cameras\n> > > stable IDs in that case is debatable, and I don't think the problem can\n> > > be solved unless the cameras report distinct serial numbers (and most\n> > > webcams don't :-S). At the moment, libcamera identifies each camera with\n> > > a stable ID (guaranteed to be unique and stable across reboots, as long\n> > > as the hardware and firmware are not modified) made of the concatenation\n> > > of\n> > >\n> > > - The USB controller DT or ACPI path\n> > > - The USB port(s) number(s) (starting at the root hub, going through\n> > > every hub)\n> > > - The USB device VID:PID\n> > >\n> > > The Android ID is then assigned based on the libcamera ID, with a cache\n> > > to ensure that the same HAL ID will be given to the same libcamera ID.\n> > > If we were to drop the USB port number, we'd have to find another way to\n> > > create a stable and unique camera ID.\n> > >\n> > > > > Is that acceptable from a Chrome OS point of view ?\n> > > > >\n> > > > > Would you happen to know how Android uses camera IDs, and whether it\n> > > > > requires or could benefit from ID caching ?\n> > > >\n> > > > Each camera app could have a similar preference caching system like the\n> > > > settings UI, so it would be better to reuse the same id for the same camera.\n> > > > AFAIK there is no such requirement at the framework level.\n> >\n> > This worries me a bit. Are we not altering the libcamera HAL to fit a\n> > single use-case, all be it an important user. But if the framework does\n> > not document an expected behavior is it not tricky to implement\n> > correctly? What if we later find a different usage pattern in another\n> > application which conflicts with what we are trying to do here?\n>\n> This is an undocumented area, so HALs can do pretty much what they want.\n> If different platforms have different requirements, then we'll have to\n> deal with it, possibly working with the platform vendors to align the\n> requirements.\n\nYes and we need to live with Hyrum's Law :P\nIf it's not handled here, we would still need to have another id mapping layer\nsomewhere else.\n\n>\n> > > > > > > > > > CameraDevice is now a shared object and cameras_ vector stores shared\n> > > > > > > > > > pointers to CameraDevice. This is done in order to introduce reference\n> > > > > > > > > > counting for CameraDevice objects - especially to handle hot-unplug\n> > > > > > > > > > events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > > > > > > > > >\n> > > > > > > > > > Signed-off-by: Umang Jain \n> > > > > > > > > > ---\n> > > > > > > > > > src/android/camera_device.cpp | 15 +++\n> > > > > > > > > > src/android/camera_device.h | 8 +-\n> > > > > > > > > > src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > > > > > > > > > src/android/camera_hal_manager.h | 15 ++-\n> > > > > > > > > > 4 files changed, 166 insertions(+), 25 deletions(-)\n> > > > > > > > > >\n> > > > > > > > > > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > > > > > > > > > index d918350..a79bb69 100644\n> > > > > > > > > > --- a/src/android/camera_device.cpp\n> > > > > > > > > > +++ b/src/android/camera_device.cpp\n> > > > > > > > > > @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > > > > > > > > > delete it.second;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > +std::shared_ptr CameraDevice::create(unsigned int id,\n> > > > > > > > > > + const std::shared_ptr &cam)\n> > > > > > > > > > +{\n> > > > > > > > > > + struct Deleter : std::default_delete {\n> > > > > > > > > > + void operator()(CameraDevice *camera)\n> > > > > > > > > > + {\n> > > > > > > > > > + delete camera;\n> > > > > > > > > > + }\n> > > > > > > > > > + };\n> > > > > > > > > > +\n> > > > > > > > > > + CameraDevice *camera = new CameraDevice(id, cam);\n> > > > > > > > > > +\n> > > > > > > > > > + return std::shared_ptr(camera, Deleter());\n> > > > > > > > > > +}\n> > > > > > > > >\n> > > > > > > > > As Kieran points out I think this should be added in a separate as this\n> > > > > > > > > one is quiet large and therefore hard to review.\n> > > > > > > > >\n> > > > > > > > > > +\n> > > > > > > > > > /*\n> > > > > > > > > > * Initialize the camera static information.\n> > > > > > > > > > * This method is called before the camera device is opened.\n> > > > > > > > > > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > > > > > > > > > index 7be9e11..7f9e010 100644\n> > > > > > > > > > --- a/src/android/camera_device.h\n> > > > > > > > > > +++ b/src/android/camera_device.h\n> > > > > > > > > > @@ -47,8 +47,8 @@ struct CameraStream {\n> > > > > > > > > > class CameraDevice : protected libcamera::Loggable\n> > > > > > > > > > {\n> > > > > > > > > > public:\n> > > > > > > > > > - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > > > > - ~CameraDevice();\n> > > > > > > > > > + static std::shared_ptr create(unsigned int id,\n> > > > > > > > > > + const std::shared_ptr &cam);\n> > > > > > > > > >\n> > > > > > > > > > int initialize();\n> > > > > > > > > >\n> > > > > > > > > > @@ -57,6 +57,7 @@ public:\n> > > > > > > > > >\n> > > > > > > > > > unsigned int id() const { return id_; }\n> > > > > > > > > > camera3_device_t *camera3Device() { return &camera3Device_; }\n> > > > > > > > > > + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > > > > > > > >\n> > > > > > > > > Same here this can be done in a separate commit. Also I think this could\n> > > > > > > > > be named camera() instead of getCamera()\n> > > > > > > > >\n> > > > > > > > > >\n> > > > > > > > > > int facing() const { return facing_; }\n> > > > > > > > > > int orientation() const { return orientation_; }\n> > > > > > > > > > @@ -72,6 +73,9 @@ protected:\n> > > > > > > > > > std::string logPrefix() const override;\n> > > > > > > > > >\n> > > > > > > > > > private:\n> > > > > > > > > > + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > > > > > > > > > + ~CameraDevice();\n> > > > > > > > > > +\n> > > > > > > > > > struct Camera3RequestDescriptor {\n> > > > > > > > > > Camera3RequestDescriptor(unsigned int frameNumber,\n> > > > > > > > > > unsigned int numBuffers);\n> > > > > > > > > > diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > > > > > > > > > index 3d6d2b4..fdde2c0 100644\n> > > > > > > > > > --- a/src/android/camera_hal_manager.cpp\n> > > > > > > > > > +++ b/src/android/camera_hal_manager.cpp\n> > > > > > > > > > @@ -8,6 +8,7 @@\n> > > > > > > > > > #include \"camera_hal_manager.h\"\n> > > > > > > > > >\n> > > > > > > > > > #include \n> > > > > > > > > > +#include \n> > > > > > > > > >\n> > > > > > > > > > #include \"libcamera/internal/log.h\"\n> > > > > > > > > >\n> > > > > > > > > > @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > > > > > > > > > CameraHalManager::~CameraHalManager()\n> > > > > > > > > > {\n> > > > > > > > > > cameras_.clear();\n> > > > > > > > > > + camerasMap_.clear();\n> > > > > > > > > >\n> > > > > > > > > > if (cameraManager_) {\n> > > > > > > > > > cameraManager_->stop();\n> > > > > > > > > > @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > > > > > > > > > {\n> > > > > > > > > > cameraManager_ = new CameraManager();\n> > > > > > > > > >\n> > > > > > > > > > + /* Support camera hotplug. */\n> > > > > > > > > > + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > > > > > > > > > + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > > > > > > > > > +\n> > > > > > > > > > + cameraCounter_ = 0;\n> > > > > > > > > > + externalCameraCounter_ = 1000;\n> > > > > > > > > > +\n> > > > > > > > > > int ret = cameraManager_->start();\n> > > > > > > > > > if (ret) {\n> > > > > > > > > > LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > > > > > > > > > @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > > > > > > > > > return ret;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > - /*\n> > > > > > > > > > - * For each Camera registered in the system, a CameraDevice\n> > > > > > > > > > - * gets created here to wraps a libcamera Camera instance.\n> > > > > > > > > > - *\n> > > > > > > > > > - * \\todo Support camera hotplug.\n> > > > > > > > > > - */\n> > > > > > > > > > - unsigned int index = 0;\n> > > > > > > > > > - for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > > > > - CameraDevice *camera = new CameraDevice(index, cam);\n> > > > > > > > > > - ret = camera->initialize();\n> > > > > > > > > > - if (ret)\n> > > > > > > > > > - continue;\n> > > > > > > > > > -\n> > > > > > > > > > - cameras_.emplace_back(camera);\n> > > > > > > > > > - ++index;\n> > > > > > > > > > - }\n> > > > > > > > > > -\n> > > > > > > > > > return 0;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > > > > const hw_module_t *hardwareModule)\n> > > > > > > > > > {\n> > > > > > > > > > - if (id >= numCameras()) {\n> > > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > > +\n> > > > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > > > > + return cam->id() == id;\n> > > > > > > > > > + });\n> > > > > > > > > > + if (iter == cameras_.end()) {\n> > > > > > > > >\n> > > > > > > > > I think you should break this (and the similar ones below) into private\n> > > > > > > > > helpers instead if implementing the logic in-place\n> > > > > > > > >\n> > > > > > > > > CameraHalManager::cameraFromAndroidId(..);\n> > > > > > > > > CameraHalManager::androidIdFromCamera(..);\n> > > > > > > > >\n> > > > > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > > > > return nullptr;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > > > > +\n> > > > > > > > > > if (camera->open(hardwareModule))\n> > > > > > > > > > return nullptr;\n> > > > > > > > > >\n> > > > > > > > > > @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > > > > > > > > > return camera;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > > > > > > > > > +{\n> > > > > > > > > > + unsigned int id;\n> > > > > > > > > > + bool isCameraExternal = false;\n> > > > > > > > > > + bool isCameraNew = false;\n> > > > > > > > > > +\n> > > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > > +\n> > > > > > > > > > + /* Each camera is assigned a unique integer id when it is seen for the\n> > > > > > > > > > + * first time. If the camera has been seen before, the id is reused and\n> > > > > > > > > > + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > > > > > > > > > + *\n> > > > > > > > > > + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > > > > > > > > > + */\n> > > > > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > > > > + if (iter != camerasMap_.end()) {\n> > > > > > > > > > + id = iter->second;\n> > > > > > > > > > + } else {\n> > > > > > > > > > + isCameraNew = true;\n> > > > > > > > > > +\n> > > > > > > > > > + /*\n> > > > > > > > > > + * Now check if this is an external camera and assign\n> > > > > > > > > > + * its id accordingly.\n> > > > > > > > > > + */\n> > > > > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > > > > + properties.get(properties::Location) &\n> > > > > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > > > > + isCameraExternal = true;\n> > > > > > > > > > + id = externalCameraCounter_;\n> > > > > > > > > > + } else {\n> > > > > > > > > > + id = cameraCounter_;\n> > > > > > > > > > + }\n> > > > > > > > >\n> > > > > > > > > As I understand it the information the HAL wants is whether or not the\n> > > > > > > > > camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > > > > > > > >\n> > > > > > > > > If so I wonder if using the camera location as a judgment of the camera\n> > > > > > > > > is hot-plugged or not is the way to go here? Imagine a device where the\n> > > > > > > > > camera is permanently attached (not hot-unpluggable) but not fixated in\n> > > > > > > > > a location. I'm thinking cameras mounted at the end of instruments\n> > > > > > > > > (medical instruments, hand held tools) or robotics (mounted at the arm\n> > > > > > > > > of a welding robot). I would imagine those cameras would be marked as\n> > > > > > > > > located externally but they would not really be hot-pluggable.\n> > > > > > > > >\n> > > > > > > > > I understand we have no other way to report or detect this at the moment\n> > > > > > > > > and I'm not pushing hard for this to be solved as part of this series if\n> > > > > > > > > it's not easy. But I think a bigger comment here is needed explaining\n> > > > > > > > > that the HAL wants to know if a camera is hot-pluggable or not and does\n> > > > > > > > > not really care if it's located internally or externally. I also think a\n> > > > > > > > > \\todo should be added so it's not forgotten.\n> > > > > > > > >\n> > > > > > > > > > + }\n> > > > > > > > > > +\n> > > > > > > > > > + /*\n> > > > > > > > > > + * For each Camera registered in the system, a CameraDevice\n> > > > > > > > > > + * gets created here to wraps a libcamera Camera instance.\n> > > > > > > > > > + */\n> > > > > > > > > > + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > > > > > > > > > + int ret = camera->initialize();\n> > > > > > > > > > + if (ret) {\n> > > > > > > > > > + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > > > > > > > > > + return;\n> > > > > > > > > > + }\n> > > > > > > > > > +\n> > > > > > > > > > + if (isCameraNew) {\n> > > > > > > > > > + camerasMap_.emplace(cam->id(), id);\n> > > > > > > > > > +\n> > > > > > > > > > + if (isCameraExternal)\n> > > > > > > > > > + externalCameraCounter_++;\n> > > > > > > > > > + else\n> > > > > > > > > > + cameraCounter_++;\n> > > > > > > > > > + }\n> > > > > > > > > > +\n> > > > > > > > > > + cameras_.emplace_back(std::move(camera));\n> > > > > > > > > > +\n> > > > > > > > > > + if (callbacks_)\n> > > > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > > > > > > > > > +}\n> > > > > > > > > > +\n> > > > > > > > > > +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > > > > > > > > > +{\n> > > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > > +\n> > > > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > > > + [cam](std::shared_ptr &camera) {\n> > > > > > > > > > + return cam.get() == camera->getCamera();\n> > > > > > > > > > + });\n> > > > > > > > > > + if (iter == cameras_.end())\n> > > > > > > > > > + return;\n> > > > > > > > > > +\n> > > > > > > > > > + unsigned int id = (*iter)->id();\n> > > > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > > > + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > > > > > > > > > + cameras_.erase(iter);\n> > > > > > > > > > +\n> > > > > > > > > > + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > > > > > > > > > +}\n> > > > > > > > > > +\n> > > > > > > > > > unsigned int CameraHalManager::numCameras() const\n> > > > > > > > > > {\n> > > > > > > > > > - return cameraManager_->cameras().size();\n> > > > > > > > > > + return cameraCounter_;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > > > @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > > > if (!info)\n> > > > > > > > > > return -EINVAL;\n> > > > > > > > > >\n> > > > > > > > > > - if (id >= numCameras()) {\n> > > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > > +\n> > > > > > > > > > + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > > > > > > > > > + [id](std::shared_ptr &cam) {\n> > > > > > > > > > + return cam->id() == id;\n> > > > > > > > > > + });\n> > > > > > > > > > + if (iter == cameras_.end()) {\n> > > > > > > > > > LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > > > > > > > > > return -EINVAL;\n> > > > > > > > > > }\n> > > > > > > > > >\n> > > > > > > > > > - CameraDevice *camera = cameras_[id].get();\n> > > > > > > > > > + CameraDevice *camera = iter->get();\n> > > > > > > > > >\n> > > > > > > > > > info->facing = camera->facing();\n> > > > > > > > > > info->orientation = camera->orientation();\n> > > > > > > > > > @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > > > > > > > > > void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > > > > > > > > > {\n> > > > > > > > > > callbacks_ = callbacks;\n> > > > > > > > > > +\n> > > > > > > > > > + MutexLocker locker(mutex_);\n> > > > > > > > > > + /*\n> > > > > > > > > > + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > > > > > > > > > + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > > > > > > > > > + * This hold only for external cameras, as internal cameras are assumed to\n> > > > > > > > > > + * be present at module load time, by the framework.\n> > > > > > > > > > + */\n> > > > > > > > > > + for (auto &cam : cameraManager_->cameras()) {\n> > > > > > > > > > + auto iter = camerasMap_.find(cam->id());\n> > > > > > > > > > + if (iter == camerasMap_.end())\n> > > > > > > > > > + continue;\n> > > > > > > > > > +\n> > > > > > > > > > + unsigned int id = iter->second;\n> > > > > > > > > > + const ControlList &properties = cam->properties();\n> > > > > > > > > > + if (properties.contains(properties::Location) &&\n> > > > > > > > > > + properties.get(properties::Location) &\n> > > > > > > > > > + properties::CameraLocationExternal) {\n> > > > > > > > > > + callbacks_->camera_device_status_change(callbacks_, id,\n> > > > > > > > > > + CAMERA_DEVICE_STATUS_PRESENT);\n> > > > > > > > > > + }\n> > > > > > > > > > + }\n> > > > > > > > > > }\n> > > > > > > > > > diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > > > > > > > > > index a582f04..7c481d4 100644\n> > > > > > > > > > --- a/src/android/camera_hal_manager.h\n> > > > > > > > > > +++ b/src/android/camera_hal_manager.h\n> > > > > > > > > > @@ -7,6 +7,8 @@\n> > > > > > > > > > #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > > > > #define __ANDROID_CAMERA_MANAGER_H__\n> > > > > > > > > >\n> > > > > > > > > > +#include \n> > > > > > > > > > +#include \n> > > > > > > > > > #include \n> > > > > > > > > > #include \n> > > > > > > > > >\n> > > > > > > > > > @@ -18,6 +20,9 @@\n> > > > > > > > > >\n> > > > > > > > > > class CameraDevice;\n> > > > > > > > > >\n> > > > > > > > > > +using Mutex = std::mutex;\n> > > > > > > > > > +using MutexLocker = std::unique_lock;\n> > > > > > > > > > +\n> > > > > > > > > > class CameraHalManager\n> > > > > > > > > > {\n> > > > > > > > > > public:\n> > > > > > > > > > @@ -33,10 +38,18 @@ public:\n> > > > > > > > > > void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > > > > > > > > >\n> > > > > > > > > > private:\n> > > > > > > > > > + void cameraAdded(std::shared_ptr cam);\n> > > > > > > > > > + void cameraRemoved(std::shared_ptr cam);\n> > > > > > > > > > +\n> > > > > > > > > > libcamera::CameraManager *cameraManager_;\n> > > > > > > > > >\n> > > > > > > > > > const camera_module_callbacks_t *callbacks_;\n> > > > > > > > > > - std::vector> cameras_;\n> > > > > > > > > > + std::vector> cameras_;\n> > > > > > > > > > + std::map camerasMap_;\n> > > > > > > > >\n> > > > > > > > > If each hot-plugged camera where treated as a new camera cameras_ and\n> > > > > > > > > camerasMap_ could be merged to a\n> > > > > > > > >\n> > > > > > > > > std::map> cameras_;\n> > > > > > > > >\n> > > > > > > > > Which would eliminate the possibility of them going out-of-sync.\n> > > > > > > > >\n> > > > > > > > > > + Mutex mutex_;\n> > > > > > > > > > +\n> > > > > > > > > > + unsigned int externalCameraCounter_;\n> > > > > > > > > > + unsigned int cameraCounter_;\n> > > > > > > > > > };\n> > > > > > > > > >\n> > > > > > > > > > #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n\n- shik","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 22A59BD87C\n\tfor ;\n\tThu, 20 Aug 2020 12:44:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AB8D461F0F;\n\tThu, 20 Aug 2020 14:44:52 +0200 (CEST)","from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com\n\t[IPv6:2607:f8b0:4864:20::72b])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DF47960381\n\tfor ;\n\tThu, 20 Aug 2020 14:44:50 +0200 (CEST)","by mail-qk1-x72b.google.com with SMTP id 62so1350077qkj.7\n\tfor ;\n\tThu, 20 Aug 2020 05:44:50 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"lVXtBLD7\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=20161025; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=r9YVrYSmrOL2tcbDnIKDGfM/p6yIsX3WV/vzWqhXXsU=;\n\tb=lVXtBLD7zlZoyIwOmQei3/+2wn+RbuvwsN9ChFzE/tc8mANandnY4V2FVEMDgyfglO\n\tpf3cG9IykeC0v147hW1WKliXFCxkRzjE212h71h2ytCXlBjWzBMGaG6bCUdaTu21n7iJ\n\t5WJRcrofMGI6A3EyBcw88K0ETb5q7oYYmqZq4DfWuu6hd4rW8UVwNBMMRmCMILwoSlU7\n\tX+mK5lBc2djGLarc2HClWA+I3ezSqaMHQWWGCfE2Kn7t1rO/xCEPDOxz18FEQNysem5e\n\tiuoTWKjTdwexQ3FkwWlBBuHRjZueSZIsMAo7+mIvruUkoR30o+Sz+83P+GS6F29FhfFt\n\tvfPA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=r9YVrYSmrOL2tcbDnIKDGfM/p6yIsX3WV/vzWqhXXsU=;\n\tb=lXQrqjfJhhbEVK+S6Rolh1hdrLZxX4nH7EDvqRZkr8T5NL1yFNpvh0uyF/k8fLmk+A\n\tSnN6oAL6I6M6b8NAbeerFLRr5cFBi2lto/kW5PW0ccI4D72sgERRc1jzZkHha6Il/sZk\n\tO6dgoLcvsRypo8JsayMK8zcxu0TmQtBqnNMy/hXOU2wF2SemQKe8RvCdtA1KWdk+qxfa\n\t3mmNrpaRV6zNXGgPizcQ03gmgFirWTiuodCE+lCP+7RkOUNko4uNcg2d7XEi4mdXmW6Z\n\tXaXHlnZr95r+XUGB0CtLuIuQGuD+PA5LCyeUb781Wwut5Nbh1w3OyVJkZzRHd/JU21ia\n\tps2Q==","X-Gm-Message-State":"AOAM5337dnprgbu5I5Wa0CEgPsvO22SlRnRnPVnN9rzJFIQjHfJZy3/o\n\tAP/1JbpH11w8UwMQ8LSyawY3LSc/vkw3so68GyP8wQ==","X-Google-Smtp-Source":"ABdhPJwEb9XIEV1j2JTV61m3fUVYTX3lA/4QvuqFQL2YQkjans7IXK8jQUO7bszCB+TruAG+ZTCFbTmPbylPVUvhKPk=","X-Received":"by 2002:a37:556:: with SMTP id 83mr2381345qkf.208.1597927488683; \n\tThu, 20 Aug 2020 05:44:48 -0700 (PDT)","MIME-Version":"1.0","References":"<20200810120406.52654-1-email@uajain.com>\n\t<20200810120406.52654-4-email@uajain.com>\n\t<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t\n\t<20200819162140.GQ6049@pendragon.ideasonboard.com>\n\t<20200820071050.GB39265@wyvern>\n\t<20200820101028.GA6593@pendragon.ideasonboard.com>","In-Reply-To":"<20200820101028.GA6593@pendragon.ideasonboard.com>","From":"Shik Chen ","Date":"Thu, 20 Aug 2020 20:44:31 +0800","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12071,"web_url":"https://patchwork.libcamera.org/comment/12071/","msgid":"<20200820144215.GH6593@pendragon.ideasonboard.com>","date":"2020-08-20T14:42:15","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Shik,\n\nOn Thu, Aug 20, 2020 at 08:44:31PM +0800, Shik Chen wrote:\n> On Thu, Aug 20, 2020 at 6:10 PM Laurent Pinchart wrote:\n> > On Thu, Aug 20, 2020 at 09:12:10AM +0200, Niklas Söderlund wrote:\n> >> On 2020-08-19 19:21:40 +0300, Laurent Pinchart wrote:\n> >>> On Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> >>>> On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> >>>>> On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> >>>>>> On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> >>>>>>>\n> >>>>>>> Hi Shik,\n> >>>>>>>\n> >>>>>>> Would it be possible for you to have a look at the question below ?\n> >>>>>>>\n> >>>>>>> On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> >>>>>>>> Hi Niklas,\n> >>>>>>>>\n> >>>>>>>> (CC'ing Shik)\n> >>>>>>>>\n> >>>>>>>> On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> >>>>>>>>> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> >>>>>>>>>> Extend the support for camera hotplug from libcamera's CameraManager\n> >>>>>>>>>> to CameraHalManager. Use camera module callbacks to let the framework\n> >>>>>>>>>> know about the hotplug events and change the status of cameras being\n> >>>>>>>>>> being hotplugged or unplugged via camera_device_status_change().\n> >>>>>>>>>>\n> >>>>>>>>>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> >>>>>>>>>> past by the CameraHalManager. If the camera is seen for the first time,\n> >>>>>>>>>> a new id is assigned to it. If the camera has been seen before by the\n> >>>>>>>>>> manager, it's old id is reused. IDs for internal cameras start with\n> >>>>>>>>>> '0' and for external cameras, they start with '1000'. Note, for the\n> >>>>>>>>>> current implementation, we assume all UVC cameras are external cameras.\n> >>>>>>>>>\n> >>>>>>>>> I wonder if keeping the cache of previously seen cameras or if we should\n> >>>>>>>>> not just treat any hot-plugged camera as a new one? I can't think of any\n> >>>>>>>>> benefit of preserving the same numerical ID between two plug events,\n> >>>>>>>>> while I can think of quiet a few cons.\n> >>>>>>>>>\n> >>>>>>>>> - It's confusing when debugging as un-plugging and then replugging the\n> >>>>>>>>> same camera will result in logs where the numerical ID is the same for\n> >>>>>>>>> both. This may even result in things working by \"chance\" is it reuses\n> >>>>>>>>> an already known numerical ID.\n> >>>>>>>>>\n> >>>>>>>>> - Looking at the code plugging a UVC camera in a different USB port will\n> >>>>>>>>> generate a different Camera::id() result and then be treated as a new\n> >>>>>>>>> camera vs plugging it in the same port and it then being treatad as a\n> >>>>>>>>> new camera.\n> >>>>>>>>>\n> >>>>>>>>> - The logic in this patch is more complex due to it both having to deal\n> >>>>>>>>> with known and new cameras.\n> >>>>>>>>>\n> >>>>>>>>> What is the benefit of this cache that I'm missing?\n> >>>>>>>>\n> >>>>>>>> It's not clear whether Android requires it. It requires different\n> >>>>>>>> cameras to have different IDs, but I couldn't find a mention of how\n> >>>>>>>> identical cameras should be treated. The Chrome OS UVC HAL caches the\n> >>>>>>>> ID, I don't know if it's mandatory though.\n> >>>>>>>>\n> >>>>>>>> Shik, as you've worked on the UVC HAL, would you happen to know if there\n> >>>>>>>> are requirements to keep the same ID when a camera is unplugged and\n> >>>>>>>> replugged ?\n> >>>>>>\n> >>>>>> It's for persistency in settings UI. Chrome uses the camera id as the key of\n> >>>>>> the default camera in chrome://settings/content/camera. Reuse the same id for\n> >>>>>> the same camera so the default camera would be changed back after replug.\n> >>>>>>\n> >>>>>> It's introduced in this CL: https://crrev.com/c/1608985\n> >>>>>\n> >>>>> Thank you for the information. We have a similar caching system, which\n> >>>>> however suffers from two limitations:\n> >>\n> >> We have more limitations I'm afraid :-(\n> >>\n> >> If I understand Shik correctly the numerical camera ID is stored\n> >> persistently in the camera settings UI. Our code does not guarantee that\n> >> the numerical ID in the libcamera HAL is persistent. The cameras in the\n> >> CameraManager is stored in a vector and is therefor susceptible to be\n> >> enumerated in different order each time the HAL is started.\n> >>\n> >> We could of course remedy this by sorting the camera vecotr before\n> >> assigning numerical IDs. But this would still not be guaranteed to be\n> >> persistent as more/less cameras registered in the HAL could alter the\n> >> read out order. If we really wish to do this should we save/load the\n> >> camera ID to numerical ID mapping to file (not nice)?\n> >\n> > Shik, does the ID cache need to be persistent across reboots (or\n> > restarts of the camera service) ?\n> \n> For built-in cameras, yes. They need to be persistent across reboots and order\n> the camera ids by facing. There is a common implicit assumption in many\n> Android camera apps that assume the front camera is 0 and back camera is 1.\n\nOne may wish this would be documented in the HAL API ;-)\n\n> For external cameras, ideally yes. But for most of the use cases currently,\n> it's acceptable to use a solution that only persists the ids in a boot cycle,\n> and always assign the same id for the first external camera after boots.\n\nThere are conflicting requirements there. We can't assign the same ID\nfor the first external camera after boot (we use 1000 today) and also\nhave stable IDs across boots.\n\n> >>>>> - If a camera is unplugged, and a different camera with the same VID:PID\n> >>>>> is plugged into the same USB port, the new camera be given the same ID\n> >>>>> as the previous camera.\n> >>>>\n> >>>> The current CrOS implementation will also use the same id in this case, and I\n> >>>> think it should be fine.\n> >>>>\n> >>>>> - If a camera is unplugged and replugged in a different USB port, it\n> >>>>> will be given a different ID.\n> >>>>\n> >>>> Hmm this looks non-ideal and may confuse the users.\n> >>>\n> >>> We will try to fix that on top\n> >>>\n> >>> This isn't trivial to handle, and there are corner cases. I'm thinking\n> >>> in particular about a system where two identical external USB cameras\n> >>> would be set up to point at particular locations. How to handle cameras\n> >>> stable IDs in that case is debatable, and I don't think the problem can\n> >>> be solved unless the cameras report distinct serial numbers (and most\n> >>> webcams don't :-S). At the moment, libcamera identifies each camera with\n> >>> a stable ID (guaranteed to be unique and stable across reboots, as long\n> >>> as the hardware and firmware are not modified) made of the concatenation\n> >>> of\n> >>>\n> >>> - The USB controller DT or ACPI path\n> >>> - The USB port(s) number(s) (starting at the root hub, going through\n> >>> every hub)\n> >>> - The USB device VID:PID\n> >>>\n> >>> The Android ID is then assigned based on the libcamera ID, with a cache\n> >>> to ensure that the same HAL ID will be given to the same libcamera ID.\n> >>> If we were to drop the USB port number, we'd have to find another way to\n> >>> create a stable and unique camera ID.\n> >>>\n> >>>>> Is that acceptable from a Chrome OS point of view ?\n> >>>>>\n> >>>>> Would you happen to know how Android uses camera IDs, and whether it\n> >>>>> requires or could benefit from ID caching ?\n> >>>>\n> >>>> Each camera app could have a similar preference caching system like the\n> >>>> settings UI, so it would be better to reuse the same id for the same camera.\n> >>>> AFAIK there is no such requirement at the framework level.\n> >>\n> >> This worries me a bit. Are we not altering the libcamera HAL to fit a\n> >> single use-case, all be it an important user. But if the framework does\n> >> not document an expected behavior is it not tricky to implement\n> >> correctly? What if we later find a different usage pattern in another\n> >> application which conflicts with what we are trying to do here?\n> >\n> > This is an undocumented area, so HALs can do pretty much what they want.\n> > If different platforms have different requirements, then we'll have to\n> > deal with it, possibly working with the platform vendors to align the\n> > requirements.\n> \n> Yes and we need to live with Hyrum's Law :P\n> If it's not handled here, we would still need to have another id mapping layer\n> somewhere else.\n\nThat may be needed down the line though. As Chrome OS uses the Android\ncamera HAL API, if differences in requirements are found between the two\nsystem, supporting the Chrome OS requirements in the Chrome OS camera\nservice, on top of the HAL, may be the best option. We'll see when we'll\nrun into actual differences.\n\n> >>>>>>>>>> CameraDevice is now a shared object and cameras_ vector stores shared\n> >>>>>>>>>> pointers to CameraDevice. This is done in order to introduce reference\n> >>>>>>>>>> counting for CameraDevice objects - especially to handle hot-unplug\n> >>>>>>>>>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n> >>>>>>>>>>\n> >>>>>>>>>> Signed-off-by: Umang Jain \n> >>>>>>>>>> ---\n> >>>>>>>>>> src/android/camera_device.cpp | 15 +++\n> >>>>>>>>>> src/android/camera_device.h | 8 +-\n> >>>>>>>>>> src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> >>>>>>>>>> src/android/camera_hal_manager.h | 15 ++-\n> >>>>>>>>>> 4 files changed, 166 insertions(+), 25 deletions(-)\n> >>>>>>>>>>\n> >>>>>>>>>> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> >>>>>>>>>> index d918350..a79bb69 100644\n> >>>>>>>>>> --- a/src/android/camera_device.cpp\n> >>>>>>>>>> +++ b/src/android/camera_device.cpp\n> >>>>>>>>>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> >>>>>>>>>> delete it.second;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> +std::shared_ptr CameraDevice::create(unsigned int id,\n> >>>>>>>>>> + const std::shared_ptr &cam)\n> >>>>>>>>>> +{\n> >>>>>>>>>> + struct Deleter : std::default_delete {\n> >>>>>>>>>> + void operator()(CameraDevice *camera)\n> >>>>>>>>>> + {\n> >>>>>>>>>> + delete camera;\n> >>>>>>>>>> + }\n> >>>>>>>>>> + };\n> >>>>>>>>>> +\n> >>>>>>>>>> + CameraDevice *camera = new CameraDevice(id, cam);\n> >>>>>>>>>> +\n> >>>>>>>>>> + return std::shared_ptr(camera, Deleter());\n> >>>>>>>>>> +}\n> >>>>>>>>>\n> >>>>>>>>> As Kieran points out I think this should be added in a separate as this\n> >>>>>>>>> one is quiet large and therefore hard to review.\n> >>>>>>>>>\n> >>>>>>>>>> +\n> >>>>>>>>>> /*\n> >>>>>>>>>> * Initialize the camera static information.\n> >>>>>>>>>> * This method is called before the camera device is opened.\n> >>>>>>>>>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> >>>>>>>>>> index 7be9e11..7f9e010 100644\n> >>>>>>>>>> --- a/src/android/camera_device.h\n> >>>>>>>>>> +++ b/src/android/camera_device.h\n> >>>>>>>>>> @@ -47,8 +47,8 @@ struct CameraStream {\n> >>>>>>>>>> class CameraDevice : protected libcamera::Loggable\n> >>>>>>>>>> {\n> >>>>>>>>>> public:\n> >>>>>>>>>> - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> >>>>>>>>>> - ~CameraDevice();\n> >>>>>>>>>> + static std::shared_ptr create(unsigned int id,\n> >>>>>>>>>> + const std::shared_ptr &cam);\n> >>>>>>>>>>\n> >>>>>>>>>> int initialize();\n> >>>>>>>>>>\n> >>>>>>>>>> @@ -57,6 +57,7 @@ public:\n> >>>>>>>>>>\n> >>>>>>>>>> unsigned int id() const { return id_; }\n> >>>>>>>>>> camera3_device_t *camera3Device() { return &camera3Device_; }\n> >>>>>>>>>> + const libcamera::Camera *getCamera() { return camera_.get(); };\n> >>>>>>>>>\n> >>>>>>>>> Same here this can be done in a separate commit. Also I think this could\n> >>>>>>>>> be named camera() instead of getCamera()\n> >>>>>>>>>\n> >>>>>>>>>>\n> >>>>>>>>>> int facing() const { return facing_; }\n> >>>>>>>>>> int orientation() const { return orientation_; }\n> >>>>>>>>>> @@ -72,6 +73,9 @@ protected:\n> >>>>>>>>>> std::string logPrefix() const override;\n> >>>>>>>>>>\n> >>>>>>>>>> private:\n> >>>>>>>>>> + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> >>>>>>>>>> + ~CameraDevice();\n> >>>>>>>>>> +\n> >>>>>>>>>> struct Camera3RequestDescriptor {\n> >>>>>>>>>> Camera3RequestDescriptor(unsigned int frameNumber,\n> >>>>>>>>>> unsigned int numBuffers);\n> >>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>> index 3d6d2b4..fdde2c0 100644\n> >>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>> @@ -8,6 +8,7 @@\n> >>>>>>>>>> #include \"camera_hal_manager.h\"\n> >>>>>>>>>>\n> >>>>>>>>>> #include \n> >>>>>>>>>> +#include \n> >>>>>>>>>>\n> >>>>>>>>>> #include \"libcamera/internal/log.h\"\n> >>>>>>>>>>\n> >>>>>>>>>> @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> >>>>>>>>>> CameraHalManager::~CameraHalManager()\n> >>>>>>>>>> {\n> >>>>>>>>>> cameras_.clear();\n> >>>>>>>>>> + camerasMap_.clear();\n> >>>>>>>>>>\n> >>>>>>>>>> if (cameraManager_) {\n> >>>>>>>>>> cameraManager_->stop();\n> >>>>>>>>>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> >>>>>>>>>> {\n> >>>>>>>>>> cameraManager_ = new CameraManager();\n> >>>>>>>>>>\n> >>>>>>>>>> + /* Support camera hotplug. */\n> >>>>>>>>>> + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> >>>>>>>>>> + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> >>>>>>>>>> +\n> >>>>>>>>>> + cameraCounter_ = 0;\n> >>>>>>>>>> + externalCameraCounter_ = 1000;\n> >>>>>>>>>> +\n> >>>>>>>>>> int ret = cameraManager_->start();\n> >>>>>>>>>> if (ret) {\n> >>>>>>>>>> LOG(HAL, Error) << \"Failed to start camera manager: \"\n> >>>>>>>>>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> >>>>>>>>>> return ret;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> - /*\n> >>>>>>>>>> - * For each Camera registered in the system, a CameraDevice\n> >>>>>>>>>> - * gets created here to wraps a libcamera Camera instance.\n> >>>>>>>>>> - *\n> >>>>>>>>>> - * \\todo Support camera hotplug.\n> >>>>>>>>>> - */\n> >>>>>>>>>> - unsigned int index = 0;\n> >>>>>>>>>> - for (auto &cam : cameraManager_->cameras()) {\n> >>>>>>>>>> - CameraDevice *camera = new CameraDevice(index, cam);\n> >>>>>>>>>> - ret = camera->initialize();\n> >>>>>>>>>> - if (ret)\n> >>>>>>>>>> - continue;\n> >>>>>>>>>> -\n> >>>>>>>>>> - cameras_.emplace_back(camera);\n> >>>>>>>>>> - ++index;\n> >>>>>>>>>> - }\n> >>>>>>>>>> -\n> >>>>>>>>>> return 0;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> CameraDevice *CameraHalManager::open(unsigned int id,\n> >>>>>>>>>> const hw_module_t *hardwareModule)\n> >>>>>>>>>> {\n> >>>>>>>>>> - if (id >= numCameras()) {\n> >>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>> +\n> >>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> >>>>>>>>>> + [id](std::shared_ptr &cam) {\n> >>>>>>>>>> + return cam->id() == id;\n> >>>>>>>>>> + });\n> >>>>>>>>>> + if (iter == cameras_.end()) {\n> >>>>>>>>>\n> >>>>>>>>> I think you should break this (and the similar ones below) into private\n> >>>>>>>>> helpers instead if implementing the logic in-place\n> >>>>>>>>>\n> >>>>>>>>> CameraHalManager::cameraFromAndroidId(..);\n> >>>>>>>>> CameraHalManager::androidIdFromCamera(..);\n> >>>>>>>>>\n> >>>>>>>>>> LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> >>>>>>>>>> return nullptr;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> - CameraDevice *camera = cameras_[id].get();\n> >>>>>>>>>> + CameraDevice *camera = iter->get();\n> >>>>>>>>>> +\n> >>>>>>>>>> if (camera->open(hardwareModule))\n> >>>>>>>>>> return nullptr;\n> >>>>>>>>>>\n> >>>>>>>>>> @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> >>>>>>>>>> return camera;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> >>>>>>>>>> +{\n> >>>>>>>>>> + unsigned int id;\n> >>>>>>>>>> + bool isCameraExternal = false;\n> >>>>>>>>>> + bool isCameraNew = false;\n> >>>>>>>>>> +\n> >>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>> +\n> >>>>>>>>>> + /* Each camera is assigned a unique integer id when it is seen for the\n> >>>>>>>>>> + * first time. If the camera has been seen before, the id is reused and\n> >>>>>>>>>> + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> >>>>>>>>>> + *\n> >>>>>>>>>> + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> >>>>>>>>>> + */\n> >>>>>>>>>> + auto iter = camerasMap_.find(cam->id());\n> >>>>>>>>>> + if (iter != camerasMap_.end()) {\n> >>>>>>>>>> + id = iter->second;\n> >>>>>>>>>> + } else {\n> >>>>>>>>>> + isCameraNew = true;\n> >>>>>>>>>> +\n> >>>>>>>>>> + /*\n> >>>>>>>>>> + * Now check if this is an external camera and assign\n> >>>>>>>>>> + * its id accordingly.\n> >>>>>>>>>> + */\n> >>>>>>>>>> + const ControlList &properties = cam->properties();\n> >>>>>>>>>> + if (properties.contains(properties::Location) &&\n> >>>>>>>>>> + properties.get(properties::Location) &\n> >>>>>>>>>> + properties::CameraLocationExternal) {\n> >>>>>>>>>> + isCameraExternal = true;\n> >>>>>>>>>> + id = externalCameraCounter_;\n> >>>>>>>>>> + } else {\n> >>>>>>>>>> + id = cameraCounter_;\n> >>>>>>>>>> + }\n> >>>>>>>>>\n> >>>>>>>>> As I understand it the information the HAL wants is whether or not the\n> >>>>>>>>> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> >>>>>>>>>\n> >>>>>>>>> If so I wonder if using the camera location as a judgment of the camera\n> >>>>>>>>> is hot-plugged or not is the way to go here? Imagine a device where the\n> >>>>>>>>> camera is permanently attached (not hot-unpluggable) but not fixated in\n> >>>>>>>>> a location. I'm thinking cameras mounted at the end of instruments\n> >>>>>>>>> (medical instruments, hand held tools) or robotics (mounted at the arm\n> >>>>>>>>> of a welding robot). I would imagine those cameras would be marked as\n> >>>>>>>>> located externally but they would not really be hot-pluggable.\n> >>>>>>>>>\n> >>>>>>>>> I understand we have no other way to report or detect this at the moment\n> >>>>>>>>> and I'm not pushing hard for this to be solved as part of this series if\n> >>>>>>>>> it's not easy. But I think a bigger comment here is needed explaining\n> >>>>>>>>> that the HAL wants to know if a camera is hot-pluggable or not and does\n> >>>>>>>>> not really care if it's located internally or externally. I also think a\n> >>>>>>>>> \\todo should be added so it's not forgotten.\n> >>>>>>>>>\n> >>>>>>>>>> + }\n> >>>>>>>>>> +\n> >>>>>>>>>> + /*\n> >>>>>>>>>> + * For each Camera registered in the system, a CameraDevice\n> >>>>>>>>>> + * gets created here to wraps a libcamera Camera instance.\n> >>>>>>>>>> + */\n> >>>>>>>>>> + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> >>>>>>>>>> + int ret = camera->initialize();\n> >>>>>>>>>> + if (ret) {\n> >>>>>>>>>> + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> >>>>>>>>>> + return;\n> >>>>>>>>>> + }\n> >>>>>>>>>> +\n> >>>>>>>>>> + if (isCameraNew) {\n> >>>>>>>>>> + camerasMap_.emplace(cam->id(), id);\n> >>>>>>>>>> +\n> >>>>>>>>>> + if (isCameraExternal)\n> >>>>>>>>>> + externalCameraCounter_++;\n> >>>>>>>>>> + else\n> >>>>>>>>>> + cameraCounter_++;\n> >>>>>>>>>> + }\n> >>>>>>>>>> +\n> >>>>>>>>>> + cameras_.emplace_back(std::move(camera));\n> >>>>>>>>>> +\n> >>>>>>>>>> + if (callbacks_)\n> >>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> >>>>>>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n> >>>>>>>>>> + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> >>>>>>>>>> +}\n> >>>>>>>>>> +\n> >>>>>>>>>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> >>>>>>>>>> +{\n> >>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>> +\n> >>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> >>>>>>>>>> + [cam](std::shared_ptr &camera) {\n> >>>>>>>>>> + return cam.get() == camera->getCamera();\n> >>>>>>>>>> + });\n> >>>>>>>>>> + if (iter == cameras_.end())\n> >>>>>>>>>> + return;\n> >>>>>>>>>> +\n> >>>>>>>>>> + unsigned int id = (*iter)->id();\n> >>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> >>>>>>>>>> + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> >>>>>>>>>> + cameras_.erase(iter);\n> >>>>>>>>>> +\n> >>>>>>>>>> + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> >>>>>>>>>> +}\n> >>>>>>>>>> +\n> >>>>>>>>>> unsigned int CameraHalManager::numCameras() const\n> >>>>>>>>>> {\n> >>>>>>>>>> - return cameraManager_->cameras().size();\n> >>>>>>>>>> + return cameraCounter_;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> >>>>>>>>>> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> >>>>>>>>>> if (!info)\n> >>>>>>>>>> return -EINVAL;\n> >>>>>>>>>>\n> >>>>>>>>>> - if (id >= numCameras()) {\n> >>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>> +\n> >>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> >>>>>>>>>> + [id](std::shared_ptr &cam) {\n> >>>>>>>>>> + return cam->id() == id;\n> >>>>>>>>>> + });\n> >>>>>>>>>> + if (iter == cameras_.end()) {\n> >>>>>>>>>> LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> >>>>>>>>>> return -EINVAL;\n> >>>>>>>>>> }\n> >>>>>>>>>>\n> >>>>>>>>>> - CameraDevice *camera = cameras_[id].get();\n> >>>>>>>>>> + CameraDevice *camera = iter->get();\n> >>>>>>>>>>\n> >>>>>>>>>> info->facing = camera->facing();\n> >>>>>>>>>> info->orientation = camera->orientation();\n> >>>>>>>>>> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> >>>>>>>>>> void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> >>>>>>>>>> {\n> >>>>>>>>>> callbacks_ = callbacks;\n> >>>>>>>>>> +\n> >>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>> + /*\n> >>>>>>>>>> + * Few cameras might have been hotplugged before setting callbacks_ here.\n> >>>>>>>>>> + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> >>>>>>>>>> + * This hold only for external cameras, as internal cameras are assumed to\n> >>>>>>>>>> + * be present at module load time, by the framework.\n> >>>>>>>>>> + */\n> >>>>>>>>>> + for (auto &cam : cameraManager_->cameras()) {\n> >>>>>>>>>> + auto iter = camerasMap_.find(cam->id());\n> >>>>>>>>>> + if (iter == camerasMap_.end())\n> >>>>>>>>>> + continue;\n> >>>>>>>>>> +\n> >>>>>>>>>> + unsigned int id = iter->second;\n> >>>>>>>>>> + const ControlList &properties = cam->properties();\n> >>>>>>>>>> + if (properties.contains(properties::Location) &&\n> >>>>>>>>>> + properties.get(properties::Location) &\n> >>>>>>>>>> + properties::CameraLocationExternal) {\n> >>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> >>>>>>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n> >>>>>>>>>> + }\n> >>>>>>>>>> + }\n> >>>>>>>>>> }\n> >>>>>>>>>> diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> >>>>>>>>>> index a582f04..7c481d4 100644\n> >>>>>>>>>> --- a/src/android/camera_hal_manager.h\n> >>>>>>>>>> +++ b/src/android/camera_hal_manager.h\n> >>>>>>>>>> @@ -7,6 +7,8 @@\n> >>>>>>>>>> #ifndef __ANDROID_CAMERA_MANAGER_H__\n> >>>>>>>>>> #define __ANDROID_CAMERA_MANAGER_H__\n> >>>>>>>>>>\n> >>>>>>>>>> +#include \n> >>>>>>>>>> +#include \n> >>>>>>>>>> #include \n> >>>>>>>>>> #include \n> >>>>>>>>>>\n> >>>>>>>>>> @@ -18,6 +20,9 @@\n> >>>>>>>>>>\n> >>>>>>>>>> class CameraDevice;\n> >>>>>>>>>>\n> >>>>>>>>>> +using Mutex = std::mutex;\n> >>>>>>>>>> +using MutexLocker = std::unique_lock;\n> >>>>>>>>>> +\n> >>>>>>>>>> class CameraHalManager\n> >>>>>>>>>> {\n> >>>>>>>>>> public:\n> >>>>>>>>>> @@ -33,10 +38,18 @@ public:\n> >>>>>>>>>> void setCallbacks(const camera_module_callbacks_t *callbacks);\n> >>>>>>>>>>\n> >>>>>>>>>> private:\n> >>>>>>>>>> + void cameraAdded(std::shared_ptr cam);\n> >>>>>>>>>> + void cameraRemoved(std::shared_ptr cam);\n> >>>>>>>>>> +\n> >>>>>>>>>> libcamera::CameraManager *cameraManager_;\n> >>>>>>>>>>\n> >>>>>>>>>> const camera_module_callbacks_t *callbacks_;\n> >>>>>>>>>> - std::vector> cameras_;\n> >>>>>>>>>> + std::vector> cameras_;\n> >>>>>>>>>> + std::map camerasMap_;\n> >>>>>>>>>\n> >>>>>>>>> If each hot-plugged camera where treated as a new camera cameras_ and\n> >>>>>>>>> camerasMap_ could be merged to a\n> >>>>>>>>>\n> >>>>>>>>> std::map> cameras_;\n> >>>>>>>>>\n> >>>>>>>>> Which would eliminate the possibility of them going out-of-sync.\n> >>>>>>>>>\n> >>>>>>>>>> + Mutex mutex_;\n> >>>>>>>>>> +\n> >>>>>>>>>> + unsigned int externalCameraCounter_;\n> >>>>>>>>>> + unsigned int cameraCounter_;\n> >>>>>>>>>> };\n> >>>>>>>>>>\n> >>>>>>>>>> #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DCFAABD87C\n\tfor ;\n\tThu, 20 Aug 2020 14:42:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1BF1B61F7B;\n\tThu, 20 Aug 2020 16:42:36 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5BD4360381\n\tfor ;\n\tThu, 20 Aug 2020 16:42:34 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D0CB223D;\n\tThu, 20 Aug 2020 16:42:33 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"j4CL6kl1\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597934554;\n\tbh=4/dagxvZrgSxFVPWwRW6tNTEGKfuLxWC/T1GAsYrR4c=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=j4CL6kl1x4gHAD0H32A2kLvQFErirdI3HvWoJW9ChCQIfrG+zP7B3S3C5Xz6O1kvb\n\to9nwhuK8uDyOx1yHInxA1CyhT9QqnSMY/tiiYoYnaVB4+GtvVHLCh1HHDTRx6PWXAY\n\tgj7VCTElTOmOORS038oTF0kvGHEugHhX3igcJCYU=","Date":"Thu, 20 Aug 2020 17:42:15 +0300","From":"Laurent Pinchart ","To":"Shik Chen ","Message-ID":"<20200820144215.GH6593@pendragon.ideasonboard.com>","References":"<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t\n\t<20200819162140.GQ6049@pendragon.ideasonboard.com>\n\t<20200820071050.GB39265@wyvern>\n\t<20200820101028.GA6593@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12078,"web_url":"https://patchwork.libcamera.org/comment/12078/","msgid":"","date":"2020-08-20T15:18:02","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":8,"url":"https://patchwork.libcamera.org/api/people/8/","name":"Shik Chen","email":"shik@google.com"},"content":"Hi Laurent,\n\n\nOn Thu, Aug 20, 2020 at 10:42 PM Laurent Pinchart\n wrote:\n>\n> Hi Shik,\n>\n> On Thu, Aug 20, 2020 at 08:44:31PM +0800, Shik Chen wrote:\n> > On Thu, Aug 20, 2020 at 6:10 PM Laurent Pinchart wrote:\n> > > On Thu, Aug 20, 2020 at 09:12:10AM +0200, Niklas Söderlund wrote:\n> > >> On 2020-08-19 19:21:40 +0300, Laurent Pinchart wrote:\n> > >>> On Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> > >>>> On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> > >>>>> On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> > >>>>>> On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> > >>>>>>>\n> > >>>>>>> Hi Shik,\n> > >>>>>>>\n> > >>>>>>> Would it be possible for you to have a look at the question below ?\n> > >>>>>>>\n> > >>>>>>> On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> > >>>>>>>> Hi Niklas,\n> > >>>>>>>>\n> > >>>>>>>> (CC'ing Shik)\n> > >>>>>>>>\n> > >>>>>>>> On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> > >>>>>>>>> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> > >>>>>>>>>> Extend the support for camera hotplug from libcamera's CameraManager\n> > >>>>>>>>>> to CameraHalManager. Use camera module callbacks to let the framework\n> > >>>>>>>>>> know about the hotplug events and change the status of cameras being\n> > >>>>>>>>>> being hotplugged or unplugged via camera_device_status_change().\n> > >>>>>>>>>>\n> > >>>>>>>>>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> > >>>>>>>>>> past by the CameraHalManager. If the camera is seen for the first time,\n> > >>>>>>>>>> a new id is assigned to it. If the camera has been seen before by the\n> > >>>>>>>>>> manager, it's old id is reused. IDs for internal cameras start with\n> > >>>>>>>>>> '0' and for external cameras, they start with '1000'. Note, for the\n> > >>>>>>>>>> current implementation, we assume all UVC cameras are external cameras.\n> > >>>>>>>>>\n> > >>>>>>>>> I wonder if keeping the cache of previously seen cameras or if we should\n> > >>>>>>>>> not just treat any hot-plugged camera as a new one? I can't think of any\n> > >>>>>>>>> benefit of preserving the same numerical ID between two plug events,\n> > >>>>>>>>> while I can think of quiet a few cons.\n> > >>>>>>>>>\n> > >>>>>>>>> - It's confusing when debugging as un-plugging and then replugging the\n> > >>>>>>>>> same camera will result in logs where the numerical ID is the same for\n> > >>>>>>>>> both. This may even result in things working by \"chance\" is it reuses\n> > >>>>>>>>> an already known numerical ID.\n> > >>>>>>>>>\n> > >>>>>>>>> - Looking at the code plugging a UVC camera in a different USB port will\n> > >>>>>>>>> generate a different Camera::id() result and then be treated as a new\n> > >>>>>>>>> camera vs plugging it in the same port and it then being treatad as a\n> > >>>>>>>>> new camera.\n> > >>>>>>>>>\n> > >>>>>>>>> - The logic in this patch is more complex due to it both having to deal\n> > >>>>>>>>> with known and new cameras.\n> > >>>>>>>>>\n> > >>>>>>>>> What is the benefit of this cache that I'm missing?\n> > >>>>>>>>\n> > >>>>>>>> It's not clear whether Android requires it. It requires different\n> > >>>>>>>> cameras to have different IDs, but I couldn't find a mention of how\n> > >>>>>>>> identical cameras should be treated. The Chrome OS UVC HAL caches the\n> > >>>>>>>> ID, I don't know if it's mandatory though.\n> > >>>>>>>>\n> > >>>>>>>> Shik, as you've worked on the UVC HAL, would you happen to know if there\n> > >>>>>>>> are requirements to keep the same ID when a camera is unplugged and\n> > >>>>>>>> replugged ?\n> > >>>>>>\n> > >>>>>> It's for persistency in settings UI. Chrome uses the camera id as the key of\n> > >>>>>> the default camera in chrome://settings/content/camera. Reuse the same id for\n> > >>>>>> the same camera so the default camera would be changed back after replug.\n> > >>>>>>\n> > >>>>>> It's introduced in this CL: https://crrev.com/c/1608985\n> > >>>>>\n> > >>>>> Thank you for the information. We have a similar caching system, which\n> > >>>>> however suffers from two limitations:\n> > >>\n> > >> We have more limitations I'm afraid :-(\n> > >>\n> > >> If I understand Shik correctly the numerical camera ID is stored\n> > >> persistently in the camera settings UI. Our code does not guarantee that\n> > >> the numerical ID in the libcamera HAL is persistent. The cameras in the\n> > >> CameraManager is stored in a vector and is therefor susceptible to be\n> > >> enumerated in different order each time the HAL is started.\n> > >>\n> > >> We could of course remedy this by sorting the camera vecotr before\n> > >> assigning numerical IDs. But this would still not be guaranteed to be\n> > >> persistent as more/less cameras registered in the HAL could alter the\n> > >> read out order. If we really wish to do this should we save/load the\n> > >> camera ID to numerical ID mapping to file (not nice)?\n> > >\n> > > Shik, does the ID cache need to be persistent across reboots (or\n> > > restarts of the camera service) ?\n> >\n> > For built-in cameras, yes. They need to be persistent across reboots and order\n> > the camera ids by facing. There is a common implicit assumption in many\n> > Android camera apps that assume the front camera is 0 and back camera is 1.\n>\n> One may wish this would be documented in the HAL API ;-)\n>\n> > For external cameras, ideally yes. But for most of the use cases currently,\n> > it's acceptable to use a solution that only persists the ids in a boot cycle,\n> > and always assign the same id for the first external camera after boots.\n>\n> There are conflicting requirements there. We can't assign the same ID\n> for the first external camera after boot (we use 1000 today) and also\n> have stable IDs across boots.\n\nThese two are not the requirements that need to be held for the same time.\nAssigning the same ID for the first external camera is a workaround and an\napproximation for the ideal solution, which works fine when there is only one\nexternal camera.\n\n>\n> > >>>>> - If a camera is unplugged, and a different camera with the same VID:PID\n> > >>>>> is plugged into the same USB port, the new camera be given the same ID\n> > >>>>> as the previous camera.\n> > >>>>\n> > >>>> The current CrOS implementation will also use the same id in this case, and I\n> > >>>> think it should be fine.\n> > >>>>\n> > >>>>> - If a camera is unplugged and replugged in a different USB port, it\n> > >>>>> will be given a different ID.\n> > >>>>\n> > >>>> Hmm this looks non-ideal and may confuse the users.\n> > >>>\n> > >>> We will try to fix that on top\n> > >>>\n> > >>> This isn't trivial to handle, and there are corner cases. I'm thinking\n> > >>> in particular about a system where two identical external USB cameras\n> > >>> would be set up to point at particular locations. How to handle cameras\n> > >>> stable IDs in that case is debatable, and I don't think the problem can\n> > >>> be solved unless the cameras report distinct serial numbers (and most\n> > >>> webcams don't :-S). At the moment, libcamera identifies each camera with\n> > >>> a stable ID (guaranteed to be unique and stable across reboots, as long\n> > >>> as the hardware and firmware are not modified) made of the concatenation\n> > >>> of\n> > >>>\n> > >>> - The USB controller DT or ACPI path\n> > >>> - The USB port(s) number(s) (starting at the root hub, going through\n> > >>> every hub)\n> > >>> - The USB device VID:PID\n> > >>>\n> > >>> The Android ID is then assigned based on the libcamera ID, with a cache\n> > >>> to ensure that the same HAL ID will be given to the same libcamera ID.\n> > >>> If we were to drop the USB port number, we'd have to find another way to\n> > >>> create a stable and unique camera ID.\n> > >>>\n> > >>>>> Is that acceptable from a Chrome OS point of view ?\n> > >>>>>\n> > >>>>> Would you happen to know how Android uses camera IDs, and whether it\n> > >>>>> requires or could benefit from ID caching ?\n> > >>>>\n> > >>>> Each camera app could have a similar preference caching system like the\n> > >>>> settings UI, so it would be better to reuse the same id for the same camera.\n> > >>>> AFAIK there is no such requirement at the framework level.\n> > >>\n> > >> This worries me a bit. Are we not altering the libcamera HAL to fit a\n> > >> single use-case, all be it an important user. But if the framework does\n> > >> not document an expected behavior is it not tricky to implement\n> > >> correctly? What if we later find a different usage pattern in another\n> > >> application which conflicts with what we are trying to do here?\n> > >\n> > > This is an undocumented area, so HALs can do pretty much what they want.\n> > > If different platforms have different requirements, then we'll have to\n> > > deal with it, possibly working with the platform vendors to align the\n> > > requirements.\n> >\n> > Yes and we need to live with Hyrum's Law :P\n> > If it's not handled here, we would still need to have another id mapping layer\n> > somewhere else.\n>\n> That may be needed down the line though. As Chrome OS uses the Android\n> camera HAL API, if differences in requirements are found between the two\n> system, supporting the Chrome OS requirements in the Chrome OS camera\n> service, on top of the HAL, may be the best option. We'll see when we'll\n> run into actual differences.\n>\n\nAs long as the HAL provides enough information for the upper layer, I am fine\nwith either approach. We can do some experiments and see which one fits better.\n\nNote that some extra fields are exposed through vendor tags currently, for\nvarious reasons:\n1. The human readable camera name\n2. USB vendor id and product id\n3. The device path\n\nThose are lessons and requirements we learnt from migrating the camera stack of\nChrome on Chrome OS to the new Android HALv3 Camera API based service. Hope\nthis information would help :)\n\n> > >>>>>>>>>> CameraDevice is now a shared object and cameras_ vector stores shared\n> > >>>>>>>>>> pointers to CameraDevice. This is done in order to introduce reference\n> > >>>>>>>>>> counting for CameraDevice objects - especially to handle hot-unplug\n> > >>>>>>>>>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n> > >>>>>>>>>>\n> > >>>>>>>>>> Signed-off-by: Umang Jain \n> > >>>>>>>>>> ---\n> > >>>>>>>>>> src/android/camera_device.cpp | 15 +++\n> > >>>>>>>>>> src/android/camera_device.h | 8 +-\n> > >>>>>>>>>> src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> > >>>>>>>>>> src/android/camera_hal_manager.h | 15 ++-\n> > >>>>>>>>>> 4 files changed, 166 insertions(+), 25 deletions(-)\n> > >>>>>>>>>>\n> > >>>>>>>>>> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > >>>>>>>>>> index d918350..a79bb69 100644\n> > >>>>>>>>>> --- a/src/android/camera_device.cpp\n> > >>>>>>>>>> +++ b/src/android/camera_device.cpp\n> > >>>>>>>>>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> > >>>>>>>>>> delete it.second;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> +std::shared_ptr CameraDevice::create(unsigned int id,\n> > >>>>>>>>>> + const std::shared_ptr &cam)\n> > >>>>>>>>>> +{\n> > >>>>>>>>>> + struct Deleter : std::default_delete {\n> > >>>>>>>>>> + void operator()(CameraDevice *camera)\n> > >>>>>>>>>> + {\n> > >>>>>>>>>> + delete camera;\n> > >>>>>>>>>> + }\n> > >>>>>>>>>> + };\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + CameraDevice *camera = new CameraDevice(id, cam);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + return std::shared_ptr(camera, Deleter());\n> > >>>>>>>>>> +}\n> > >>>>>>>>>\n> > >>>>>>>>> As Kieran points out I think this should be added in a separate as this\n> > >>>>>>>>> one is quiet large and therefore hard to review.\n> > >>>>>>>>>\n> > >>>>>>>>>> +\n> > >>>>>>>>>> /*\n> > >>>>>>>>>> * Initialize the camera static information.\n> > >>>>>>>>>> * This method is called before the camera device is opened.\n> > >>>>>>>>>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > >>>>>>>>>> index 7be9e11..7f9e010 100644\n> > >>>>>>>>>> --- a/src/android/camera_device.h\n> > >>>>>>>>>> +++ b/src/android/camera_device.h\n> > >>>>>>>>>> @@ -47,8 +47,8 @@ struct CameraStream {\n> > >>>>>>>>>> class CameraDevice : protected libcamera::Loggable\n> > >>>>>>>>>> {\n> > >>>>>>>>>> public:\n> > >>>>>>>>>> - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > >>>>>>>>>> - ~CameraDevice();\n> > >>>>>>>>>> + static std::shared_ptr create(unsigned int id,\n> > >>>>>>>>>> + const std::shared_ptr &cam);\n> > >>>>>>>>>>\n> > >>>>>>>>>> int initialize();\n> > >>>>>>>>>>\n> > >>>>>>>>>> @@ -57,6 +57,7 @@ public:\n> > >>>>>>>>>>\n> > >>>>>>>>>> unsigned int id() const { return id_; }\n> > >>>>>>>>>> camera3_device_t *camera3Device() { return &camera3Device_; }\n> > >>>>>>>>>> + const libcamera::Camera *getCamera() { return camera_.get(); };\n> > >>>>>>>>>\n> > >>>>>>>>> Same here this can be done in a separate commit. Also I think this could\n> > >>>>>>>>> be named camera() instead of getCamera()\n> > >>>>>>>>>\n> > >>>>>>>>>>\n> > >>>>>>>>>> int facing() const { return facing_; }\n> > >>>>>>>>>> int orientation() const { return orientation_; }\n> > >>>>>>>>>> @@ -72,6 +73,9 @@ protected:\n> > >>>>>>>>>> std::string logPrefix() const override;\n> > >>>>>>>>>>\n> > >>>>>>>>>> private:\n> > >>>>>>>>>> + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> > >>>>>>>>>> + ~CameraDevice();\n> > >>>>>>>>>> +\n> > >>>>>>>>>> struct Camera3RequestDescriptor {\n> > >>>>>>>>>> Camera3RequestDescriptor(unsigned int frameNumber,\n> > >>>>>>>>>> unsigned int numBuffers);\n> > >>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>> index 3d6d2b4..fdde2c0 100644\n> > >>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> > >>>>>>>>>> @@ -8,6 +8,7 @@\n> > >>>>>>>>>> #include \"camera_hal_manager.h\"\n> > >>>>>>>>>>\n> > >>>>>>>>>> #include \n> > >>>>>>>>>> +#include \n> > >>>>>>>>>>\n> > >>>>>>>>>> #include \"libcamera/internal/log.h\"\n> > >>>>>>>>>>\n> > >>>>>>>>>> @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> > >>>>>>>>>> CameraHalManager::~CameraHalManager()\n> > >>>>>>>>>> {\n> > >>>>>>>>>> cameras_.clear();\n> > >>>>>>>>>> + camerasMap_.clear();\n> > >>>>>>>>>>\n> > >>>>>>>>>> if (cameraManager_) {\n> > >>>>>>>>>> cameraManager_->stop();\n> > >>>>>>>>>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> > >>>>>>>>>> {\n> > >>>>>>>>>> cameraManager_ = new CameraManager();\n> > >>>>>>>>>>\n> > >>>>>>>>>> + /* Support camera hotplug. */\n> > >>>>>>>>>> + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> > >>>>>>>>>> + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + cameraCounter_ = 0;\n> > >>>>>>>>>> + externalCameraCounter_ = 1000;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> int ret = cameraManager_->start();\n> > >>>>>>>>>> if (ret) {\n> > >>>>>>>>>> LOG(HAL, Error) << \"Failed to start camera manager: \"\n> > >>>>>>>>>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> > >>>>>>>>>> return ret;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> - /*\n> > >>>>>>>>>> - * For each Camera registered in the system, a CameraDevice\n> > >>>>>>>>>> - * gets created here to wraps a libcamera Camera instance.\n> > >>>>>>>>>> - *\n> > >>>>>>>>>> - * \\todo Support camera hotplug.\n> > >>>>>>>>>> - */\n> > >>>>>>>>>> - unsigned int index = 0;\n> > >>>>>>>>>> - for (auto &cam : cameraManager_->cameras()) {\n> > >>>>>>>>>> - CameraDevice *camera = new CameraDevice(index, cam);\n> > >>>>>>>>>> - ret = camera->initialize();\n> > >>>>>>>>>> - if (ret)\n> > >>>>>>>>>> - continue;\n> > >>>>>>>>>> -\n> > >>>>>>>>>> - cameras_.emplace_back(camera);\n> > >>>>>>>>>> - ++index;\n> > >>>>>>>>>> - }\n> > >>>>>>>>>> -\n> > >>>>>>>>>> return 0;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> CameraDevice *CameraHalManager::open(unsigned int id,\n> > >>>>>>>>>> const hw_module_t *hardwareModule)\n> > >>>>>>>>>> {\n> > >>>>>>>>>> - if (id >= numCameras()) {\n> > >>>>>>>>>> + MutexLocker locker(mutex_);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > >>>>>>>>>> + [id](std::shared_ptr &cam) {\n> > >>>>>>>>>> + return cam->id() == id;\n> > >>>>>>>>>> + });\n> > >>>>>>>>>> + if (iter == cameras_.end()) {\n> > >>>>>>>>>\n> > >>>>>>>>> I think you should break this (and the similar ones below) into private\n> > >>>>>>>>> helpers instead if implementing the logic in-place\n> > >>>>>>>>>\n> > >>>>>>>>> CameraHalManager::cameraFromAndroidId(..);\n> > >>>>>>>>> CameraHalManager::androidIdFromCamera(..);\n> > >>>>>>>>>\n> > >>>>>>>>>> LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > >>>>>>>>>> return nullptr;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> - CameraDevice *camera = cameras_[id].get();\n> > >>>>>>>>>> + CameraDevice *camera = iter->get();\n> > >>>>>>>>>> +\n> > >>>>>>>>>> if (camera->open(hardwareModule))\n> > >>>>>>>>>> return nullptr;\n> > >>>>>>>>>>\n> > >>>>>>>>>> @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> > >>>>>>>>>> return camera;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> > >>>>>>>>>> +{\n> > >>>>>>>>>> + unsigned int id;\n> > >>>>>>>>>> + bool isCameraExternal = false;\n> > >>>>>>>>>> + bool isCameraNew = false;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + MutexLocker locker(mutex_);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + /* Each camera is assigned a unique integer id when it is seen for the\n> > >>>>>>>>>> + * first time. If the camera has been seen before, the id is reused and\n> > >>>>>>>>>> + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> > >>>>>>>>>> + *\n> > >>>>>>>>>> + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> > >>>>>>>>>> + */\n> > >>>>>>>>>> + auto iter = camerasMap_.find(cam->id());\n> > >>>>>>>>>> + if (iter != camerasMap_.end()) {\n> > >>>>>>>>>> + id = iter->second;\n> > >>>>>>>>>> + } else {\n> > >>>>>>>>>> + isCameraNew = true;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + /*\n> > >>>>>>>>>> + * Now check if this is an external camera and assign\n> > >>>>>>>>>> + * its id accordingly.\n> > >>>>>>>>>> + */\n> > >>>>>>>>>> + const ControlList &properties = cam->properties();\n> > >>>>>>>>>> + if (properties.contains(properties::Location) &&\n> > >>>>>>>>>> + properties.get(properties::Location) &\n> > >>>>>>>>>> + properties::CameraLocationExternal) {\n> > >>>>>>>>>> + isCameraExternal = true;\n> > >>>>>>>>>> + id = externalCameraCounter_;\n> > >>>>>>>>>> + } else {\n> > >>>>>>>>>> + id = cameraCounter_;\n> > >>>>>>>>>> + }\n> > >>>>>>>>>\n> > >>>>>>>>> As I understand it the information the HAL wants is whether or not the\n> > >>>>>>>>> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> > >>>>>>>>>\n> > >>>>>>>>> If so I wonder if using the camera location as a judgment of the camera\n> > >>>>>>>>> is hot-plugged or not is the way to go here? Imagine a device where the\n> > >>>>>>>>> camera is permanently attached (not hot-unpluggable) but not fixated in\n> > >>>>>>>>> a location. I'm thinking cameras mounted at the end of instruments\n> > >>>>>>>>> (medical instruments, hand held tools) or robotics (mounted at the arm\n> > >>>>>>>>> of a welding robot). I would imagine those cameras would be marked as\n> > >>>>>>>>> located externally but they would not really be hot-pluggable.\n> > >>>>>>>>>\n> > >>>>>>>>> I understand we have no other way to report or detect this at the moment\n> > >>>>>>>>> and I'm not pushing hard for this to be solved as part of this series if\n> > >>>>>>>>> it's not easy. But I think a bigger comment here is needed explaining\n> > >>>>>>>>> that the HAL wants to know if a camera is hot-pluggable or not and does\n> > >>>>>>>>> not really care if it's located internally or externally. I also think a\n> > >>>>>>>>> \\todo should be added so it's not forgotten.\n> > >>>>>>>>>\n> > >>>>>>>>>> + }\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + /*\n> > >>>>>>>>>> + * For each Camera registered in the system, a CameraDevice\n> > >>>>>>>>>> + * gets created here to wraps a libcamera Camera instance.\n> > >>>>>>>>>> + */\n> > >>>>>>>>>> + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> > >>>>>>>>>> + int ret = camera->initialize();\n> > >>>>>>>>>> + if (ret) {\n> > >>>>>>>>>> + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> > >>>>>>>>>> + return;\n> > >>>>>>>>>> + }\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + if (isCameraNew) {\n> > >>>>>>>>>> + camerasMap_.emplace(cam->id(), id);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + if (isCameraExternal)\n> > >>>>>>>>>> + externalCameraCounter_++;\n> > >>>>>>>>>> + else\n> > >>>>>>>>>> + cameraCounter_++;\n> > >>>>>>>>>> + }\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + cameras_.emplace_back(std::move(camera));\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + if (callbacks_)\n> > >>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> > >>>>>>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n> > >>>>>>>>>> + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> > >>>>>>>>>> +}\n> > >>>>>>>>>> +\n> > >>>>>>>>>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> > >>>>>>>>>> +{\n> > >>>>>>>>>> + MutexLocker locker(mutex_);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > >>>>>>>>>> + [cam](std::shared_ptr &camera) {\n> > >>>>>>>>>> + return cam.get() == camera->getCamera();\n> > >>>>>>>>>> + });\n> > >>>>>>>>>> + if (iter == cameras_.end())\n> > >>>>>>>>>> + return;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + unsigned int id = (*iter)->id();\n> > >>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> > >>>>>>>>>> + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> > >>>>>>>>>> + cameras_.erase(iter);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> > >>>>>>>>>> +}\n> > >>>>>>>>>> +\n> > >>>>>>>>>> unsigned int CameraHalManager::numCameras() const\n> > >>>>>>>>>> {\n> > >>>>>>>>>> - return cameraManager_->cameras().size();\n> > >>>>>>>>>> + return cameraCounter_;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > >>>>>>>>>> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > >>>>>>>>>> if (!info)\n> > >>>>>>>>>> return -EINVAL;\n> > >>>>>>>>>>\n> > >>>>>>>>>> - if (id >= numCameras()) {\n> > >>>>>>>>>> + MutexLocker locker(mutex_);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> > >>>>>>>>>> + [id](std::shared_ptr &cam) {\n> > >>>>>>>>>> + return cam->id() == id;\n> > >>>>>>>>>> + });\n> > >>>>>>>>>> + if (iter == cameras_.end()) {\n> > >>>>>>>>>> LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> > >>>>>>>>>> return -EINVAL;\n> > >>>>>>>>>> }\n> > >>>>>>>>>>\n> > >>>>>>>>>> - CameraDevice *camera = cameras_[id].get();\n> > >>>>>>>>>> + CameraDevice *camera = iter->get();\n> > >>>>>>>>>>\n> > >>>>>>>>>> info->facing = camera->facing();\n> > >>>>>>>>>> info->orientation = camera->orientation();\n> > >>>>>>>>>> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> > >>>>>>>>>> void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> > >>>>>>>>>> {\n> > >>>>>>>>>> callbacks_ = callbacks;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + MutexLocker locker(mutex_);\n> > >>>>>>>>>> + /*\n> > >>>>>>>>>> + * Few cameras might have been hotplugged before setting callbacks_ here.\n> > >>>>>>>>>> + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> > >>>>>>>>>> + * This hold only for external cameras, as internal cameras are assumed to\n> > >>>>>>>>>> + * be present at module load time, by the framework.\n> > >>>>>>>>>> + */\n> > >>>>>>>>>> + for (auto &cam : cameraManager_->cameras()) {\n> > >>>>>>>>>> + auto iter = camerasMap_.find(cam->id());\n> > >>>>>>>>>> + if (iter == camerasMap_.end())\n> > >>>>>>>>>> + continue;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + unsigned int id = iter->second;\n> > >>>>>>>>>> + const ControlList &properties = cam->properties();\n> > >>>>>>>>>> + if (properties.contains(properties::Location) &&\n> > >>>>>>>>>> + properties.get(properties::Location) &\n> > >>>>>>>>>> + properties::CameraLocationExternal) {\n> > >>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> > >>>>>>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n> > >>>>>>>>>> + }\n> > >>>>>>>>>> + }\n> > >>>>>>>>>> }\n> > >>>>>>>>>> diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> > >>>>>>>>>> index a582f04..7c481d4 100644\n> > >>>>>>>>>> --- a/src/android/camera_hal_manager.h\n> > >>>>>>>>>> +++ b/src/android/camera_hal_manager.h\n> > >>>>>>>>>> @@ -7,6 +7,8 @@\n> > >>>>>>>>>> #ifndef __ANDROID_CAMERA_MANAGER_H__\n> > >>>>>>>>>> #define __ANDROID_CAMERA_MANAGER_H__\n> > >>>>>>>>>>\n> > >>>>>>>>>> +#include \n> > >>>>>>>>>> +#include \n> > >>>>>>>>>> #include \n> > >>>>>>>>>> #include \n> > >>>>>>>>>>\n> > >>>>>>>>>> @@ -18,6 +20,9 @@\n> > >>>>>>>>>>\n> > >>>>>>>>>> class CameraDevice;\n> > >>>>>>>>>>\n> > >>>>>>>>>> +using Mutex = std::mutex;\n> > >>>>>>>>>> +using MutexLocker = std::unique_lock;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> class CameraHalManager\n> > >>>>>>>>>> {\n> > >>>>>>>>>> public:\n> > >>>>>>>>>> @@ -33,10 +38,18 @@ public:\n> > >>>>>>>>>> void setCallbacks(const camera_module_callbacks_t *callbacks);\n> > >>>>>>>>>>\n> > >>>>>>>>>> private:\n> > >>>>>>>>>> + void cameraAdded(std::shared_ptr cam);\n> > >>>>>>>>>> + void cameraRemoved(std::shared_ptr cam);\n> > >>>>>>>>>> +\n> > >>>>>>>>>> libcamera::CameraManager *cameraManager_;\n> > >>>>>>>>>>\n> > >>>>>>>>>> const camera_module_callbacks_t *callbacks_;\n> > >>>>>>>>>> - std::vector> cameras_;\n> > >>>>>>>>>> + std::vector> cameras_;\n> > >>>>>>>>>> + std::map camerasMap_;\n> > >>>>>>>>>\n> > >>>>>>>>> If each hot-plugged camera where treated as a new camera cameras_ and\n> > >>>>>>>>> camerasMap_ could be merged to a\n> > >>>>>>>>>\n> > >>>>>>>>> std::map> cameras_;\n> > >>>>>>>>>\n> > >>>>>>>>> Which would eliminate the possibility of them going out-of-sync.\n> > >>>>>>>>>\n> > >>>>>>>>>> + Mutex mutex_;\n> > >>>>>>>>>> +\n> > >>>>>>>>>> + unsigned int externalCameraCounter_;\n> > >>>>>>>>>> + unsigned int cameraCounter_;\n> > >>>>>>>>>> };\n> > >>>>>>>>>>\n> > >>>>>>>>>> #endif /* __ANDROID_CAMERA_MANAGER_H__ */\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n\n- shik","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C53A7BD87C\n\tfor ;\n\tThu, 20 Aug 2020 15:18:22 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 49C8360381;\n\tThu, 20 Aug 2020 17:18:22 +0200 (CEST)","from mail-qk1-x730.google.com (mail-qk1-x730.google.com\n\t[IPv6:2607:f8b0:4864:20::730])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 25ACE60381\n\tfor ;\n\tThu, 20 Aug 2020 17:18:21 +0200 (CEST)","by mail-qk1-x730.google.com with SMTP id 77so1815642qkm.5\n\tfor ;\n\tThu, 20 Aug 2020 08:18:21 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"biPHg06V\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=20161025; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=h3KZa7rPdlzYvyDEMPgS160J8iYd2AGbXH2SXKaUCmo=;\n\tb=biPHg06VEOLhW0UDIFxLjvqW7WIrA50g2oqD94ErK3dxuUJ2nQEYB2/IO5JKEctksQ\n\tWXfz+R68Q5RNSjGYETmKxXGafCg5n0FVjGLSSc3LNgYpYdCagUX6j22zNVSUAYnN8B+H\n\tsVvcKBNTw42+9wJnRGPCwWOrqUJNm6D+3a9Zztfj/VRh1iBBHd+VPUM+aJEMmrD4fVH4\n\trkKAW/vLZZW0EYDMccMEZ4IMCZgySAoTio3QywITmgIwQ8EhuHkU84/dltwHLyXHG+vR\n\td1kmAxWqtgCePKm0ecLonfISe7IHB74GF4IUi5GbBxiO3EyE8IcJh2i3DYY5zLaT81ds\n\tEzQg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=h3KZa7rPdlzYvyDEMPgS160J8iYd2AGbXH2SXKaUCmo=;\n\tb=Q9qIYO6hLWAHdJgLxM8JiLspzxKbJF1PbhFvpg+ipBEV/mx+o1LZoDCm+e4O5P7Vho\n\teN/uYYF3S6zgyJ/MGgIHsdNavBv77UXbQCoRfd+LDGNFZXWuVGYrLn8BE1P2s3FcCD2F\n\t8nPslbxU9sr7T9oNJEdCVnR7Q++ULWs8/sK4ckx2lnGr4BFYmRrFbsHcElXX89EtSYev\n\tG0+jhjzunNo8OojoaZmhixMEzbdrylyom+dKMm3I9NPj46EAoR8NfDXV7s9W5nozC1T6\n\tjy1uJDbu7o6Z64FLdNKpsXdW36f6YHX6hgx+fttZgNpLJH+mktkfLyxCkBOW1xRss8Yk\n\t0NbQ==","X-Gm-Message-State":"AOAM533j3zaM6aewVEA/4K/ep+wYxhEO+6xIVPfQH5/iWLPLTTb9FaBR\n\tZQFr0dnb26htVokD+4WvYL/RUodvvt8rM4y89HyGyw==","X-Google-Smtp-Source":"ABdhPJz/V7GicmNFIkMzFk90BBecZ3eMsjxeJA4O4BVNlQZUPxtPEvZQTz43X/NnPAsYYXcTTzdrrvX1PAPeU8wq98o=","X-Received":"by 2002:a37:9a93:: with SMTP id\n\tc141mr2923180qke.145.1597936699162; \n\tThu, 20 Aug 2020 08:18:19 -0700 (PDT)","MIME-Version":"1.0","References":"<20200813114418.GE1061689@oden.dyn.berto.se>\n\t<20200813120529.GE6057@pendragon.ideasonboard.com>\n\t<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t\n\t<20200819162140.GQ6049@pendragon.ideasonboard.com>\n\t<20200820071050.GB39265@wyvern>\n\t<20200820101028.GA6593@pendragon.ideasonboard.com>\n\t\n\t<20200820144215.GH6593@pendragon.ideasonboard.com>","In-Reply-To":"<20200820144215.GH6593@pendragon.ideasonboard.com>","From":"Shik Chen ","Date":"Thu, 20 Aug 2020 23:18:02 +0800","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12080,"web_url":"https://patchwork.libcamera.org/comment/12080/","msgid":"<20200820153154.GP6593@pendragon.ideasonboard.com>","date":"2020-08-20T15:31:54","subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Shik,\n\nOn Thu, Aug 20, 2020 at 11:18:02PM +0800, Shik Chen wrote:\n> On Thu, Aug 20, 2020 at 10:42 PM Laurent Pinchart wrote:\n> > On Thu, Aug 20, 2020 at 08:44:31PM +0800, Shik Chen wrote:\n> >> On Thu, Aug 20, 2020 at 6:10 PM Laurent Pinchart wrote:\n> >>> On Thu, Aug 20, 2020 at 09:12:10AM +0200, Niklas Söderlund wrote:\n> >>>> On 2020-08-19 19:21:40 +0300, Laurent Pinchart wrote:\n> >>>>> On Thu, Aug 20, 2020 at 12:04:41AM +0800, Shik Chen wrote:\n> >>>>>> On Wed, Aug 19, 2020 at 11:52 PM Laurent Pinchart wrote:\n> >>>>>>> On Wed, Aug 19, 2020 at 11:41:45PM +0800, Shik Chen wrote:\n> >>>>>>>> On Tue, Aug 18, 2020 at 4:36 PM Laurent Pinchart wrote:\n> >>>>>>>>>\n> >>>>>>>>> Hi Shik,\n> >>>>>>>>>\n> >>>>>>>>> Would it be possible for you to have a look at the question below ?\n> >>>>>>>>>\n> >>>>>>>>> On Thu, Aug 13, 2020 at 03:05:30PM +0300, Laurent Pinchart wrote:\n> >>>>>>>>>> Hi Niklas,\n> >>>>>>>>>>\n> >>>>>>>>>> (CC'ing Shik)\n> >>>>>>>>>>\n> >>>>>>>>>> On Thu, Aug 13, 2020 at 01:44:18PM +0200, Niklas Söderlund wrote:\n> >>>>>>>>>>> On 2020-08-10 12:04:14 +0000, Umang Jain wrote:\n> >>>>>>>>>>>> Extend the support for camera hotplug from libcamera's CameraManager\n> >>>>>>>>>>>> to CameraHalManager. Use camera module callbacks to let the framework\n> >>>>>>>>>>>> know about the hotplug events and change the status of cameras being\n> >>>>>>>>>>>> being hotplugged or unplugged via camera_device_status_change().\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> Introduce a map camerasMap_ which book-keeps all cameras seen in the\n> >>>>>>>>>>>> past by the CameraHalManager. If the camera is seen for the first time,\n> >>>>>>>>>>>> a new id is assigned to it. If the camera has been seen before by the\n> >>>>>>>>>>>> manager, it's old id is reused. IDs for internal cameras start with\n> >>>>>>>>>>>> '0' and for external cameras, they start with '1000'. Note, for the\n> >>>>>>>>>>>> current implementation, we assume all UVC cameras are external cameras.\n> >>>>>>>>>>>\n> >>>>>>>>>>> I wonder if keeping the cache of previously seen cameras or if we should\n> >>>>>>>>>>> not just treat any hot-plugged camera as a new one? I can't think of any\n> >>>>>>>>>>> benefit of preserving the same numerical ID between two plug events,\n> >>>>>>>>>>> while I can think of quiet a few cons.\n> >>>>>>>>>>>\n> >>>>>>>>>>> - It's confusing when debugging as un-plugging and then replugging the\n> >>>>>>>>>>> same camera will result in logs where the numerical ID is the same for\n> >>>>>>>>>>> both. This may even result in things working by \"chance\" is it reuses\n> >>>>>>>>>>> an already known numerical ID.\n> >>>>>>>>>>>\n> >>>>>>>>>>> - Looking at the code plugging a UVC camera in a different USB port will\n> >>>>>>>>>>> generate a different Camera::id() result and then be treated as a new\n> >>>>>>>>>>> camera vs plugging it in the same port and it then being treatad as a\n> >>>>>>>>>>> new camera.\n> >>>>>>>>>>>\n> >>>>>>>>>>> - The logic in this patch is more complex due to it both having to deal\n> >>>>>>>>>>> with known and new cameras.\n> >>>>>>>>>>>\n> >>>>>>>>>>> What is the benefit of this cache that I'm missing?\n> >>>>>>>>>>\n> >>>>>>>>>> It's not clear whether Android requires it. It requires different\n> >>>>>>>>>> cameras to have different IDs, but I couldn't find a mention of how\n> >>>>>>>>>> identical cameras should be treated. The Chrome OS UVC HAL caches the\n> >>>>>>>>>> ID, I don't know if it's mandatory though.\n> >>>>>>>>>>\n> >>>>>>>>>> Shik, as you've worked on the UVC HAL, would you happen to know if there\n> >>>>>>>>>> are requirements to keep the same ID when a camera is unplugged and\n> >>>>>>>>>> replugged ?\n> >>>>>>>>\n> >>>>>>>> It's for persistency in settings UI. Chrome uses the camera id as the key of\n> >>>>>>>> the default camera in chrome://settings/content/camera. Reuse the same id for\n> >>>>>>>> the same camera so the default camera would be changed back after replug.\n> >>>>>>>>\n> >>>>>>>> It's introduced in this CL: https://crrev.com/c/1608985\n> >>>>>>>\n> >>>>>>> Thank you for the information. We have a similar caching system, which\n> >>>>>>> however suffers from two limitations:\n> >>>>\n> >>>> We have more limitations I'm afraid :-(\n> >>>>\n> >>>> If I understand Shik correctly the numerical camera ID is stored\n> >>>> persistently in the camera settings UI. Our code does not guarantee that\n> >>>> the numerical ID in the libcamera HAL is persistent. The cameras in the\n> >>>> CameraManager is stored in a vector and is therefor susceptible to be\n> >>>> enumerated in different order each time the HAL is started.\n> >>>>\n> >>>> We could of course remedy this by sorting the camera vecotr before\n> >>>> assigning numerical IDs. But this would still not be guaranteed to be\n> >>>> persistent as more/less cameras registered in the HAL could alter the\n> >>>> read out order. If we really wish to do this should we save/load the\n> >>>> camera ID to numerical ID mapping to file (not nice)?\n> >>>\n> >>> Shik, does the ID cache need to be persistent across reboots (or\n> >>> restarts of the camera service) ?\n> >>\n> >> For built-in cameras, yes. They need to be persistent across reboots and order\n> >> the camera ids by facing. There is a common implicit assumption in many\n> >> Android camera apps that assume the front camera is 0 and back camera is 1.\n> >\n> > One may wish this would be documented in the HAL API ;-)\n> >\n> >> For external cameras, ideally yes. But for most of the use cases currently,\n> >> it's acceptable to use a solution that only persists the ids in a boot cycle,\n> >> and always assign the same id for the first external camera after boots.\n> >\n> > There are conflicting requirements there. We can't assign the same ID\n> > for the first external camera after boot (we use 1000 today) and also\n> > have stable IDs across boots.\n> \n> These two are not the requirements that need to be held for the same time.\n> Assigning the same ID for the first external camera is a workaround and an\n> approximation for the ideal solution, which works fine when there is only one\n> external camera.\n\nAh right, I had misunderstood you, sorry.\n\n> >>>>>>> - If a camera is unplugged, and a different camera with the same VID:PID\n> >>>>>>> is plugged into the same USB port, the new camera be given the same ID\n> >>>>>>> as the previous camera.\n> >>>>>>\n> >>>>>> The current CrOS implementation will also use the same id in this case, and I\n> >>>>>> think it should be fine.\n> >>>>>>\n> >>>>>>> - If a camera is unplugged and replugged in a different USB port, it\n> >>>>>>> will be given a different ID.\n> >>>>>>\n> >>>>>> Hmm this looks non-ideal and may confuse the users.\n> >>>>>\n> >>>>> We will try to fix that on top\n> >>>>>\n> >>>>> This isn't trivial to handle, and there are corner cases. I'm thinking\n> >>>>> in particular about a system where two identical external USB cameras\n> >>>>> would be set up to point at particular locations. How to handle cameras\n> >>>>> stable IDs in that case is debatable, and I don't think the problem can\n> >>>>> be solved unless the cameras report distinct serial numbers (and most\n> >>>>> webcams don't :-S). At the moment, libcamera identifies each camera with\n> >>>>> a stable ID (guaranteed to be unique and stable across reboots, as long\n> >>>>> as the hardware and firmware are not modified) made of the concatenation\n> >>>>> of\n> >>>>>\n> >>>>> - The USB controller DT or ACPI path\n> >>>>> - The USB port(s) number(s) (starting at the root hub, going through\n> >>>>> every hub)\n> >>>>> - The USB device VID:PID\n> >>>>>\n> >>>>> The Android ID is then assigned based on the libcamera ID, with a cache\n> >>>>> to ensure that the same HAL ID will be given to the same libcamera ID.\n> >>>>> If we were to drop the USB port number, we'd have to find another way to\n> >>>>> create a stable and unique camera ID.\n> >>>>>\n> >>>>>>> Is that acceptable from a Chrome OS point of view ?\n> >>>>>>>\n> >>>>>>> Would you happen to know how Android uses camera IDs, and whether it\n> >>>>>>> requires or could benefit from ID caching ?\n> >>>>>>\n> >>>>>> Each camera app could have a similar preference caching system like the\n> >>>>>> settings UI, so it would be better to reuse the same id for the same camera.\n> >>>>>> AFAIK there is no such requirement at the framework level.\n> >>>>\n> >>>> This worries me a bit. Are we not altering the libcamera HAL to fit a\n> >>>> single use-case, all be it an important user. But if the framework does\n> >>>> not document an expected behavior is it not tricky to implement\n> >>>> correctly? What if we later find a different usage pattern in another\n> >>>> application which conflicts with what we are trying to do here?\n> >>>\n> >>> This is an undocumented area, so HALs can do pretty much what they want.\n> >>> If different platforms have different requirements, then we'll have to\n> >>> deal with it, possibly working with the platform vendors to align the\n> >>> requirements.\n> >>\n> >> Yes and we need to live with Hyrum's Law :P\n> >> If it's not handled here, we would still need to have another id mapping layer\n> >> somewhere else.\n> >\n> > That may be needed down the line though. As Chrome OS uses the Android\n> > camera HAL API, if differences in requirements are found between the two\n> > system, supporting the Chrome OS requirements in the Chrome OS camera\n> > service, on top of the HAL, may be the best option. We'll see when we'll\n> > run into actual differences.\n> \n> As long as the HAL provides enough information for the upper layer, I am fine\n> with either approach. We can do some experiments and see which one fits better.\n> \n> Note that some extra fields are exposed through vendor tags currently, for\n> various reasons:\n> 1. The human readable camera name\n> 2. USB vendor id and product id\n> 3. The device path\n> \n> Those are lessons and requirements we learnt from migrating the camera stack of\n> Chrome on Chrome OS to the new Android HALv3 Camera API based service. Hope\n> this information would help :)\n\nIt is, thank you.\n\nThe approach taken by libcamera today is to expose both a stable ID (in\nthe form of an opaque string, not human-readable) and enough information\nto construct a human-readable camera name. The human-readable camera\nname itself isn't constructed by libcamera, in order to let the user\nlocalize it (e.g. display \"Caméra intégrée arrière\" instead of\n\"Integrated back camera\"). Not all the required information is currently\nexposed, we're still working on it.\n\nOnce this is in place, we can create vendor-specific tags in the HAL\nimplementation to expose this to the camera service. We could construct\nthe same vendor tags as the USB HAL exposes today, but it could also\nmake sense to expose the libcamera information instead and construct the\ndata that Chrome OS requires in the camera service to avoid hardcoding\nChrome OS policies (such as how to construct a human readable name) in\nlibcamera. We can revisit this topic once the necessary infrastructure\nwill be available in libcamera.\n\n> >>>>>>>>>>>> CameraDevice is now a shared object and cameras_ vector stores shared\n> >>>>>>>>>>>> pointers to CameraDevice. This is done in order to introduce reference\n> >>>>>>>>>>>> counting for CameraDevice objects - especially to handle hot-unplug\n> >>>>>>>>>>>> events. Both camerasMap_ and cameras_ are protected by a mutex.\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> Signed-off-by: Umang Jain \n> >>>>>>>>>>>> ---\n> >>>>>>>>>>>> src/android/camera_device.cpp | 15 +++\n> >>>>>>>>>>>> src/android/camera_device.h | 8 +-\n> >>>>>>>>>>>> src/android/camera_hal_manager.cpp | 153 ++++++++++++++++++++++++-----\n> >>>>>>>>>>>> src/android/camera_hal_manager.h | 15 ++-\n> >>>>>>>>>>>> 4 files changed, 166 insertions(+), 25 deletions(-)\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> >>>>>>>>>>>> index d918350..a79bb69 100644\n> >>>>>>>>>>>> --- a/src/android/camera_device.cpp\n> >>>>>>>>>>>> +++ b/src/android/camera_device.cpp\n> >>>>>>>>>>>> @@ -233,6 +233,21 @@ CameraDevice::~CameraDevice()\n> >>>>>>>>>>>> delete it.second;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> +std::shared_ptr CameraDevice::create(unsigned int id,\n> >>>>>>>>>>>> + const std::shared_ptr &cam)\n> >>>>>>>>>>>> +{\n> >>>>>>>>>>>> + struct Deleter : std::default_delete {\n> >>>>>>>>>>>> + void operator()(CameraDevice *camera)\n> >>>>>>>>>>>> + {\n> >>>>>>>>>>>> + delete camera;\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>> + };\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + CameraDevice *camera = new CameraDevice(id, cam);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + return std::shared_ptr(camera, Deleter());\n> >>>>>>>>>>>> +}\n> >>>>>>>>>>>\n> >>>>>>>>>>> As Kieran points out I think this should be added in a separate as this\n> >>>>>>>>>>> one is quiet large and therefore hard to review.\n> >>>>>>>>>>>\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> /*\n> >>>>>>>>>>>> * Initialize the camera static information.\n> >>>>>>>>>>>> * This method is called before the camera device is opened.\n> >>>>>>>>>>>> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> >>>>>>>>>>>> index 7be9e11..7f9e010 100644\n> >>>>>>>>>>>> --- a/src/android/camera_device.h\n> >>>>>>>>>>>> +++ b/src/android/camera_device.h\n> >>>>>>>>>>>> @@ -47,8 +47,8 @@ struct CameraStream {\n> >>>>>>>>>>>> class CameraDevice : protected libcamera::Loggable\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> public:\n> >>>>>>>>>>>> - CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> >>>>>>>>>>>> - ~CameraDevice();\n> >>>>>>>>>>>> + static std::shared_ptr create(unsigned int id,\n> >>>>>>>>>>>> + const std::shared_ptr &cam);\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> int initialize();\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> @@ -57,6 +57,7 @@ public:\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> unsigned int id() const { return id_; }\n> >>>>>>>>>>>> camera3_device_t *camera3Device() { return &camera3Device_; }\n> >>>>>>>>>>>> + const libcamera::Camera *getCamera() { return camera_.get(); };\n> >>>>>>>>>>>\n> >>>>>>>>>>> Same here this can be done in a separate commit. Also I think this could\n> >>>>>>>>>>> be named camera() instead of getCamera()\n> >>>>>>>>>>>\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> int facing() const { return facing_; }\n> >>>>>>>>>>>> int orientation() const { return orientation_; }\n> >>>>>>>>>>>> @@ -72,6 +73,9 @@ protected:\n> >>>>>>>>>>>> std::string logPrefix() const override;\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> private:\n> >>>>>>>>>>>> + CameraDevice(unsigned int id, const std::shared_ptr &camera);\n> >>>>>>>>>>>> + ~CameraDevice();\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> struct Camera3RequestDescriptor {\n> >>>>>>>>>>>> Camera3RequestDescriptor(unsigned int frameNumber,\n> >>>>>>>>>>>> unsigned int numBuffers);\n> >>>>>>>>>>>> diff --git a/src/android/camera_hal_manager.cpp b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>>>> index 3d6d2b4..fdde2c0 100644\n> >>>>>>>>>>>> --- a/src/android/camera_hal_manager.cpp\n> >>>>>>>>>>>> +++ b/src/android/camera_hal_manager.cpp\n> >>>>>>>>>>>> @@ -8,6 +8,7 @@\n> >>>>>>>>>>>> #include \"camera_hal_manager.h\"\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> #include \n> >>>>>>>>>>>> +#include \n> >>>>>>>>>>>>\n> >>>>>>>>>>>> #include \"libcamera/internal/log.h\"\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> @@ -35,6 +36,7 @@ CameraHalManager::CameraHalManager()\n> >>>>>>>>>>>> CameraHalManager::~CameraHalManager()\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> cameras_.clear();\n> >>>>>>>>>>>> + camerasMap_.clear();\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> if (cameraManager_) {\n> >>>>>>>>>>>> cameraManager_->stop();\n> >>>>>>>>>>>> @@ -47,6 +49,13 @@ int CameraHalManager::init()\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> cameraManager_ = new CameraManager();\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> + /* Support camera hotplug. */\n> >>>>>>>>>>>> + cameraManager_->cameraAdded.connect(this, &CameraHalManager::cameraAdded);\n> >>>>>>>>>>>> + cameraManager_->cameraRemoved.connect(this, &CameraHalManager::cameraRemoved);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + cameraCounter_ = 0;\n> >>>>>>>>>>>> + externalCameraCounter_ = 1000;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> int ret = cameraManager_->start();\n> >>>>>>>>>>>> if (ret) {\n> >>>>>>>>>>>> LOG(HAL, Error) << \"Failed to start camera manager: \"\n> >>>>>>>>>>>> @@ -56,35 +65,25 @@ int CameraHalManager::init()\n> >>>>>>>>>>>> return ret;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> - /*\n> >>>>>>>>>>>> - * For each Camera registered in the system, a CameraDevice\n> >>>>>>>>>>>> - * gets created here to wraps a libcamera Camera instance.\n> >>>>>>>>>>>> - *\n> >>>>>>>>>>>> - * \\todo Support camera hotplug.\n> >>>>>>>>>>>> - */\n> >>>>>>>>>>>> - unsigned int index = 0;\n> >>>>>>>>>>>> - for (auto &cam : cameraManager_->cameras()) {\n> >>>>>>>>>>>> - CameraDevice *camera = new CameraDevice(index, cam);\n> >>>>>>>>>>>> - ret = camera->initialize();\n> >>>>>>>>>>>> - if (ret)\n> >>>>>>>>>>>> - continue;\n> >>>>>>>>>>>> -\n> >>>>>>>>>>>> - cameras_.emplace_back(camera);\n> >>>>>>>>>>>> - ++index;\n> >>>>>>>>>>>> - }\n> >>>>>>>>>>>> -\n> >>>>>>>>>>>> return 0;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> CameraDevice *CameraHalManager::open(unsigned int id,\n> >>>>>>>>>>>> const hw_module_t *hardwareModule)\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> - if (id >= numCameras()) {\n> >>>>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> >>>>>>>>>>>> + [id](std::shared_ptr &cam) {\n> >>>>>>>>>>>> + return cam->id() == id;\n> >>>>>>>>>>>> + });\n> >>>>>>>>>>>> + if (iter == cameras_.end()) {\n> >>>>>>>>>>>\n> >>>>>>>>>>> I think you should break this (and the similar ones below) into private\n> >>>>>>>>>>> helpers instead if implementing the logic in-place\n> >>>>>>>>>>>\n> >>>>>>>>>>> CameraHalManager::cameraFromAndroidId(..);\n> >>>>>>>>>>> CameraHalManager::androidIdFromCamera(..);\n> >>>>>>>>>>>\n> >>>>>>>>>>>> LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> >>>>>>>>>>>> return nullptr;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> - CameraDevice *camera = cameras_[id].get();\n> >>>>>>>>>>>> + CameraDevice *camera = iter->get();\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> if (camera->open(hardwareModule))\n> >>>>>>>>>>>> return nullptr;\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> @@ -93,9 +92,91 @@ CameraDevice *CameraHalManager::open(unsigned int id,\n> >>>>>>>>>>>> return camera;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> +void CameraHalManager::cameraAdded(std::shared_ptr cam)\n> >>>>>>>>>>>> +{\n> >>>>>>>>>>>> + unsigned int id;\n> >>>>>>>>>>>> + bool isCameraExternal = false;\n> >>>>>>>>>>>> + bool isCameraNew = false;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + /* Each camera is assigned a unique integer id when it is seen for the\n> >>>>>>>>>>>> + * first time. If the camera has been seen before, the id is reused and\n> >>>>>>>>>>>> + * the camera is marked as CAMERA_DEVICE_STATUS_PRESENT subsequently.\n> >>>>>>>>>>>> + *\n> >>>>>>>>>>>> + * ID starts from '0' for internal cameras and '1000' for external cameras.\n> >>>>>>>>>>>> + */\n> >>>>>>>>>>>> + auto iter = camerasMap_.find(cam->id());\n> >>>>>>>>>>>> + if (iter != camerasMap_.end()) {\n> >>>>>>>>>>>> + id = iter->second;\n> >>>>>>>>>>>> + } else {\n> >>>>>>>>>>>> + isCameraNew = true;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + /*\n> >>>>>>>>>>>> + * Now check if this is an external camera and assign\n> >>>>>>>>>>>> + * its id accordingly.\n> >>>>>>>>>>>> + */\n> >>>>>>>>>>>> + const ControlList &properties = cam->properties();\n> >>>>>>>>>>>> + if (properties.contains(properties::Location) &&\n> >>>>>>>>>>>> + properties.get(properties::Location) &\n> >>>>>>>>>>>> + properties::CameraLocationExternal) {\n> >>>>>>>>>>>> + isCameraExternal = true;\n> >>>>>>>>>>>> + id = externalCameraCounter_;\n> >>>>>>>>>>>> + } else {\n> >>>>>>>>>>>> + id = cameraCounter_;\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>\n> >>>>>>>>>>> As I understand it the information the HAL wants is whether or not the\n> >>>>>>>>>>> camera can be hot-plugged (id >= 1000) or nor (id < 1000)?\n> >>>>>>>>>>>\n> >>>>>>>>>>> If so I wonder if using the camera location as a judgment of the camera\n> >>>>>>>>>>> is hot-plugged or not is the way to go here? Imagine a device where the\n> >>>>>>>>>>> camera is permanently attached (not hot-unpluggable) but not fixated in\n> >>>>>>>>>>> a location. I'm thinking cameras mounted at the end of instruments\n> >>>>>>>>>>> (medical instruments, hand held tools) or robotics (mounted at the arm\n> >>>>>>>>>>> of a welding robot). I would imagine those cameras would be marked as\n> >>>>>>>>>>> located externally but they would not really be hot-pluggable.\n> >>>>>>>>>>>\n> >>>>>>>>>>> I understand we have no other way to report or detect this at the moment\n> >>>>>>>>>>> and I'm not pushing hard for this to be solved as part of this series if\n> >>>>>>>>>>> it's not easy. But I think a bigger comment here is needed explaining\n> >>>>>>>>>>> that the HAL wants to know if a camera is hot-pluggable or not and does\n> >>>>>>>>>>> not really care if it's located internally or externally. I also think a\n> >>>>>>>>>>> \\todo should be added so it's not forgotten.\n> >>>>>>>>>>>\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + /*\n> >>>>>>>>>>>> + * For each Camera registered in the system, a CameraDevice\n> >>>>>>>>>>>> + * gets created here to wraps a libcamera Camera instance.\n> >>>>>>>>>>>> + */\n> >>>>>>>>>>>> + std::shared_ptr camera = CameraDevice::create(id, std::move(cam));\n> >>>>>>>>>>>> + int ret = camera->initialize();\n> >>>>>>>>>>>> + if (ret) {\n> >>>>>>>>>>>> + LOG(HAL, Error) << \"Failed to initialize camera: \" << cam->id();\n> >>>>>>>>>>>> + return;\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + if (isCameraNew) {\n> >>>>>>>>>>>> + camerasMap_.emplace(cam->id(), id);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + if (isCameraExternal)\n> >>>>>>>>>>>> + externalCameraCounter_++;\n> >>>>>>>>>>>> + else\n> >>>>>>>>>>>> + cameraCounter_++;\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + cameras_.emplace_back(std::move(camera));\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + if (callbacks_)\n> >>>>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> >>>>>>>>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n> >>>>>>>>>>>> + LOG(HAL, Debug) << \"Camera ID: \" << id << \" added successfully.\";\n> >>>>>>>>>>>> +}\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> +void CameraHalManager::cameraRemoved(std::shared_ptr cam)\n> >>>>>>>>>>>> +{\n> >>>>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> >>>>>>>>>>>> + [cam](std::shared_ptr &camera) {\n> >>>>>>>>>>>> + return cam.get() == camera->getCamera();\n> >>>>>>>>>>>> + });\n> >>>>>>>>>>>> + if (iter == cameras_.end())\n> >>>>>>>>>>>> + return;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + unsigned int id = (*iter)->id();\n> >>>>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> >>>>>>>>>>>> + CAMERA_DEVICE_STATUS_NOT_PRESENT);\n> >>>>>>>>>>>> + cameras_.erase(iter);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + LOG(HAL, Debug) << \"Camera ID: \" << id << \" removed successfully.\";\n> >>>>>>>>>>>> +}\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> unsigned int CameraHalManager::numCameras() const\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> - return cameraManager_->cameras().size();\n> >>>>>>>>>>>> + return cameraCounter_;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> >>>>>>>>>>>> @@ -103,12 +184,18 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> >>>>>>>>>>>> if (!info)\n> >>>>>>>>>>>> return -EINVAL;\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> - if (id >= numCameras()) {\n> >>>>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + auto iter = std::find_if(cameras_.begin(), cameras_.end(),\n> >>>>>>>>>>>> + [id](std::shared_ptr &cam) {\n> >>>>>>>>>>>> + return cam->id() == id;\n> >>>>>>>>>>>> + });\n> >>>>>>>>>>>> + if (iter == cameras_.end()) {\n> >>>>>>>>>>>> LOG(HAL, Error) << \"Invalid camera id '\" << id << \"'\";\n> >>>>>>>>>>>> return -EINVAL;\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> - CameraDevice *camera = cameras_[id].get();\n> >>>>>>>>>>>> + CameraDevice *camera = iter->get();\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> info->facing = camera->facing();\n> >>>>>>>>>>>> info->orientation = camera->orientation();\n> >>>>>>>>>>>> @@ -124,4 +211,26 @@ int CameraHalManager::getCameraInfo(unsigned int id, struct camera_info *info)\n> >>>>>>>>>>>> void CameraHalManager::setCallbacks(const camera_module_callbacks_t *callbacks)\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> callbacks_ = callbacks;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + MutexLocker locker(mutex_);\n> >>>>>>>>>>>> + /*\n> >>>>>>>>>>>> + * Few cameras might have been hotplugged before setting callbacks_ here.\n> >>>>>>>>>>>> + * We need to mark CAMERA_DEVICE_STATUS_PRESENT for them explicitly.\n> >>>>>>>>>>>> + * This hold only for external cameras, as internal cameras are assumed to\n> >>>>>>>>>>>> + * be present at module load time, by the framework.\n> >>>>>>>>>>>> + */\n> >>>>>>>>>>>> + for (auto &cam : cameraManager_->cameras()) {\n> >>>>>>>>>>>> + auto iter = camerasMap_.find(cam->id());\n> >>>>>>>>>>>> + if (iter == camerasMap_.end())\n> >>>>>>>>>>>> + continue;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + unsigned int id = iter->second;\n> >>>>>>>>>>>> + const ControlList &properties = cam->properties();\n> >>>>>>>>>>>> + if (properties.contains(properties::Location) &&\n> >>>>>>>>>>>> + properties.get(properties::Location) &\n> >>>>>>>>>>>> + properties::CameraLocationExternal) {\n> >>>>>>>>>>>> + callbacks_->camera_device_status_change(callbacks_, id,\n> >>>>>>>>>>>> + CAMERA_DEVICE_STATUS_PRESENT);\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>> + }\n> >>>>>>>>>>>> }\n> >>>>>>>>>>>> diff --git a/src/android/camera_hal_manager.h b/src/android/camera_hal_manager.h\n> >>>>>>>>>>>> index a582f04..7c481d4 100644\n> >>>>>>>>>>>> --- a/src/android/camera_hal_manager.h\n> >>>>>>>>>>>> +++ b/src/android/camera_hal_manager.h\n> >>>>>>>>>>>> @@ -7,6 +7,8 @@\n> >>>>>>>>>>>> #ifndef __ANDROID_CAMERA_MANAGER_H__\n> >>>>>>>>>>>> #define __ANDROID_CAMERA_MANAGER_H__\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> +#include \n> >>>>>>>>>>>> +#include \n> >>>>>>>>>>>> #include \n> >>>>>>>>>>>> #include \n> >>>>>>>>>>>>\n> >>>>>>>>>>>> @@ -18,6 +20,9 @@\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> class CameraDevice;\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> +using Mutex = std::mutex;\n> >>>>>>>>>>>> +using MutexLocker = std::unique_lock;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> class CameraHalManager\n> >>>>>>>>>>>> {\n> >>>>>>>>>>>> public:\n> >>>>>>>>>>>> @@ -33,10 +38,18 @@ public:\n> >>>>>>>>>>>> void setCallbacks(const camera_module_callbacks_t *callbacks);\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> private:\n> >>>>>>>>>>>> + void cameraAdded(std::shared_ptr cam);\n> >>>>>>>>>>>> + void cameraRemoved(std::shared_ptr cam);\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> libcamera::CameraManager *cameraManager_;\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> const camera_module_callbacks_t *callbacks_;\n> >>>>>>>>>>>> - std::vector> cameras_;\n> >>>>>>>>>>>> + std::vector> cameras_;\n> >>>>>>>>>>>> + std::map camerasMap_;\n> >>>>>>>>>>>\n> >>>>>>>>>>> If each hot-plugged camera where treated as a new camera cameras_ and\n> >>>>>>>>>>> camerasMap_ could be merged to a\n> >>>>>>>>>>>\n> >>>>>>>>>>> std::map> cameras_;\n> >>>>>>>>>>>\n> >>>>>>>>>>> Which would eliminate the possibility of them going out-of-sync.\n> >>>>>>>>>>>\n> >>>>>>>>>>>> + Mutex mutex_;\n> >>>>>>>>>>>> +\n> >>>>>>>>>>>> + unsigned int externalCameraCounter_;\n> >>>>>>>>>>>> + unsigned int cameraCounter_;\n> >>>>>>>>>>>> };\n> >>>>>>>>>>>>\n> >>>>>>>>>>>> #endif /* __ANDROID_CAMERA_MANAGER_H__ */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C06B2BE173\n\tfor ;\n\tThu, 20 Aug 2020 15:32:14 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 3BD4162082;\n\tThu, 20 Aug 2020 17:32:14 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E4E7160381\n\tfor ;\n\tThu, 20 Aug 2020 17:32:12 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 65392329;\n\tThu, 20 Aug 2020 17:32:12 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"fU6PvvZN\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597937532;\n\tbh=hvnYCt2geU07wC8M61cSc0EjVQNR02Tn7Kqt53WSB7Y=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=fU6PvvZN9DeDjs4t2VDcsQbivNuQ3qMoh82CqnX8Jul4ExAdcmvE6rYvVjbPbF0Gq\n\t/DWIWFjOrIItZ+WxTfns/De+fMaO7du2liC3MKr/PO+S+aQSoTJs51te7z8YVecHze\n\ta8kgMdcw2XUg6BaxUe5d+FRrm+qaAQpGMzuzaHKI=","Date":"Thu, 20 Aug 2020 18:31:54 +0300","From":"Laurent Pinchart ","To":"Shik Chen ","Message-ID":"<20200820153154.GP6593@pendragon.ideasonboard.com>","References":"<20200818083615.GA6029@pendragon.ideasonboard.com>\n\t\n\t<20200819155221.GP6049@pendragon.ideasonboard.com>\n\t\n\t<20200819162140.GQ6049@pendragon.ideasonboard.com>\n\t<20200820071050.GB39265@wyvern>\n\t<20200820101028.GA6593@pendragon.ideasonboard.com>\n\t\n\t<20200820144215.GH6593@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 3/4] android: camera_hal_manager:\n\tSupport camera hotplug","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]