{"id":16504,"url":"https://patchwork.libcamera.org/api/1.1/patches/16504/?format=json","web_url":"https://patchwork.libcamera.org/patch/16504/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220701084521.31831-12-tomi.valkeinen@ideasonboard.com>","date":"2022-07-01T08:45:15","name":"[libcamera-devel,v3,11/17] py: New event handling","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"15e501071e5887eaf007a49db9d860352e0470a9","submitter":{"id":109,"url":"https://patchwork.libcamera.org/api/1.1/people/109/?format=json","name":"Tomi Valkeinen","email":"tomi.valkeinen@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/16504/mbox/","series":[{"id":3241,"url":"https://patchwork.libcamera.org/api/1.1/series/3241/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3241","date":"2022-07-01T08:45:04","name":"Python bindings event handling","version":3,"mbox":"https://patchwork.libcamera.org/series/3241/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/16504/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/16504/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4A889BD808\n\tfor ;\n\tFri, 1 Jul 2022 08:45:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0517B656AA;\n\tFri, 1 Jul 2022 10:45:56 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F407265652\n\tfor ;\n\tFri, 1 Jul 2022 10:45:45 +0200 (CEST)","from deskari.lan (91-158-154-79.elisa-laajakaista.fi\n\t[91.158.154.79])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 74E0725C;\n\tFri, 1 Jul 2022 10:45:45 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1656665156;\n\tbh=/YkbYWzQtsOn9IeurNkp3sgIkzOM+11dUkx+At5lrSU=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=D/Ujq4GO06hgQtQ9QGI65W4BJnFeyOMHb9hEZeKEadTiJrnYOGjXHcPx7iYoeh1EN\n\tCYY5FhhPyOkpFjZ0TywUJuTCHAQi2OhDO7KYENiIXKTlCAuANKDqK+W/APjwVGrgGd\n\tLHDshrtjgip0NGpLWw1kdz/WFzGMPFFisLoHqeIZhCTcINOrs9CttH8oWRHf8kgoV4\n\tFS+tr0Btc6FsDrF7GbdKqEDxxkjPhjQcUlK0KE6CAfFQXLLjpVbGZU4aP4Vz9x4rOu\n\tulwBPlJDgmzmn2BaWO4e17MTczsTsgN4N5rx1ikK138nvLdYLLJlZgoFYskc5zH1uA\n\txCbRPwZWnyDtA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1656665145;\n\tbh=/YkbYWzQtsOn9IeurNkp3sgIkzOM+11dUkx+At5lrSU=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=vHNdKkBpuugy+5Po2MIvTxankoLgX9qazH4VxlUj0nDNnlvjz57J9Syc0CjAmQ+6E\n\tYnPWHHifctGF+CC8U0Gw9pbO8kag9LKWeaSlVDrc88wncHkp8oYU8fhglf3Rg5QHTe\n\tizdr+qxxwJg9xqUFpmcRl0refcd6Ol/hgPznYytI="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"vHNdKkBp\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org,\n\tDavid Plowman ,\n\tKieran Bingham ,\n\tLaurent Pinchart ,\n\tJacopo Mondi ","Date":"Fri, 1 Jul 2022 11:45:15 +0300","Message-Id":"<20220701084521.31831-12-tomi.valkeinen@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20220701084521.31831-1-tomi.valkeinen@ideasonboard.com>","References":"<20220701084521.31831-1-tomi.valkeinen@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 11/17] py: New event handling","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Tomi Valkeinen via libcamera-devel\n\t","Reply-To":"Tomi Valkeinen ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"At the moment the Python bindings only handle the requestCompleted\nevents. But we have others, bufferCompleted and disconnect from the\nCamera, and cameraAdded and cameraRemoved from the CameraManager.\n\nThis patch makes all those events available to the users.\n\nThe get_ready_requests() method is now deprecated, but available to keep\nbackward compatibility.\n\nThe new event handling works like get_ready_requests(), but instead of\nreturning only Requests from requestCompleted events, it returns all\nevent types using a new Event class.\n\nAdditionally camera.stop() has been changed to return events for that\ncamera. This serves two purposes: first, it removes the requestCompleted\nevents from the event queue, thus preventing the old events being\nreturned when the camera is started again, and second, it allows the\nuser to process those events if it so wants.\n\nAll other event types are always collected and returned, except\nbufferCompleted which needs to be enabled by setting the\nCameraManager.buffer_completed_active to True. This is to reduce the\noverhead a bit. It is not clear if this is a necessary optimization or\nnot.\n\nTODO: Documentation.\n\nSigned-off-by: Tomi Valkeinen \n---\n src/py/libcamera/py_camera_manager.cpp | 191 +++++++++++++++++++++++--\n src/py/libcamera/py_camera_manager.h | 64 ++++++++-\n src/py/libcamera/py_main.cpp | 45 +++++-\n 3 files changed, 276 insertions(+), 24 deletions(-)","diff":"diff --git a/src/py/libcamera/py_camera_manager.cpp b/src/py/libcamera/py_camera_manager.cpp\nindex 3dd8668e..d1d63690 100644\n--- a/src/py/libcamera/py_camera_manager.cpp\n+++ b/src/py/libcamera/py_camera_manager.cpp\n@@ -5,6 +5,7 @@\n \n #include \"py_camera_manager.h\"\n \n+#include \n #include \n #include \n #include \n@@ -33,11 +34,17 @@ PyCameraManager::PyCameraManager()\n \tif (ret)\n \t\tthrow std::system_error(-ret, std::generic_category(),\n \t\t\t\t\t\"Failed to start CameraManager\");\n+\n+\tcameraManager_->cameraAdded.connect(this, &PyCameraManager::handleCameraAdded);\n+\tcameraManager_->cameraRemoved.connect(this, &PyCameraManager::handleCameraRemoved);\n }\n \n PyCameraManager::~PyCameraManager()\n {\n \tLOG(Python, Debug) << \"~PyCameraManager()\";\n+\n+\tcameraManager_->cameraAdded.disconnect();\n+\tcameraManager_->cameraRemoved.disconnect();\n }\n \n py::list PyCameraManager::cameras()\n@@ -58,6 +65,50 @@ py::list PyCameraManager::cameras()\n \treturn l;\n }\n \n+PyCameraEvent PyCameraManager::convertEvent(const CameraEvent &event)\n+{\n+\tPyCameraEvent pyevent;\n+\n+\tpyevent.type_ = event.type_;\n+\n+\t/*\n+\t * We need to set a keep-alive here so that the camera keeps the\n+\t * camera manager alive.\n+\t */\n+\tpy::object py_cm = py::cast(this);\n+\tpy::object py_cam = py::cast(event.camera_);\n+\tpy::detail::keep_alive_impl(py_cam, py_cm);\n+\n+\tpyevent.camera_ = py_cam;\n+\n+\tswitch (event.type_) {\n+\tcase CameraEventType::CameraAdded:\n+\tcase CameraEventType::CameraRemoved:\n+\tcase CameraEventType::Disconnect:\n+\t\t/* No additional parameters to add */\n+\t\tbreak;\n+\n+\tcase CameraEventType::BufferCompleted: {\n+\t\tpyevent.request_ = py::cast(event.request_);\n+\t\tpyevent.fb_ = py::cast(event.fb_);\n+\t\tbreak;\n+\t}\n+\tcase CameraEventType::RequestCompleted: {\n+\t\tpyevent.request_ = py::cast(event.request_);\n+\n+\t\t/* Decrease the ref increased in Camera.queue_request() */\n+\t\tpyevent.request_.dec_ref();\n+\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tASSERT(false);\n+\t}\n+\n+\treturn pyevent;\n+}\n+\n+/* DEPRECATED */\n std::vector PyCameraManager::getReadyRequests()\n {\n \tint ret = readFd();\n@@ -70,21 +121,125 @@ std::vector PyCameraManager::getReadyRequests()\n \n \tstd::vector py_reqs;\n \n-\tfor (Request *request : getCompletedRequests()) {\n-\t\tpy::object o = py::cast(request);\n-\t\t/* Decrease the ref increased in Camera.queue_request() */\n-\t\to.dec_ref();\n-\t\tpy_reqs.push_back(o);\n+\tfor (const auto &ev : getEvents()) {\n+\t\tif (ev.type_ != CameraEventType::RequestCompleted)\n+\t\t\tcontinue;\n+\n+\t\tPyCameraEvent pyev = convertEvent(ev);\n+\t\tpy_reqs.push_back(pyev.request_);\n \t}\n \n \treturn py_reqs;\n }\n \n+std::vector PyCameraManager::getPyEvents()\n+{\n+\tint ret = readFd();\n+\n+\tif (ret == EAGAIN) {\n+\t\tLOG(Python, Debug) << \"No events\";\n+\t\treturn {};\n+\t}\n+\n+\tif (ret != 0)\n+\t\tthrow std::system_error(ret, std::generic_category());\n+\n+\tstd::vector events = getEvents();\n+\n+\tLOG(Python, Debug) << \"Get \" << events.size() << \" events\";\n+\n+\tstd::vector pyevents(events.size());\n+\n+\tstd::transform(events.begin(), events.end(), pyevents.begin(),\n+\t\t [this](const CameraEvent &ev) {\n+\t\t\t return convertEvent(ev);\n+\t\t });\n+\n+\treturn pyevents;\n+}\n+\n+static bool isCameraSpecificEvent(const CameraEvent &event, std::shared_ptr &camera)\n+{\n+\treturn event.camera_ == camera &&\n+\t (event.type_ == CameraEventType::RequestCompleted ||\n+\t\tevent.type_ == CameraEventType::BufferCompleted ||\n+\t\tevent.type_ == CameraEventType::Disconnect);\n+}\n+\n+std::vector PyCameraManager::getPyCameraEvents(std::shared_ptr camera)\n+{\n+\tMutexLocker guard(eventsMutex_);\n+\n+\tstd::vector pyevents;\n+\n+\t/* Get events related to the given camera */\n+\n+\tfor (const auto &event : events_) {\n+\t\tif (!isCameraSpecificEvent(event, camera))\n+\t\t\tcontinue;\n+\n+\t\tPyCameraEvent pyev = convertEvent(event);\n+\t\tpyevents.push_back(pyev);\n+\t}\n+\n+\t/* Drop the events from the events_ list */\n+\n+\tevents_.erase(std::remove_if(events_.begin(), events_.end(),\n+\t\t[&camera](const CameraEvent &ev) {\n+\t\t\treturn isCameraSpecificEvent(ev, camera);\n+\t\t}),\n+\t\tevents_.end());\n+\n+\tLOG(Python, Debug) << \"Get \" << pyevents.size() << \" camera events, \"\n+\t\t\t << events_.size() << \" unhandled events left\";\n+\n+\treturn pyevents;\n+}\n+\n /* Note: Called from another thread */\n-void PyCameraManager::handleRequestCompleted(Request *req)\n+void PyCameraManager::handleBufferCompleted(std::shared_ptr cam, Request *req, FrameBuffer *fb)\n {\n-\tpushRequest(req);\n-\twriteFd();\n+\tif (!bufferCompletedEventActive_)\n+\t\treturn;\n+\n+\tCameraEvent ev(CameraEventType::BufferCompleted, cam);\n+\tev.request_ = req;\n+\tev.fb_ = fb;\n+\n+\tpushEvent(ev);\n+}\n+\n+/* Note: Called from another thread */\n+void PyCameraManager::handleRequestCompleted(std::shared_ptr cam, Request *req)\n+{\n+\tCameraEvent ev(CameraEventType::RequestCompleted, cam);\n+\tev.request_ = req;\n+\n+\tpushEvent(ev);\n+}\n+\n+/* Note: Called from another thread */\n+void PyCameraManager::handleDisconnected(std::shared_ptr cam)\n+{\n+\tCameraEvent ev(CameraEventType::Disconnect, cam);\n+\n+\tpushEvent(ev);\n+}\n+\n+/* Note: Called from another thread */\n+void PyCameraManager::handleCameraAdded(std::shared_ptr cam)\n+{\n+\tCameraEvent ev(CameraEventType::CameraAdded, cam);\n+\n+\tpushEvent(ev);\n+}\n+\n+/* Note: Called from another thread */\n+void PyCameraManager::handleCameraRemoved(std::shared_ptr cam)\n+{\n+\tCameraEvent ev(CameraEventType::CameraRemoved, cam);\n+\n+\tpushEvent(ev);\n }\n \n void PyCameraManager::writeFd()\n@@ -110,16 +265,22 @@ int PyCameraManager::readFd()\n \treturn 0;\n }\n \n-void PyCameraManager::pushRequest(Request *req)\n+void PyCameraManager::pushEvent(const CameraEvent &ev)\n {\n-\tMutexLocker guard(completedRequestsMutex_);\n-\tcompletedRequests_.push_back(req);\n+\tMutexLocker guard(eventsMutex_);\n+\tevents_.push_back(ev);\n+\n+\twriteFd();\n+\n+\tLOG(Python, Debug) << \"Queued events: \" << events_.size();\n }\n \n-std::vector PyCameraManager::getCompletedRequests()\n+std::vector PyCameraManager::getEvents()\n {\n-\tstd::vector v;\n-\tMutexLocker guard(completedRequestsMutex_);\n-\tswap(v, completedRequests_);\n+\tstd::vector v;\n+\n+\tMutexLocker guard(eventsMutex_);\n+\tswap(v, events_);\n+\n \treturn v;\n }\ndiff --git a/src/py/libcamera/py_camera_manager.h b/src/py/libcamera/py_camera_manager.h\nindex 3525057d..b313ba9b 100644\n--- a/src/py/libcamera/py_camera_manager.h\n+++ b/src/py/libcamera/py_camera_manager.h\n@@ -13,6 +13,47 @@\n \n using namespace libcamera;\n \n+enum class CameraEventType {\n+\tUndefined = 0,\n+\tCameraAdded,\n+\tCameraRemoved,\n+\tDisconnect,\n+\tRequestCompleted,\n+\tBufferCompleted,\n+};\n+\n+/*\n+ * This event struct is used internally to queue the events we receive from\n+ * other threads.\n+ */\n+struct CameraEvent {\n+\tCameraEvent(CameraEventType type, std::shared_ptr camera)\n+\t\t: type_(type), camera_(camera)\n+\t{\n+\t}\n+\n+\tCameraEventType type_ = CameraEventType::Undefined;\n+\n+\tstd::shared_ptr camera_;\n+\n+\tRequest *request_ = nullptr;\n+\tFrameBuffer *fb_ = nullptr;\n+};\n+\n+/*\n+ * This event struct is passed to Python. We need to use pybind11::object here\n+ * instead of a C++ pointer so that we keep a ref to the Request, and a\n+ * keep-alive from the camera to the camera manager.\n+ */\n+struct PyCameraEvent {\n+\tCameraEventType type_ = CameraEventType::Undefined;\n+\n+\tpybind11::object camera_;\n+\n+\tpybind11::object request_;\n+\tpybind11::object fb_;\n+};\n+\n class PyCameraManager\n {\n public:\n@@ -26,20 +67,29 @@ public:\n \n \tint eventFd() const { return eventFd_.get(); }\n \n-\tstd::vector getReadyRequests();\n+\tstd::vector getReadyRequests(); /* DEPRECATED */\n+\tstd::vector getPyEvents();\n+\tstd::vector getPyCameraEvents(std::shared_ptr camera);\n \n-\tvoid handleRequestCompleted(Request *req);\n+\tvoid handleBufferCompleted(std::shared_ptr cam, Request *req, FrameBuffer *fb);\n+\tvoid handleRequestCompleted(std::shared_ptr cam, Request *req);\n+\tvoid handleDisconnected(std::shared_ptr cam);\n+\tvoid handleCameraAdded(std::shared_ptr cam);\n+\tvoid handleCameraRemoved(std::shared_ptr cam);\n \n+\tbool bufferCompletedEventActive_ = false;\n private:\n \tstd::unique_ptr cameraManager_;\n \n \tUniqueFD eventFd_;\n-\tlibcamera::Mutex completedRequestsMutex_;\n-\tstd::vector completedRequests_\n-\t\tLIBCAMERA_TSA_GUARDED_BY(completedRequestsMutex_);\n+\tlibcamera::Mutex eventsMutex_;\n+\tstd::vector events_\n+\t\tLIBCAMERA_TSA_GUARDED_BY(eventsMutex_);\n \n \tvoid writeFd();\n \tint readFd();\n-\tvoid pushRequest(Request *req);\n-\tstd::vector getCompletedRequests();\n+\tvoid pushEvent(const CameraEvent &ev);\n+\tstd::vector getEvents();\n+\n+\tPyCameraEvent convertEvent(const CameraEvent &event);\n };\ndiff --git a/src/py/libcamera/py_main.cpp b/src/py/libcamera/py_main.cpp\nindex 6cd99919..b4f756d7 100644\n--- a/src/py/libcamera/py_main.cpp\n+++ b/src/py/libcamera/py_main.cpp\n@@ -58,6 +58,7 @@ PYBIND11_MODULE(_libcamera, m)\n \t * https://pybind11.readthedocs.io/en/latest/advanced/misc.html#avoiding-c-types-in-docstrings\n \t */\n \n+\tauto pyEvent = py::class_(m, \"Event\");\n \tauto pyCameraManager = py::class_(m, \"CameraManager\");\n \tauto pyCamera = py::class_(m, \"Camera\");\n \tauto pyCameraConfiguration = py::class_(m, \"CameraConfiguration\");\n@@ -90,6 +91,21 @@ PYBIND11_MODULE(_libcamera, m)\n \tm.def(\"log_set_level\", &logSetLevel);\n \n \t/* Classes */\n+\n+\tpy::enum_(pyEvent, \"Type\")\n+\t\t.value(\"Undefined\", CameraEventType::Undefined)\n+\t\t.value(\"CameraAdded\", CameraEventType::CameraAdded)\n+\t\t.value(\"CameraRemoved\", CameraEventType::CameraRemoved)\n+\t\t.value(\"Disconnect\", CameraEventType::Disconnect)\n+\t\t.value(\"RequestCompleted\", CameraEventType::RequestCompleted)\n+\t\t.value(\"BufferCompleted\", CameraEventType::BufferCompleted);\n+\n+\tpyEvent\n+\t\t.def_readonly(\"type\", &PyCameraEvent::type_)\n+\t\t.def_readonly(\"camera\", &PyCameraEvent::camera_)\n+\t\t.def_readonly(\"request\", &PyCameraEvent::request_)\n+\t\t.def_readonly(\"fb\", &PyCameraEvent::fb_);\n+\n \tpyCameraManager\n \t\t.def_static(\"singleton\", []() {\n \t\t\tstd::shared_ptr cm = gCameraManager.lock();\n@@ -107,7 +123,13 @@ PYBIND11_MODULE(_libcamera, m)\n \t\t.def_property_readonly(\"cameras\", &PyCameraManager::cameras)\n \n \t\t.def_property_readonly(\"event_fd\", &PyCameraManager::eventFd)\n-\t\t.def(\"get_ready_requests\", &PyCameraManager::getReadyRequests);\n+\n+\t\t/* DEPRECATED */\n+\t\t.def(\"get_ready_requests\", &PyCameraManager::getReadyRequests)\n+\n+\t\t.def(\"get_events\", &PyCameraManager::getPyEvents)\n+\n+\t\t.def_readwrite(\"buffer_completed_active\", &PyCameraManager::bufferCompletedEventActive_);\n \n \tpyCamera\n \t\t.def_property_readonly(\"id\", &Camera::id)\n@@ -131,7 +153,17 @@ PYBIND11_MODULE(_libcamera, m)\n \t\t\tauto cm = gCameraManager.lock();\n \t\t\tASSERT(cm);\n \n-\t\t\tself.requestCompleted.connect(cm.get(), &PyCameraManager::handleRequestCompleted);\n+\t\t\tself.requestCompleted.connect(&self, [cm, camera=self.shared_from_this()](Request *req) {\n+\t\t\t\tcm->handleRequestCompleted(camera, req);\n+\t\t\t});\n+\n+\t\t\tself.bufferCompleted.connect(&self, [cm, camera=self.shared_from_this()](Request *req, FrameBuffer *fb) {\n+\t\t\t\tcm->handleBufferCompleted(camera, req, fb);\n+\t\t\t});\n+\n+\t\t\tself.disconnected.connect(&self, [cm, camera=self.shared_from_this()]() {\n+\t\t\t\tcm->handleDisconnected(camera);\n+\t\t\t});\n \n \t\t\tControlList controlList(self.controls());\n \n@@ -152,11 +184,20 @@ PYBIND11_MODULE(_libcamera, m)\n \t\t\tint ret = self.stop();\n \n \t\t\tself.requestCompleted.disconnect();\n+\t\t\tself.bufferCompleted.disconnect();\n+\t\t\tself.disconnected.disconnect();\n+\n+\t\t\tauto cm = gCameraManager.lock();\n+\t\t\tASSERT(cm);\n+\n+\t\t\tauto l = cm->getPyCameraEvents(self.shared_from_this());\n \n \t\t\t/* \\todo Should we just ignore the error? */\n \t\t\tif (ret)\n \t\t\t\tthrow std::system_error(-ret, std::generic_category(),\n \t\t\t\t\t\t\t\"Failed to start camera\");\n+\n+\t\t\treturn l;\n \t\t})\n \n \t\t.def(\"__str__\", [](Camera &self) {\n","prefixes":["libcamera-devel","v3","11/17"]}