[{"id":13719,"web_url":"https://patchwork.libcamera.org/comment/13719/","msgid":"<15de198b-1f9f-f6d0-d98d-c054978d2d29@ideasonboard.com>","date":"2020-11-16T11:47:49","subject":"Re: [libcamera-devel] [PATCH v1 0/5] cam: Move request processing\n\tto main thread","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Laurent,\n\nOn 13/11/2020 06:38, Laurent Pinchart wrote:\n> Hello,\n> \n> This patch series improves the cam application by moving request\n> processing to the main thread to avoid blocking the camera manager\n> thread (in which the request completion signal is emitted) for extended\n> periods of time. This helps meeting the realtime constraints of pipeline\n> handlers.\n> \n> Note that move requesting processing to a different thread isn't a\n> magical solution, if the average processing time for a request is larger\n> than the frame interval, a request queue underrun will still occur. It\n> however helps handling random delays in request processing.\n> \n> Included in the series is a move to libevent to implement the cam event\n> loop. The event loop is currently based on the libcamera event\n> dispatcher for historical reasons. The event dispatcher API exposed by\n> libcamera is incomplete, as thread support is internal to libcamera.\n> Instead of reimplementing the missing parts in the cam event loop, or\n> exposing thread support in the libcamera public API, moving to libevent\n> allows making the event dispatcher implementation fully private (in\n> patch 3/5). It isn't the job of libcamera to provide such generic APIs.\n\nThis series breaks simple-cam.\n\nWill you plan to update that too?\nIf not I'll handle it, just let me know.\n\n--\nThanks\n\nKieran\n\n\n> Laurent Pinchart (5):\n> test: Get event dispatcher from current thread\n> cam: Use libevent to implement event loop\n> libcamera: Move EventDispatcher to internal API\n> cam: event_loop: Add deferred calls support\n> cam: Move request processing to main thread\n> \n> include/libcamera/camera_manager.h | 4 --\n> .../{ => internal}/event_dispatcher.h | 0\n> .../internal/event_dispatcher_poll.h | 2 +-\n> .../libcamera/{ => internal}/event_notifier.h | 0\n> include/libcamera/internal/meson.build | 3 +\n> include/libcamera/{ => internal}/timer.h | 0\n> include/libcamera/meson.build | 3 -\n> src/cam/capture.cpp | 9 +++\n> src/cam/capture.h | 1 +\n> src/cam/event_loop.cpp | 63 ++++++++++++++++---\n> src/cam/event_loop.h | 24 ++++---\n> src/cam/main.cpp | 16 ++---\n> src/cam/meson.build | 13 +++-\n> src/libcamera/camera_manager.cpp | 44 +------------\n> src/libcamera/device_enumerator_udev.cpp | 3 +-\n> src/libcamera/event_dispatcher.cpp | 2 +-\n> src/libcamera/event_dispatcher_poll.cpp | 5 +-\n> src/libcamera/event_notifier.cpp | 4 +-\n> src/libcamera/ipc_unixsocket.cpp | 3 +-\n> src/libcamera/pipeline/rkisp1/timeline.h | 3 +-\n> src/libcamera/process.cpp | 3 +-\n> .../proxy/worker/ipa_proxy_linux_worker.cpp | 2 +-\n> src/libcamera/thread.cpp | 3 +-\n> src/libcamera/timer.cpp | 4 +-\n> src/libcamera/v4l2_device.cpp | 3 +-\n> src/libcamera/v4l2_videodevice.cpp | 2 +-\n> test/camera/buffer_import.cpp | 8 +--\n> test/camera/capture.cpp | 9 ++-\n> test/event-dispatcher.cpp | 5 +-\n> test/event-thread.cpp | 5 +-\n> test/event.cpp | 7 +--\n> test/hotplug-cameras.cpp | 4 +-\n> test/ipa/ipa_interface_test.cpp | 6 +-\n> test/ipc/unixsocket.cpp | 5 +-\n> test/log/log_process.cpp | 4 +-\n> test/object-invoke.cpp | 2 +-\n> test/process/process_test.cpp | 5 +-\n> test/timer-thread.cpp | 5 +-\n> test/timer.cpp | 5 +-\n> test/v4l2_videodevice/buffer_sharing.cpp | 4 +-\n> test/v4l2_videodevice/capture_async.cpp | 4 +-\n> test/v4l2_videodevice/v4l2_m2mdevice.cpp | 4 +-\n> 42 files changed, 159 insertions(+), 142 deletions(-)\n> rename include/libcamera/{ => internal}/event_dispatcher.h (100%)\n> rename include/libcamera/{ => internal}/event_notifier.h (100%)\n> rename include/libcamera/{ => internal}/timer.h (100%)\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 366B4BE081\n\tfor ;\n\tMon, 16 Nov 2020 11:47:54 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AA35F632C3;\n\tMon, 16 Nov 2020 12:47:53 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 16D21631B5\n\tfor ;\n\tMon, 16 Nov 2020 12:47:53 +0100 (CET)","from [192.168.0.217]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8CB15A1B;\n\tMon, 16 Nov 2020 12:47:52 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"ecnRHHRL\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1605527272;\n\tbh=HkvukX4L48sd23NeE4+ZOXofJyrzJVLZpE3C34+pg6w=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=ecnRHHRL8nXjoJNsgmDiBuqKASvvtaDe4w9Mxzv8KvBNuzfbNS9a+SqFCb1e14sic\n\tXfalzkADimDu/valImwjZJQetGLbmC2ZJlcu2HyRRhAvgSqwezzdcihssM6/k5JCH/\n\tLwCwhYYP+GP2GuBCx6MTLjlFUyZEZhwup4QXtjic=","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20201113063815.10288-1-laurent.pinchart@ideasonboard.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<15de198b-1f9f-f6d0-d98d-c054978d2d29@ideasonboard.com>","Date":"Mon, 16 Nov 2020 11:47:49 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20201113063815.10288-1-laurent.pinchart@ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v1 0/5] cam: Move request processing\n\tto main thread","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":13724,"web_url":"https://patchwork.libcamera.org/comment/13724/","msgid":"<20201116123528.GI6540@pendragon.ideasonboard.com>","date":"2020-11-16T12:35:28","subject":"Re: [libcamera-devel] [PATCH v1 0/5] cam: Move request processing\n\tto main thread","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Mon, Nov 16, 2020 at 11:47:49AM +0000, Kieran Bingham wrote:\n> On 13/11/2020 06:38, Laurent Pinchart wrote:\n> > Hello,\n> > \n> > This patch series improves the cam application by moving request\n> > processing to the main thread to avoid blocking the camera manager\n> > thread (in which the request completion signal is emitted) for extended\n> > periods of time. This helps meeting the realtime constraints of pipeline\n> > handlers.\n> > \n> > Note that move requesting processing to a different thread isn't a\n> > magical solution, if the average processing time for a request is larger\n> > than the frame interval, a request queue underrun will still occur. It\n> > however helps handling random delays in request processing.\n> > \n> > Included in the series is a move to libevent to implement the cam event\n> > loop. The event loop is currently based on the libcamera event\n> > dispatcher for historical reasons. The event dispatcher API exposed by\n> > libcamera is incomplete, as thread support is internal to libcamera.\n> > Instead of reimplementing the missing parts in the cam event loop, or\n> > exposing thread support in the libcamera public API, moving to libevent\n> > allows making the event dispatcher implementation fully private (in\n> > patch 3/5). It isn't the job of libcamera to provide such generic APIs.\n> \n> This series breaks simple-cam.\n> \n> Will you plan to update that too?\n> If not I'll handle it, just let me know.\n\nIf you can handle it, that would be nice. Otherwise I'll do so. Sorry\nabout the breakage :-S\n\nRegarding the event loop implementation in simple-cam, should we just\ncopy the one from cam ?\n\n> > Laurent Pinchart (5):\n> > test: Get event dispatcher from current thread\n> > cam: Use libevent to implement event loop\n> > libcamera: Move EventDispatcher to internal API\n> > cam: event_loop: Add deferred calls support\n> > cam: Move request processing to main thread\n> > \n> > include/libcamera/camera_manager.h | 4 --\n> > .../{ => internal}/event_dispatcher.h | 0\n> > .../internal/event_dispatcher_poll.h | 2 +-\n> > .../libcamera/{ => internal}/event_notifier.h | 0\n> > include/libcamera/internal/meson.build | 3 +\n> > include/libcamera/{ => internal}/timer.h | 0\n> > include/libcamera/meson.build | 3 -\n> > src/cam/capture.cpp | 9 +++\n> > src/cam/capture.h | 1 +\n> > src/cam/event_loop.cpp | 63 ++++++++++++++++---\n> > src/cam/event_loop.h | 24 ++++---\n> > src/cam/main.cpp | 16 ++---\n> > src/cam/meson.build | 13 +++-\n> > src/libcamera/camera_manager.cpp | 44 +------------\n> > src/libcamera/device_enumerator_udev.cpp | 3 +-\n> > src/libcamera/event_dispatcher.cpp | 2 +-\n> > src/libcamera/event_dispatcher_poll.cpp | 5 +-\n> > src/libcamera/event_notifier.cpp | 4 +-\n> > src/libcamera/ipc_unixsocket.cpp | 3 +-\n> > src/libcamera/pipeline/rkisp1/timeline.h | 3 +-\n> > src/libcamera/process.cpp | 3 +-\n> > .../proxy/worker/ipa_proxy_linux_worker.cpp | 2 +-\n> > src/libcamera/thread.cpp | 3 +-\n> > src/libcamera/timer.cpp | 4 +-\n> > src/libcamera/v4l2_device.cpp | 3 +-\n> > src/libcamera/v4l2_videodevice.cpp | 2 +-\n> > test/camera/buffer_import.cpp | 8 +--\n> > test/camera/capture.cpp | 9 ++-\n> > test/event-dispatcher.cpp | 5 +-\n> > test/event-thread.cpp | 5 +-\n> > test/event.cpp | 7 +--\n> > test/hotplug-cameras.cpp | 4 +-\n> > test/ipa/ipa_interface_test.cpp | 6 +-\n> > test/ipc/unixsocket.cpp | 5 +-\n> > test/log/log_process.cpp | 4 +-\n> > test/object-invoke.cpp | 2 +-\n> > test/process/process_test.cpp | 5 +-\n> > test/timer-thread.cpp | 5 +-\n> > test/timer.cpp | 5 +-\n> > test/v4l2_videodevice/buffer_sharing.cpp | 4 +-\n> > test/v4l2_videodevice/capture_async.cpp | 4 +-\n> > test/v4l2_videodevice/v4l2_m2mdevice.cpp | 4 +-\n> > 42 files changed, 159 insertions(+), 142 deletions(-)\n> > rename include/libcamera/{ => internal}/event_dispatcher.h (100%)\n> > rename include/libcamera/{ => internal}/event_notifier.h (100%)\n> > rename include/libcamera/{ => internal}/timer.h (100%)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5DEA9BE082\n\tfor ;\n\tMon, 16 Nov 2020 12:35:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 24470632C3;\n\tMon, 16 Nov 2020 13:35:36 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1A6BA631B5\n\tfor ;\n\tMon, 16 Nov 2020 13:35:35 +0100 (CET)","from pendragon.ideasonboard.com (85-76-51-86-nat.elisa-mobile.fi\n\t[85.76.51.86])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 92F56A1B;\n\tMon, 16 Nov 2020 13:35:34 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"s1SefjxI\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1605530134;\n\tbh=B+KoAe21GdFSXgjUxGb+oMlHIX9mbRZRdpZULYQ0Z4Y=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=s1SefjxI8qhl64bk/LnHbQrrIPWMHx8BzGceqUsBMShyZLnZUEHjympMSXfC+mRoB\n\t/lb+K382I0wWvK8u1sNTfwm+CMJIVvlXmOdPWlXFiF2f/xZ2jQ03lwx3rUeIik6YKj\n\tuLX9bW26Odt1wmLkkbpQ4eMiNBYn8HGamzlTfMmY=","Date":"Mon, 16 Nov 2020 14:35:28 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"<20201116123528.GI6540@pendragon.ideasonboard.com>","References":"<20201113063815.10288-1-laurent.pinchart@ideasonboard.com>\n\t<15de198b-1f9f-f6d0-d98d-c054978d2d29@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<15de198b-1f9f-f6d0-d98d-c054978d2d29@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v1 0/5] cam: Move request processing\n\tto main thread","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]