{"id":13231,"url":"https://patchwork.libcamera.org/api/covers/13231/?format=json","web_url":"https://patchwork.libcamera.org/cover/13231/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210805222436.6263-1-laurent.pinchart@ideasonboard.com>","date":"2021-08-05T22:24:26","name":"[libcamera-devel,00/10] Concurrent camera support in simple pipeline handler","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/cover/13231/mbox/","series":[{"id":2314,"url":"https://patchwork.libcamera.org/api/series/2314/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2314","date":"2021-08-05T22:24:26","name":"Concurrent camera support in simple pipeline handler","version":1,"mbox":"https://patchwork.libcamera.org/series/2314/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/13231/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0926AC3235\n\tfor ;\n\tThu, 5 Aug 2021 22:24:59 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 15CBB6884F;\n\tFri, 6 Aug 2021 00:24:58 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DFB006026E\n\tfor ;\n\tFri, 6 Aug 2021 00:24:55 +0200 (CEST)","from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 658404FB\n\tfor ;\n\tFri, 6 Aug 2021 00:24:55 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"O9S80/Bg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1628202295;\n\tbh=x9yo/YXSyXMQdKiu8uBqXT3ZoaUdQi4G3RbO452DHJU=;\n\th=From:To:Subject:Date:From;\n\tb=O9S80/BglOS977tNStW7e16PFwbKj2IFJTFhJMqBpIKyRTbOWNBgy38TkHx1WeNd1\n\tVR78d/NkHd4593WaTrdUHr8PwimsCm406Oci+3hJLN6gLerM+4hNwiTVdFPn1X1rlo\n\tFl/SLay2kLU9Mf9O707zly5H4dBLnpY+1hXCRbnM=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 6 Aug 2021 01:24:26 +0300","Message-Id":"<20210805222436.6263-1-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 00/10] Concurrent camera support in simple\n\tpipeline handler","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Hello,\n\nAnother day, another patch series that has been sitting in my tree and\nthat I'd like to stop rebasing.\n\nThe series reworks the simple pipeline handler to correctly support\nstreaming from multiple cameras concurrently. The reason I haven't\nposted it yet is that the libcamera core gets doesn't support this\nfeature yet. It's easy to work around by removing some checks for\ntesting, but will take more time to fix correctly.\n\nThe simple pipeline handler already supports registering multiple\ncameras, and relies on the libcamera core blocking concurrent access to\ndifferent cameras backed by the same media device. It thus doesn't\nimplement internally mutual exclusion between cameras that share\nhardware resources, but, based on the assumption that a single camera\nwill be used at a time, implements camera operation around a global\nactive camera pointer.\n\nPatch 01/10 is the only libcamera core patch. It extends the MediaEntity\nclass to expose the type of userspace interface that the entity\nimplements. Patches 02/10 to 06/10 then rework the way the pipeline\nhandler stores data about pipelines and accesses video devices and\nsubdevs.\n\nPatch 02/10 extends the SimpleCameraData entity representation to store\nmore information about graph traversal, and patch 04/10 stores the\nentity corresponding to the video node with all other pipeline entities\nin SimpleCameraData. Patches 03/10, 05/10 and 06/10 generalize handling\nof video devices and subdev by opening them all together at match time\nand storing them in a single container in the SimplePipelineHandler\nclass.\n\nWith this done, path 07/10 adds an entity reservation mechanism in the\nSimplePipelineHandler class, to handle mutual exclusion when multiple\ncameras use the same entities. With this change, the simple pipeline\nhandler handles concurrent access to cameras internally, without\ndepending on mutual exclusion implemented by the libcamera core anymore.\nThis doesn't result in any change of behaviour, but paves the way to\nconcurrent usage of multiple cameras that don't share entities once the\nrestriction in the libcamera core will be lifted.\n\nPatch 08/10 continues in that direction by moving the converter from the\nSimplePipelineHandler class to the SimpleCameraData class. This allows\nusage of the converter by multiple cameras concurrently. Patch 09/10\nthen moves the bufferReady handler to the SimpleCameraData class as\nwell, to handle buffer completion in the correct context without relying\non the global active camera pointer.\n\nFinally, patch 10/10 removes the active camera pointer as it's not used\nanymore.\n\nThis series has only been partially tested, as the libcamera core\ndoesn't correctly support concurrent streaming from cameras belonging to\nthe same pipeline handler. The code was tested with a single camera and\ndidn't introduce any regression, and was also tested on i.MX8MP with a\nfew hacks in the libcamera core to lift the restrictions and a few\nout-of-tree patches to support the i.MX8MP (those are not ready for\nsubmission yet). I would thus understand a reluctance to get this merged\nalready, even if I believe that the absence of regression, combined with\nthe cleaner (in my opinion) implementation of the simple pipeline\nhandler are good enough reasons by themselves to already merge this.\n\nLaurent Pinchart (10):\n libcamera: media_object: Expose entity type\n libcamera: pipeline: simple: Add sink and source pads to entity data\n libcamera: pipeline: simple: Delay opening of video device until\n init()\n libcamera: pipeline: simple: Store video node entity in camera data\n libcamera: pipeline: simple: Store all entity devices in common map\n libcamera: pipeline: simple: Open all video devices at match() time\n libcamera: pipeline: simple: Add pipeline pad reservation mechanism\n libcamera: pipeline: simple: Move converter to SimpleCameraData\n libcamera: pipeline: simple: Move bufferReady handler to\n SimpleCameraData\n libcamera: pipeline: simple: Remove\n SimplePipelineHandler::activeCamera_\n\n include/libcamera/internal/media_object.h | 11 +-\n src/libcamera/media_device.cpp | 9 +-\n src/libcamera/media_object.cpp | 50 ++-\n src/libcamera/pipeline/simple/simple.cpp | 506 ++++++++++++++--------\n 4 files changed, 377 insertions(+), 199 deletions(-)"}