From patchwork Thu Aug 5 22:24:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 13231 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0926AC3235 for ; Thu, 5 Aug 2021 22:24:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 15CBB6884F; Fri, 6 Aug 2021 00:24:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O9S80/Bg"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DFB006026E for ; Fri, 6 Aug 2021 00:24:55 +0200 (CEST) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 658404FB for ; Fri, 6 Aug 2021 00:24:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1628202295; bh=x9yo/YXSyXMQdKiu8uBqXT3ZoaUdQi4G3RbO452DHJU=; h=From:To:Subject:Date:From; b=O9S80/BglOS977tNStW7e16PFwbKj2IFJTFhJMqBpIKyRTbOWNBgy38TkHx1WeNd1 VR78d/NkHd4593WaTrdUHr8PwimsCm406Oci+3hJLN6gLerM+4hNwiTVdFPn1X1rlo Fl/SLay2kLU9Mf9O707zly5H4dBLnpY+1hXCRbnM= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Fri, 6 Aug 2021 01:24:26 +0300 Message-Id: <20210805222436.6263-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 00/10] Concurrent camera support in simple pipeline handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, Another day, another patch series that has been sitting in my tree and that I'd like to stop rebasing. The series reworks the simple pipeline handler to correctly support streaming from multiple cameras concurrently. The reason I haven't posted it yet is that the libcamera core gets doesn't support this feature yet. It's easy to work around by removing some checks for testing, but will take more time to fix correctly. The simple pipeline handler already supports registering multiple cameras, and relies on the libcamera core blocking concurrent access to different cameras backed by the same media device. It thus doesn't implement internally mutual exclusion between cameras that share hardware resources, but, based on the assumption that a single camera will be used at a time, implements camera operation around a global active camera pointer. Patch 01/10 is the only libcamera core patch. It extends the MediaEntity class to expose the type of userspace interface that the entity implements. Patches 02/10 to 06/10 then rework the way the pipeline handler stores data about pipelines and accesses video devices and subdevs. Patch 02/10 extends the SimpleCameraData entity representation to store more information about graph traversal, and patch 04/10 stores the entity corresponding to the video node with all other pipeline entities in SimpleCameraData. Patches 03/10, 05/10 and 06/10 generalize handling of video devices and subdev by opening them all together at match time and storing them in a single container in the SimplePipelineHandler class. With this done, path 07/10 adds an entity reservation mechanism in the SimplePipelineHandler class, to handle mutual exclusion when multiple cameras use the same entities. With this change, the simple pipeline handler handles concurrent access to cameras internally, without depending on mutual exclusion implemented by the libcamera core anymore. This doesn't result in any change of behaviour, but paves the way to concurrent usage of multiple cameras that don't share entities once the restriction in the libcamera core will be lifted. Patch 08/10 continues in that direction by moving the converter from the SimplePipelineHandler class to the SimpleCameraData class. This allows usage of the converter by multiple cameras concurrently. Patch 09/10 then moves the bufferReady handler to the SimpleCameraData class as well, to handle buffer completion in the correct context without relying on the global active camera pointer. Finally, patch 10/10 removes the active camera pointer as it's not used anymore. This series has only been partially tested, as the libcamera core doesn't correctly support concurrent streaming from cameras belonging to the same pipeline handler. The code was tested with a single camera and didn't introduce any regression, and was also tested on i.MX8MP with a few hacks in the libcamera core to lift the restrictions and a few out-of-tree patches to support the i.MX8MP (those are not ready for submission yet). I would thus understand a reluctance to get this merged already, even if I believe that the absence of regression, combined with the cleaner (in my opinion) implementation of the simple pipeline handler are good enough reasons by themselves to already merge this. Laurent Pinchart (10): libcamera: media_object: Expose entity type libcamera: pipeline: simple: Add sink and source pads to entity data libcamera: pipeline: simple: Delay opening of video device until init() libcamera: pipeline: simple: Store video node entity in camera data libcamera: pipeline: simple: Store all entity devices in common map libcamera: pipeline: simple: Open all video devices at match() time libcamera: pipeline: simple: Add pipeline pad reservation mechanism libcamera: pipeline: simple: Move converter to SimpleCameraData libcamera: pipeline: simple: Move bufferReady handler to SimpleCameraData libcamera: pipeline: simple: Remove SimplePipelineHandler::activeCamera_ include/libcamera/internal/media_object.h | 11 +- src/libcamera/media_device.cpp | 9 +- src/libcamera/media_object.cpp | 50 ++- src/libcamera/pipeline/simple/simple.cpp | 506 ++++++++++++++-------- 4 files changed, 377 insertions(+), 199 deletions(-) Tested-by: Martin Kepplinger