From patchwork Wed Jan 22 20:57:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 2716 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 002E260804 for ; Wed, 22 Jan 2020 21:57:42 +0100 (CET) Received: from pendragon.bb.dnainternet.fi (81-175-216-236.bb.dnainternet.fi [81.175.216.236]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7E5772E5 for ; Wed, 22 Jan 2020 21:57:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1579726662; bh=cxE+OaB6j4J4mZ5CaB1VwO1VLpUGTzZOTm8zevCsbCg=; h=From:To:Subject:Date:From; b=uMt3gFjjKIUQ8/OX4VahNnoxDqxBV+CtRZwn4uoyOZtbZ3jmbdl5hB3luRscw4SLg 2Dx0eLNlhpCKw+YGWki+ALQvD06rAhVMxttcU7zs65bIHmWM+ZCUURGdFTVwAWr0j6 YMKCl9KzCBjIcM277hcvxKseX9fsSSeAFlI+OPXk= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Jan 2020 22:57:10 +0200 Message-Id: <20200122205723.8865-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 00/13] Initial libcamera threading model X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2020 20:57:43 -0000 Hello, This series defines and implements an initial version of the threading model. libcamera currently relies on the application providing an event loop. This cumbersome when the direct user of libcamera is not an application but a framework, as no event loop is available in that case. The Android camera HAL and the V4L2 adaptation layers had to create an internal thread to make an event loop available to libcamera, and handle synchronization between calls from the upper framework and the internal thread. A GStreamer element would require a similar implementation, resulting in lots of code duplication. Furthermore, relying on an application event loop may block libcamera from processing events if the application performs long blocking operations. This is not compatible with the real-time constraints of pipeline handlers and IPAs. For this reason, creating and managing threads internally to libcamera is a better option. This requires defining a threading model, as all of a sudden multiple contexts of execution will be available when only one context existed before. Some of the preparatory work from v1 has been merged already, but the series still starts with 5 preparatory patches (some of them being split from the larger "libcamera: camera_manager: Run the camera manager in a thread" patch). The most notable change there is the move of private data members of the CameraManager and Camera classes to private classes, helping with ABI breakage avoidance in future work. The next two patches consist solely of documentation. Patch 06/13 defines the threading model and patch 07/13 documents it in core classes. Patch 08/13 then makes signal connection and disconnection thread-safe to support multi-threaded contexts. Patch 09/13 is the most disruptive change, as it creates an thread for the CameraManager. At the moment this is the only thread created by libcamera, and all pipeline handlers and camera are bound to it. This model is likely too simple and will need to be extended with more threads in the future to avoid low-priority blocking operations interfering with real-time camera constraints. The change is however a step in the right direction, as it brings consideration of multiple threads to all of libcamera. Patches 10/13 to 11/13 then document and implement the threading model in the Camera and PipelineHandler classes. Note that patch 09/13 breaks operation of libcamera, and patch 10/13 fixes that. This is caused by the threading model implementation being split in two patches. I have decided to organize the changes this way to keep changes reviewable, but I'm open to squashing the two patches together, especially now that 09/13 has become smaller. Patches 12/13 and 13/13 finally simplify the V4L2 compatibility and Android camera HAL layers by removing the internal threads, as libcamera doesn't require an externally-provided event loop anymore. I have tested 12/13 with yavta, but 13/13 is currently only compile-tested. Laurent Pinchart (13): libcamera: Fix documentation of buffer allocation/export functions libcamera: camera_manager: Move private data members to private implementation libcamera: camera_manager: Return a copy of the vector from cameras() libcamera: camera: Move private data members to private implementation libcamera: camera: Centralize state checks in Private class libcamera: Define the threading model libcamera: Document thread-safety attributes of core classes libcamera: signal: Make connection and disconnection thread-safe libcamera: camera_manager: Run the camera manager in a thread libcamera: camera: Implement the threading model libcamera: pipeline_handler: Document the threading model v4l2: Remove internal thread android: Remove internal thread Documentation/Doxyfile.in | 5 +- include/libcamera/camera.h | 28 +- include/libcamera/camera_manager.h | 13 +- src/android/camera3_hal.cpp | 8 +- src/android/camera_device.cpp | 48 +-- src/android/camera_device.h | 16 +- src/android/camera_hal_manager.cpp | 78 ++--- src/android/camera_hal_manager.h | 17 +- src/android/camera_ops.cpp | 96 ++++++ src/android/camera_ops.h | 15 + src/android/camera_proxy.cpp | 180 ----------- src/android/camera_proxy.h | 42 --- src/android/meson.build | 2 +- src/libcamera/camera.cpp | 362 ++++++++++++++++------- src/libcamera/camera_manager.cpp | 298 ++++++++++++++----- src/libcamera/device_enumerator.cpp | 2 + src/libcamera/event_notifier.cpp | 2 + src/libcamera/framebuffer_allocator.cpp | 50 +--- src/libcamera/include/pipeline_handler.h | 4 +- src/libcamera/ipc_unixsocket.cpp | 2 + src/libcamera/object.cpp | 10 +- src/libcamera/pipeline_handler.cpp | 44 ++- src/libcamera/signal.cpp | 33 +++ src/libcamera/thread.cpp | 86 ++++++ src/libcamera/timer.cpp | 20 +- src/libcamera/v4l2_videodevice.cpp | 5 +- src/v4l2/v4l2_camera.h | 2 +- src/v4l2/v4l2_camera_proxy.cpp | 43 +-- src/v4l2/v4l2_compat_manager.cpp | 48 +-- src/v4l2/v4l2_compat_manager.h | 13 +- 30 files changed, 906 insertions(+), 666 deletions(-) create mode 100644 src/android/camera_ops.cpp create mode 100644 src/android/camera_ops.h delete mode 100644 src/android/camera_proxy.cpp delete mode 100644 src/android/camera_proxy.h