From patchwork Sun Jan 31 22:46:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 11091 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89CCBBD808 for ; Sun, 31 Jan 2021 22:47:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4F59968408; Sun, 31 Jan 2021 23:47:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mlhY4Srw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ACD35683E6 for ; Sun, 31 Jan 2021 23:47:32 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3DC108A0; Sun, 31 Jan 2021 23:47:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1612133252; bh=ZY2jarT0TsSLPyvDzZtj+FVF6KM2VIUroJRj8LLakB8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mlhY4SrwoUN5hvzEMQh4oPI6yuk1ADL1r2Rf6RIVwlLhNliiNL+jDPrSHlQt/kdqf SwTXn7vq6gxS/ZZMWeLG5IbAdqSmaRVSDwOXFgmlU/UR+jvo86Kh0wglrrSeX8pt1w t/rj5+eMY7a6hx5TGdHQbkGeiNIbR/Zj0UMZCfTc= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Mon, 1 Feb 2021 00:46:54 +0200 Message-Id: <20210131224702.8838-13-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20210131224702.8838-1-laurent.pinchart@ideasonboard.com> References: <20210131224702.8838-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 12/20] libcamera: pipeline: simple: Document the pipeline handler design X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Phi-Bang Nguyen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The simple pipeline handler has grown over time, and isn't that simple anymore that it can easily be understood by an unfamiliar reader. Document the design to explicitly state the expectations of the pipeline handler, and to explain how it operates. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- src/libcamera/pipeline/simple/simple.cpp | 83 ++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index f072e0f1fa81..7c5a56a2f395 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -38,6 +38,89 @@ namespace libcamera { LOG_DEFINE_CATEGORY(SimplePipeline) +/* ----------------------------------------------------------------------------- + * + * Overview + * -------- + * + * The SimplePipelineHandler relies on generic kernel APIs to control a camera + * device, without any device-specific code and with limited device-specific + * static data. + * + * To quality for support by the simple pipeline handler, a device shall + * + * - be supported by V4L2 drivers, exposing the Media Controller API, the V4L2 + * subdev APIs and the media bus format-based enumeration extension for the + * VIDIOC_ENUM_FMT ioctl ; + * - not expose any device-specific API from drivers to userspace ; + * - include one or more camera sensor media entities and one or more video + * capture devices ; + * - have a capture pipeline with linear paths from the camera sensors to the + * video capture devices ; and + * - have an optional memory-to-memory device to perform format conversion + * and/or scaling, exposed as a V4L2 M2M device. + * + * As devices that require a specific pipeline handler may still match the + * above characteristics, the simple pipeline handler doesn't attempt to + * automatically determine which devices it can support. It instead relies on + * an explicit list of supported devices, provided in the supportedDevices + * array. + * + * When matching a device, the pipeline handler enumerates all camera sensors + * and attempts, for each of them, to find a path to a video captude video node. + * It does so by traversing the media graph, following the first non permanently + * disabled downstream link. If such a path is found, the pipeline handler + * creates a corresponding SimpleCameraData instance, and stores the media graph + * path in its entities_ list. + * + * A more complex graph search algorithm could be implemented if a device that + * would otherwise be compatible with the pipeline handler isn't correctly + * handled by this heuristic. + * + * Once the camera data instances have been created, the match() function + * creates a V4L2Subdevice instance for each entity used by any of the cameras + * and stores the instances in SimplePipelineHandler::subdevs_, accessible by + * the SimpleCameraData class through the SimplePipelineHandler::subdev() + * function. This avoids duplication of subdev instances between different + * cameras when the same entity is used in multiple paths. A similar mechanism + * is used for V4L2VideoDevice instances, but instances are in this case created + * on demand when access through SimplePipelineHandler::video() instead of all + * in one go at initialization time. + * + * Finally, all camera data instances are initialized to gather information + * about the possible pipeline configurations for the corresponding camera. If + * valid pipeline configurations are found, a Camera is registered for the + * SimpleCameraData instance. + * + * Pipeline Configuration + * ---------------------- + * + * The simple pipeline handler configures the pipeline by propagating V4L2 + * subdev formats from the camera sensor to the video node. The format is first + * set on the camera sensor's output, using the native camera sensor + * resolution. Then, on every link in the pipeline, the format is retrieved on + * the link source and set unmodified on the link sink. + * + * When initializating the camera data, this above procedure is repeated for + * every media bus format supported by the camera sensor. Upon reaching the + * video node, the pixel formats compatible with the media bus format are + * enumerated. Each of those pixel formats correspond to one possible pipeline + * configuration, stored as an intsance of SimpleCameraData::Configuration in + * the SimpleCameraData::formats_ map. + * + * Format Conversion and Scaling + * ----------------------------- + * + * The capture pipeline isn't expected to include a scaler, and if a scaler is + * available, it is ignored when configuring the pipeline. However, the simple + * pipeline handler supports optional memory-to-memory converters to scale the + * image and convert it to a different pixel format. If such a converter is + * present, the pipeline handler enumerates, for each pipeline configuration, + * the pixel formats and sizes that the converter can produce for the output of + * the capture video node, and stores the information in the outputFormats and + * outputSizes of the SimpleCameraData::Configuration structure. + */ + class SimplePipelineHandler; struct SimplePipelineInfo {