From patchwork Mon Aug 19 16:09:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20950 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 660A9BDB13 for ; Mon, 19 Aug 2024 16:09:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CBC6A633C9; Mon, 19 Aug 2024 18:09:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Y9ejlUxN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 243F6633B4 for ; Mon, 19 Aug 2024 18:09:36 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 868055B3; Mon, 19 Aug 2024 18:08:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083714; bh=opq5Le7Jj8z9OeTfUCDjTM+Cc6JGBcb+aOIEq4GmVyQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y9ejlUxN4eBasvzgIgJOzNXlN+3Y7SwpV6CNyeyuvWyjlh+91ndxFquJDn0baZs+i D4urhQUBBUcS0SItiRMkdPI5YyCnAwiX9cxKup8zzFOIkL8BuX6s0+CSxM5VwbIqbH UUKgcHRC4RFXaBcZJ1gxyQE0yMhbgVVn1PJDMsi0= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Laurent Pinchart Subject: [PATCH v3 1/7] Documentation: Add documentation-contents.rst Date: Mon, 19 Aug 2024 17:09:15 +0100 Message-Id: <20240819160921.468981-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new .rst file referencing the documentation contents. This file is then included in each documentation page so that we can enhance the Documentation pages on the libcamera website using it. As we do not want the appearance of the libcamera in-tree Documentation to change just yet, disable the new class using the sphinx theme's CSS. To facilitate easier distinguishing between "normal" and documentation pages on the website we want to add a "documentation" class to the content of all such pages. Since this new file will be included on each documentation page it is convenient to add the new directive here - do so. As the website uses different CSS to libcamera, move the contents on docs.rst a little so that the directive at the end of the contents block applies correctly. Reviewed-by: Laurent Pinchart Signed-off-by: Daniel Scally Reviewed-by: Stefan Klug --- Changes since v2: - Spaces instead of tabs in meson.build changes since v1: - Formatting - Properly added the new file to meson's list of docs_sources - Commented the inclusion of the rst-class directive at the end of the new file Documentation/camera-sensor-model.rst | 2 ++ Documentation/code-of-conduct.rst | 2 ++ Documentation/coding-style.rst | 2 ++ Documentation/conf.py | 7 ++++- Documentation/docs.rst | 2 ++ Documentation/documentation-contents.rst | 27 +++++++++++++++++++ Documentation/environment_variables.rst | 2 ++ Documentation/getting-started.rst | 1 + .../guides/application-developer.rst | 2 ++ Documentation/guides/introduction.rst | 2 ++ Documentation/guides/ipa.rst | 2 ++ Documentation/guides/pipeline-handler.rst | 2 ++ Documentation/guides/tracing.rst | 2 ++ Documentation/lens_driver_requirements.rst | 2 ++ Documentation/meson.build | 1 + Documentation/python-bindings.rst | 2 ++ Documentation/sensor_driver_requirements.rst | 2 ++ Documentation/software-isp-benchmarking.rst | 2 ++ Documentation/theme/static/css/theme.css | 4 +++ 19 files changed, 67 insertions(+), 1 deletion(-) create mode 100644 Documentation/documentation-contents.rst diff --git a/Documentation/camera-sensor-model.rst b/Documentation/camera-sensor-model.rst index b66c880a..87a25bf4 100644 --- a/Documentation/camera-sensor-model.rst +++ b/Documentation/camera-sensor-model.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + .. _camera-sensor-model: .. todo: Move to Doxygen-generated documentation diff --git a/Documentation/code-of-conduct.rst b/Documentation/code-of-conduct.rst index 38b7d7ad..0edd1e99 100644 --- a/Documentation/code-of-conduct.rst +++ b/Documentation/code-of-conduct.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-4.0 +.. include:: documentation-contents.rst + .. _code-of-conduct: Contributor Covenant Code of Conduct diff --git a/Documentation/coding-style.rst b/Documentation/coding-style.rst index 3352b75c..6ac3a4a0 100644 --- a/Documentation/coding-style.rst +++ b/Documentation/coding-style.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + .. _coding-style-guidelines: Coding Style Guidelines diff --git a/Documentation/conf.py b/Documentation/conf.py index 7eeea7f3..325f2759 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -61,7 +61,12 @@ language = 'en' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] +exclude_patterns = [ + '_build', + 'Thumbs.db', + '.DS_Store', + 'documentation-contents.rst', +] # The name of the Pygments (syntax highlighting) style to use. pygments_style = None diff --git a/Documentation/docs.rst b/Documentation/docs.rst index a6e8a59a..5871961c 100644 --- a/Documentation/docs.rst +++ b/Documentation/docs.rst @@ -3,6 +3,8 @@ .. contents:: :local: +.. include:: documentation-contents.rst + ************* Documentation ************* diff --git a/Documentation/documentation-contents.rst b/Documentation/documentation-contents.rst new file mode 100644 index 00000000..a6915e05 --- /dev/null +++ b/Documentation/documentation-contents.rst @@ -0,0 +1,27 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. container:: documentation-nav + + * :doc:`/api-html/index` + * :doc:`/camera-sensor-model` + * :doc:`/code-of-conduct` + * :doc:`/coding-style` + * :doc:`/environment_variables` + * :doc:`/guides/application-developer` + * :doc:`/guides/introduction` + * :doc:`/guides/ipa` + * :doc:`/guides/pipeline-handler` + * :doc:`/guides/tracing` + * :doc:`/lens_driver_requirements` + * :doc:`/python-bindings` + * :doc:`/sensor_driver_requirements` + * :doc:`/software-isp-benchmarking` + +.. + The following directive adds the "documentation" class to all of the pages + generated by sphinx. This is not relevant in libcamera nor addressed in the + theme's CSS, since all of the pages here are documentation. It **is** used + to properly format the documentation pages on libcamera.org and so should not + be removed. + +.. rst-class:: documentation diff --git a/Documentation/environment_variables.rst b/Documentation/environment_variables.rst index de434c38..7da9883a 100644 --- a/Documentation/environment_variables.rst +++ b/Documentation/environment_variables.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + Environment variables ===================== diff --git a/Documentation/getting-started.rst b/Documentation/getting-started.rst index 987f43f7..63b050eb 100644 --- a/Documentation/getting-started.rst +++ b/Documentation/getting-started.rst @@ -1,4 +1,5 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 + .. Getting started information is defined in the project README file. .. include:: ../README.rst :start-after: .. section-begin-getting-started diff --git a/Documentation/guides/application-developer.rst b/Documentation/guides/application-developer.rst index 92e2a373..1ea8c40f 100644 --- a/Documentation/guides/application-developer.rst +++ b/Documentation/guides/application-developer.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: ../documentation-contents.rst + Using libcamera in a C++ application ==================================== diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst index 700ec2d3..8368bd4a 100644 --- a/Documentation/guides/introduction.rst +++ b/Documentation/guides/introduction.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: ../documentation-contents.rst + Developers guide to libcamera ============================= diff --git a/Documentation/guides/ipa.rst b/Documentation/guides/ipa.rst index 25deadef..cd640563 100644 --- a/Documentation/guides/ipa.rst +++ b/Documentation/guides/ipa.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: ../documentation-contents.rst + IPA Writer's Guide ================== diff --git a/Documentation/guides/pipeline-handler.rst b/Documentation/guides/pipeline-handler.rst index 5aa09e90..26aea433 100644 --- a/Documentation/guides/pipeline-handler.rst +++ b/Documentation/guides/pipeline-handler.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: ../documentation-contents.rst + Pipeline Handler Writers Guide ============================== diff --git a/Documentation/guides/tracing.rst b/Documentation/guides/tracing.rst index ae960d85..537dce50 100644 --- a/Documentation/guides/tracing.rst +++ b/Documentation/guides/tracing.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: ../documentation-contents.rst + Tracing Guide ============= diff --git a/Documentation/lens_driver_requirements.rst b/Documentation/lens_driver_requirements.rst index b96e502d..85fef76f 100644 --- a/Documentation/lens_driver_requirements.rst +++ b/Documentation/lens_driver_requirements.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + .. _lens-driver-requirements: Lens Driver Requirements diff --git a/Documentation/meson.build b/Documentation/meson.build index 1ba40fdf..79135b6f 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -129,6 +129,7 @@ if sphinx.found() 'conf.py', 'contributing.rst', 'docs.rst', + 'documentation-contents.rst', 'environment_variables.rst', 'guides/application-developer.rst', 'guides/introduction.rst', diff --git a/Documentation/python-bindings.rst b/Documentation/python-bindings.rst index ed9f686b..94712238 100644 --- a/Documentation/python-bindings.rst +++ b/Documentation/python-bindings.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + .. _python-bindings: Python Bindings for libcamera diff --git a/Documentation/sensor_driver_requirements.rst b/Documentation/sensor_driver_requirements.rst index 0e516b34..fb4269d0 100644 --- a/Documentation/sensor_driver_requirements.rst +++ b/Documentation/sensor_driver_requirements.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + .. _sensor-driver-requirements: Sensor Driver Requirements diff --git a/Documentation/software-isp-benchmarking.rst b/Documentation/software-isp-benchmarking.rst index b3033132..9c2a409b 100644 --- a/Documentation/software-isp-benchmarking.rst +++ b/Documentation/software-isp-benchmarking.rst @@ -1,5 +1,7 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 +.. include:: documentation-contents.rst + .. _software-isp-benchmarking: Software ISP benchmarking diff --git a/Documentation/theme/static/css/theme.css b/Documentation/theme/static/css/theme.css index d4274ea6..2b1ed095 100644 --- a/Documentation/theme/static/css/theme.css +++ b/Documentation/theme/static/css/theme.css @@ -289,3 +289,7 @@ div#signature { padding: 0px; visibility: hidden; } + +.documentation-nav { + display: none; +} From patchwork Mon Aug 19 16:09:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20951 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56781BDB13 for ; Mon, 19 Aug 2024 16:09:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8AB3E633CC; Mon, 19 Aug 2024 18:09:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dPCgIBOi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5533F633BA for ; Mon, 19 Aug 2024 18:09:36 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D0AAD12F1; Mon, 19 Aug 2024 18:08:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083715; bh=4moLSDWkfLSLksz+cJohxsB1HEtcOKyWIwRb7gi8Viw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dPCgIBOij/tq5x8AGtW7fuD387I8xwiVIE7ElvIXUe/kj8rnvHez9UAunfLauPAZR +fct3RL1y7Byz0N5xyZhP7+7VqIiSYI/FzmJRQRNnvpJx1v/L49/42CXljv8LFNlM3 wK9u5jcMDOkw9pYuhWd2s8chYwUdB8OE2gZrHUAI= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Laurent Pinchart Subject: [PATCH v3 2/7] Documentation: Alphabetise the Documentation toctree Date: Mon, 19 Aug 2024 17:09:16 +0100 Message-Id: <20240819160921.468981-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the exception of the initial group of four links, alphabetise the pages in the Documentation toctree so adding new ones can be done in a defined order. Reviewed-by: Laurent Pinchart Signed-off-by: Daniel Scally Reviewed-by: Stefan Klug --- Changes since v2: - None Changes since v1: - None Documentation/index.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Documentation/index.rst b/Documentation/index.rst index 5442ae75..52ddc494 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -14,14 +14,14 @@ Contribute Getting Started - Developer Guide Application Writer's Guide - Pipeline Handler Writer's Guide - IPA Writer's guide - Tracing guide + Camera Sensor Model + Developer Guide Environment variables - Sensor driver requirements + IPA Writer's guide Lens driver requirements + Pipeline Handler Writer's Guide Python Bindings - Camera Sensor Model + Sensor driver requirements SoftwareISP Benchmarking + Tracing guide From patchwork Mon Aug 19 16:09:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20952 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DB26BDB13 for ; Mon, 19 Aug 2024 16:09:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6F4B8633C2; Mon, 19 Aug 2024 18:09:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DG5/+msN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A69F633BE for ; Mon, 19 Aug 2024 18:09:36 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 244575A5; Mon, 19 Aug 2024 18:08:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083715; bh=h64l9hdWuk8Tayp5roc+7xsS0eX9WfLS/HjqHyByBuw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DG5/+msN5F+1T5v0SgsIg2zU4LjnMDq2Np7FC8KGuslm+k5+i/kUsCYtYLtABwdRi jHMJ9c+s+hgWmC8JMJj8wciZ2qK2UZc8/GWJ3AYj7jhdUe4L9Ee64HAglMViOCdGTU jpnTk35gQuXx0yPBFiIil1SXe/ovVnH3cmDk7Kb0= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally Subject: [PATCH v3 3/7] Documentation: Synchronise camera stack details Date: Mon, 19 Aug 2024 17:09:17 +0100 Message-Id: <20240819160921.468981-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are two near-duplicate instances of the camera stack details in the Documentation, in docs.rst and guides/introduction.rst. Remove them from guides/introduction.rst, with the exception of the explanations of the V4L2 Compatibility Layer and the Android HAL which are moved to the Camera Stack section in docs.rst. The Docs page already had its own separate version of those details but they are distinct and seemingly out of date - remove them. Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes since v2: - Refocused this patch to focus on camera stack section instead of the libcamera architecture section. Dropped the R-b. Changes since v1: - None Documentation/docs.rst | 56 +++++++------ Documentation/guides/introduction.rst | 109 +------------------------- 2 files changed, 31 insertions(+), 134 deletions(-) diff --git a/Documentation/docs.rst b/Documentation/docs.rst index 5871961c..0eacc924 100644 --- a/Documentation/docs.rst +++ b/Documentation/docs.rst @@ -239,6 +239,36 @@ The camera stack comprises four software layers. From bottom to top: libcamera framework or libcamera adaptation, and are outside of the scope of the libcamera project. +V4L2 Compatibility Layer + V4L2 compatibility is achieved through a shared library that traps all + accesses to camera devices and routes them to libcamera to emulate high-level + V4L2 camera devices. It is injected in a process address space through + ``LD_PRELOAD`` and is completely transparent for applications. + + The compatibility layer exposes camera device features on a best-effort basis, + and aims for the level of features traditionally available from a UVC camera + designed for video conferencing. + +Android Camera HAL + Camera support for Android is achieved through a generic Android camera HAL + implementation on top of libcamera. The HAL implements features required by + Android and out of scope from libcamera, such as JPEG encoding support. + + This component is used to provide support for ChromeOS platforms + +GStreamer element (gstlibcamerasrc) + A `GStreamer element`_ is provided to allow capture from libcamera supported + devices through GStreamer pipelines, and connect to other elements for further + processing. + + Development of this element is ongoing and is limited to a single stream. + +Native libcamera API + Applications can make use of the libcamera API directly using the C++ + API. An example application and walkthrough using the libcamera API can be + followed in the :doc:`Application writer's guide ` + +.. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html libcamera Architecture ====================== @@ -374,29 +404,3 @@ Helpers and Support Classes self-contained support classes, even if such code is present only once in the code base, in order to keep the source code clean and easy to read. This should be the case for instance for plugin management. - - -V4L2 Compatibility Layer ------------------------- - -V4L2 compatibility is achieved through a shared library that traps all -accesses to camera devices and routes them to libcamera to emulate high-level -V4L2 camera devices. It is injected in a process address space through -`LD_PRELOAD` and is completely transparent for applications. - -The compatibility layer exposes camera device features on a best-effort basis, -and aims for the level of features traditionally available from a UVC camera -designed for video conferencing. - - -Android Camera HAL ------------------- - -Camera support for Android is achieved through a generic Android -camera HAL implementation on top of libcamera. The HAL will implement internally -features required by Android and missing from libcamera, such as JPEG encoding -support. - -The Android camera HAL implementation will initially target the -LIMITED hardware level, with support for the FULL level then being gradually -implemented. diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst index 8368bd4a..d1e67654 100644 --- a/Documentation/guides/introduction.rst +++ b/Documentation/guides/introduction.rst @@ -26,8 +26,7 @@ desirable results from the camera. .. _Media Controller: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/mediactl/media-controller.html -In this developers guide, we will explore the `Camera Stack`_ and how it is -can be visualised at a high level, and explore the internal `Architecture`_ of +In this developers guide, we will explore the internal `Architecture`_ of the libcamera library with its components. The current `Platform Support`_ is detailed, as well as an overview of the `Licensing`_ requirements of the project. @@ -42,112 +41,6 @@ provides a tutorial of the key APIs exposed by libcamera. .. TODO: Correctly link to the other articles of the guide -Camera Stack ------------- - -The libcamera library is implemented in userspace, and makes use of underlying -kernel drivers that directly interact with hardware. - -Applications can make use of libcamera through the native `libcamera API`_'s or -through an adaptation layer integrating libcamera into a larger framework. - -.. _libcamera API: https://www.libcamera.org/api-html/index.html - -:: - - Application Layer - / +--------------+ +--------------+ +--------------+ +--------------+ - | | Native | | Framework | | Native | | Android | - | | V4L2 | | Application | | libcamera | | Camera | - | | Application | | (gstreamer) | | Application | | Framework | - \ +--------------+ +--------------+ +--------------+ +--------------+ - - ^ ^ ^ ^ - | | | | - | | | | - v v | v - Adaptation Layer | - / +--------------+ +--------------+ | +--------------+ - | | V4L2 | | gstreamer | | | Android | - | | Compatibility| | element | | | Camera | - | | (preload) | |(libcamerasrc)| | | HAL | - \ +--------------+ +--------------+ | +--------------+ - | - ^ ^ | ^ - | | | | - | | | | - v v v v - libcamera Framework - / +--------------------------------------------------------------------+ - | | | - | | libcamera | - | | | - \ +--------------------------------------------------------------------+ - - ^ ^ ^ - Userspace | | | - --------------------- | ---------------- | ---------------- | --------------- - Kernel | | | - v v v - - +-----------+ +-----------+ +-----------+ - | Media | <--> | Video | <--> | V4L2 | - | Device | | Device | | Subdev | - +-----------+ +-----------+ +-----------+ - -The camera stack comprises of four software layers. From bottom to top: - -* The kernel drivers control the camera hardware and expose a low-level - interface to userspace through the Linux kernel V4L2 family of APIs - (Media Controller API, V4L2 Video Device API and V4L2 Subdev API). - -* The libcamera framework is the core part of the stack. It handles all control - of the camera devices in its core component, libcamera, and exposes a native - C++ API to upper layers. - -* The libcamera adaptation layer is an umbrella term designating the components - that interface to libcamera in other frameworks. Notable examples are the V4L2 - compatibility layer, the gstreamer libcamera element, and the Android camera - HAL implementation based on libcamera which are provided as a part of the - libcamera project. - -* The applications and upper level frameworks are based on the libcamera - framework or libcamera adaptation, and are outside of the scope of the - libcamera project, however example native applications (cam, qcam) are - provided for testing. - - -V4L2 Compatibility Layer - V4L2 compatibility is achieved through a shared library that traps all - accesses to camera devices and routes them to libcamera to emulate high-level - V4L2 camera devices. It is injected in a process address space through - ``LD_PRELOAD`` and is completely transparent for applications. - - The compatibility layer exposes camera device features on a best-effort basis, - and aims for the level of features traditionally available from a UVC camera - designed for video conferencing. - -Android Camera HAL - Camera support for Android is achieved through a generic Android camera HAL - implementation on top of libcamera. The HAL implements features required by - Android and out of scope from libcamera, such as JPEG encoding support. - - This component is used to provide support for ChromeOS platforms - -GStreamer element (gstlibcamerasrc) - A `GStreamer element`_ is provided to allow capture from libcamera supported - devices through GStreamer pipelines, and connect to other elements for further - processing. - - Development of this element is ongoing and is limited to a single stream. - -Native libcamera API - Applications can make use of the libcamera API directly using the C++ - API. An example application and walkthrough using the libcamera API can be - followed in the `Application Writers Guide`_ - -.. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html - Architecture ------------ From patchwork Mon Aug 19 16:09:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20953 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D68DBDB13 for ; Mon, 19 Aug 2024 16:09:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3D52E633C8; Mon, 19 Aug 2024 18:09:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ohmp7FFm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC734633BF for ; Mon, 19 Aug 2024 18:09:36 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 65EE65B3; Mon, 19 Aug 2024 18:08:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083715; bh=b7aIidYg+zOEgx1ksOM9YLi8WdUuZEJMPdC9v8tVQok=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ohmp7FFmh2vnN1uGpnG5gsCjIW5TL4gASsvGsKvNj+izkobM1D1/dDQ0HDEbHWHgt 5sbzJAVNFXQvPDKWwA7gNqXrzB8tx/bY7e94ByDSWwxUrHwwqDZPbSNeUtYZgOmtPd MUScifmgeK892DoJ2p8pmom37YfL1pSgcsEuBQEY= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally Subject: [PATCH v3 4/7] Documentation: Breakout docs.rst Date: Mon, 19 Aug 2024 17:09:18 +0100 Message-Id: <20240819160921.468981-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation for including more of the Documentation for libcamera on the website, break out the libcamera Architecture and Feature Requirements sections of docs.rst file into separate files for each section. Add all of the new files to documentation-contents.rst so they're included on the website too. Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes since v2: - Reworked to breakout libcamera architecture instead of camera stack section. Dropped the R-b. Changes since v1: - Included the new files in meson's docs_sources array Documentation/docs.rst | 278 ----------------------- Documentation/documentation-contents.rst | 2 + Documentation/feature_requirements.rst | 145 ++++++++++++ Documentation/index.rst | 2 + Documentation/libcamera_architecture.rst | 138 +++++++++++ Documentation/meson.build | 2 + 6 files changed, 289 insertions(+), 278 deletions(-) create mode 100644 Documentation/feature_requirements.rst create mode 100644 Documentation/libcamera_architecture.rst diff --git a/Documentation/docs.rst b/Documentation/docs.rst index 0eacc924..1f0d12c0 100644 --- a/Documentation/docs.rst +++ b/Documentation/docs.rst @@ -21,149 +21,6 @@ The libcamera API is extensively documented using Doxygen. The :ref:`API nightly build ` contains the most up-to-date API documentation, built from the latest master branch. -Feature Requirements -==================== - -Device enumeration ------------------- - -The library shall support enumerating all camera devices available in the -system, including both fixed cameras and hotpluggable cameras. It shall -support cameras plugged and unplugged after the initialization of the -library, and shall offer a mechanism to notify applications of camera plug -and unplug. - -The following types of cameras shall be supported: - -* Internal cameras designed for point-and-shoot still image and video - capture usage, either controlled directly by the CPU, or exposed through - an internal USB bus as a UVC device. - -* External UVC cameras designed for video conferencing usage. - -Other types of camera, including analog cameras, depth cameras, thermal -cameras, external digital picture or movie cameras, are out of scope for -this project. - -A hardware device that includes independent camera sensors, such as front -and back sensors in a phone, shall be considered as multiple camera devices -for the purpose of this library. - -Independent Camera Devices --------------------------- - -When multiple cameras are present in the system and are able to operate -independently from each other, the library shall expose them as multiple -camera devices and support parallel operation without any additional usage -restriction apart from the limitations inherent to the hardware (such as -memory bandwidth, CPU usage or number of CSI-2 receivers for instance). - -Independent processes shall be able to use independent cameras devices -without interfering with each other. A single camera device shall be -usable by a single process at a time. - -Multiple streams support ------------------------- - -The library shall support multiple video streams running in parallel -for each camera device, within the limits imposed by the system. - -Per frame controls ------------------- - -The library shall support controlling capture parameters for each stream -on a per-frame basis, on a best effort basis based on the capabilities of the -hardware and underlying software stack (including kernel drivers and -firmware). It shall apply capture parameters to the frame they target, and -report the value of the parameters that have effectively been used for each -captured frame. - -When a camera device supports multiple streams, the library shall allow both -control of each stream independently, and control of multiple streams -together. Streams that are controlled together shall be synchronized. No -synchronization is required for streams controlled independently. - -Capability Enumeration ----------------------- - -The library shall expose capabilities of each camera device in a way that -allows applications to discover those capabilities dynamically. Applications -shall be allowed to cache capabilities for as long as they are using the -library. If capabilities can change at runtime, the library shall offer a -mechanism to notify applications of such changes. Applications shall not -cache capabilities in long term storage between runs. - -Capabilities shall be discovered dynamically at runtime from the device when -possible, and may come, in part or in full, from platform configuration -data. - -Device Profiles ---------------- - -The library may define different camera device profiles, each with a minimum -set of required capabilities. Applications may use those profiles to quickly -determine the level of features exposed by a device without parsing the full -list of capabilities. Camera devices may implement additional capabilities -on top of the minimum required set for the profile they expose. - -3A and Image Enhancement Algorithms ------------------------------------ - -The camera devices shall implement auto exposure, auto gain and auto white -balance. Camera devices that include a focus lens shall implement auto -focus. Additional image enhancement algorithms, such as noise reduction or -video stabilization, may be implemented. - -All algorithms may be implemented in hardware or firmware outside of the -library, or in software in the library. They shall all be controllable by -applications. - -The library shall be architectured to isolate the 3A and image enhancement -algorithms in a component with a documented API, respectively called the 3A -component and the 3A API. The 3A API shall be stable, and shall allow both -open-source and closed-source implementations of the 3A component. - -The library may include statically-linked open-source 3A components, and -shall support dynamically-linked open-source and closed-source 3A -components. - -Closed-source 3A Component Sandboxing -------------------------------------- - -For security purposes, it may be desired to run closed-source 3A components -in a separate process. The 3A API would in such a case be transported over -IPC. The 3A API shall make it possible to use any IPC mechanism that -supports passing file descriptors. - -The library may implement an IPC mechanism, and shall support third-party -platform-specific IPC mechanisms through the implementation of a -platform-specific 3A API wrapper. No modification to the library shall be -needed to use such third-party IPC mechanisms. - -The 3A component shall not directly access any device node on the system. -Such accesses shall instead be performed through the 3A API. The library -shall validate all accesses and restrict them to what is absolutely required -by 3A components. - -V4L2 Compatibility Layer ------------------------- - -The project shall support traditional V4L2 application through an additional -libcamera wrapper library. The wrapper library shall trap all accesses to -camera devices through `LD_PRELOAD`, and route them through libcamera to -emulate a high-level V4L2 camera device. It shall expose camera device -features on a best-effort basis, and aim for the level of features -traditionally available from a UVC camera designed for video conferencing. - -Android Camera HAL v3 Compatibility ------------------------------------ - -The library API shall expose all the features required to implement an -Android Camera HAL v3 on top of libcamera. Some features of the HAL may be -omitted as long as they can be implemented separately in the HAL, such as -JPEG encoding, or YUV reprocessing. - - Camera Stack ============ @@ -269,138 +126,3 @@ Native libcamera API followed in the :doc:`Application writer's guide ` .. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html - -libcamera Architecture -====================== - -:: - - ---------------------------< libcamera Public API >--------------------------- - ^ ^ - | | - v v - +-------------+ +-------------------------------------------------+ - | Camera | | Camera Device | - | Devices | | +---------------------------------------------+ | - | Manager | | | Device-Agnostic | | - +-------------+ | | | | - ^ | | +------------------------+ | - | | | | ~~~~~~~~~~~~~~~~~~~~~ | - | | | | { +---------------+ } | - | | | | } | ////Image//// | { | - | | | | <-> | /Processing// | } | - | | | | } | /Algorithms// | { | - | | | | { +---------------+ } | - | | | | ~~~~~~~~~~~~~~~~~~~~~ | - | | | | ======================== | - | | | | +---------------+ | - | | | | | //Pipeline/// | | - | | | | <-> | ///Handler/// | | - | | | | | ///////////// | | - | | +--------------------+ +---------------+ | - | | Device-Specific | - | +-------------------------------------------------+ - | ^ ^ - | | | - v v v - +--------------------------------------------------------------------+ - | Helpers and Support Classes | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | - | | Support | | Allocator | | IPC | | Manager | | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | +-------------+ +-------------+ | - | | Pipeline | | ... | | - | | Runner | | | | - | +-------------+ +-------------+ | - +--------------------------------------------------------------------+ - - /// Device-Specific Components - ~~~ Sandboxing - -While offering a unified API towards upper layers, and presenting -itself as a single library, libcamera isn't monolithic. It exposes -multiple components through its public API, is built around a set of -separate helpers internally, uses device-specific components and can -load dynamic plugins. - -Camera Devices Manager - The Camera Devices Manager provides a view of available cameras - in the system. It performs cold enumeration and runtime camera - management, and supports a hotplug notification mechanism in its - public API. - - To avoid the cost associated with cold enumeration of all devices - at application start, and to arbitrate concurrent access to camera - devices, the Camera Devices Manager could later be split to a - separate service, possibly with integration in platform-specific - device management. - -Camera Device - The Camera Device represents a camera device to upper layers. It - exposes full control of the device through the public API, and is - thus the highest level object exposed by libcamera. - - Camera Device instances are created by the Camera Devices - Manager. An optional function to create new instances could be exposed - through the public API to speed up initialization when the upper - layer knows how to directly address camera devices present in the - system. - -Pipeline Handler - The Pipeline Handler manages complex pipelines exposed by the kernel drivers - through the Media Controller and V4L2 APIs. It abstracts pipeline handling to - hide device-specific details to the rest of the library, and implements both - pipeline configuration based on stream configuration, and pipeline runtime - execution and scheduling when needed by the device. - - This component is device-specific and is part of the libcamera code base. As - such it is covered by the same free software license as the rest of libcamera - and needs to be contributed upstream by device vendors. The Pipeline Handler - lives in the same process as the rest of the library, and has access to all - helpers and kernel camera-related devices. - -Image Processing Algorithms - Together with the hardware image processing and hardware statistics - collection, the Image Processing Algorithms implement 3A (Auto-Exposure, - Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU - and interact with the kernel camera devices to control hardware image - processing based on the parameters supplied by upper layers, closing the - control loop of the ISP. - - This component is device-specific and is loaded as an external plugin. It can - be part of the libcamera code base, in which case it is covered by the same - license, or provided externally as an open-source or closed-source component. - - The component is sandboxed and can only interact with libcamera through - internal APIs specifically marked as such. In particular it will have no - direct access to kernel camera devices, and all its accesses to image and - metadata will be mediated by dmabuf instances explicitly passed to the - component. The component must be prepared to run in a process separate from - the main libcamera process, and to have a very restricted view of the system, - including no access to networking APIs and limited access to file systems. - - The sandboxing mechanism isn't defined by libcamera. One example - implementation will be provided as part of the project, and platforms vendors - will be able to provide their own sandboxing mechanism as a plugin. - - libcamera should provide a basic implementation of Image Processing - Algorithms, to serve as a reference for the internal API. Device vendors are - expected to provide a full-fledged implementation compatible with their - Pipeline Handler. One goal of the libcamera project is to create an - environment in which the community will be able to compete with the - closed-source vendor binaries and develop a high quality open source - implementation. - -Helpers and Support Classes - While Pipeline Handlers are device-specific, implementations are expected to - share code due to usage of identical APIs towards the kernel camera drivers - and the Image Processing Algorithms. This includes without limitation handling - of the MC and V4L2 APIs, buffer management through dmabuf, and pipeline - discovery, configuration and scheduling. Such code will be factored out to - helpers when applicable. - - Other parts of libcamera will also benefit from factoring code out to - self-contained support classes, even if such code is present only once in the - code base, in order to keep the source code clean and easy to read. This - should be the case for instance for plugin management. diff --git a/Documentation/documentation-contents.rst b/Documentation/documentation-contents.rst index a6915e05..c7b78c43 100644 --- a/Documentation/documentation-contents.rst +++ b/Documentation/documentation-contents.rst @@ -7,12 +7,14 @@ * :doc:`/code-of-conduct` * :doc:`/coding-style` * :doc:`/environment_variables` + * :doc:`/feature_requirements` * :doc:`/guides/application-developer` * :doc:`/guides/introduction` * :doc:`/guides/ipa` * :doc:`/guides/pipeline-handler` * :doc:`/guides/tracing` * :doc:`/lens_driver_requirements` + * :doc:`/libcamera_architecture` * :doc:`/python-bindings` * :doc:`/sensor_driver_requirements` * :doc:`/software-isp-benchmarking` diff --git a/Documentation/feature_requirements.rst b/Documentation/feature_requirements.rst new file mode 100644 index 00000000..cae7e9ab --- /dev/null +++ b/Documentation/feature_requirements.rst @@ -0,0 +1,145 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. include:: documentation-contents.rst + +Feature Requirements +==================== + +Device enumeration +------------------ + +The library shall support enumerating all camera devices available in the +system, including both fixed cameras and hotpluggable cameras. It shall +support cameras plugged and unplugged after the initialization of the +library, and shall offer a mechanism to notify applications of camera plug +and unplug. + +The following types of cameras shall be supported: + +* Internal cameras designed for point-and-shoot still image and video + capture usage, either controlled directly by the CPU, or exposed through + an internal USB bus as a UVC device. + +* External UVC cameras designed for video conferencing usage. + +Other types of camera, including analog cameras, depth cameras, thermal +cameras, external digital picture or movie cameras, are out of scope for +this project. + +A hardware device that includes independent camera sensors, such as front +and back sensors in a phone, shall be considered as multiple camera devices +for the purpose of this library. + +Independent Camera Devices +-------------------------- + +When multiple cameras are present in the system and are able to operate +independently from each other, the library shall expose them as multiple +camera devices and support parallel operation without any additional usage +restriction apart from the limitations inherent to the hardware (such as +memory bandwidth, CPU usage or number of CSI-2 receivers for instance). + +Independent processes shall be able to use independent cameras devices +without interfering with each other. A single camera device shall be +usable by a single process at a time. + +Multiple streams support +------------------------ + +The library shall support multiple video streams running in parallel +for each camera device, within the limits imposed by the system. + +Per frame controls +------------------ + +The library shall support controlling capture parameters for each stream +on a per-frame basis, on a best effort basis based on the capabilities of the +hardware and underlying software stack (including kernel drivers and +firmware). It shall apply capture parameters to the frame they target, and +report the value of the parameters that have effectively been used for each +captured frame. + +When a camera device supports multiple streams, the library shall allow both +control of each stream independently, and control of multiple streams +together. Streams that are controlled together shall be synchronized. No +synchronization is required for streams controlled independently. + +Capability Enumeration +---------------------- + +The library shall expose capabilities of each camera device in a way that +allows applications to discover those capabilities dynamically. Applications +shall be allowed to cache capabilities for as long as they are using the +library. If capabilities can change at runtime, the library shall offer a +mechanism to notify applications of such changes. Applications shall not +cache capabilities in long term storage between runs. + +Capabilities shall be discovered dynamically at runtime from the device when +possible, and may come, in part or in full, from platform configuration +data. + +Device Profiles +--------------- + +The library may define different camera device profiles, each with a minimum +set of required capabilities. Applications may use those profiles to quickly +determine the level of features exposed by a device without parsing the full +list of capabilities. Camera devices may implement additional capabilities +on top of the minimum required set for the profile they expose. + +3A and Image Enhancement Algorithms +----------------------------------- + +The camera devices shall implement auto exposure, auto gain and auto white +balance. Camera devices that include a focus lens shall implement auto +focus. Additional image enhancement algorithms, such as noise reduction or +video stabilization, may be implemented. + +All algorithms may be implemented in hardware or firmware outside of the +library, or in software in the library. They shall all be controllable by +applications. + +The library shall be architectured to isolate the 3A and image enhancement +algorithms in a component with a documented API, respectively called the 3A +component and the 3A API. The 3A API shall be stable, and shall allow both +open-source and closed-source implementations of the 3A component. + +The library may include statically-linked open-source 3A components, and +shall support dynamically-linked open-source and closed-source 3A +components. + +Closed-source 3A Component Sandboxing +------------------------------------- + +For security purposes, it may be desired to run closed-source 3A components +in a separate process. The 3A API would in such a case be transported over +IPC. The 3A API shall make it possible to use any IPC mechanism that +supports passing file descriptors. + +The library may implement an IPC mechanism, and shall support third-party +platform-specific IPC mechanisms through the implementation of a +platform-specific 3A API wrapper. No modification to the library shall be +needed to use such third-party IPC mechanisms. + +The 3A component shall not directly access any device node on the system. +Such accesses shall instead be performed through the 3A API. The library +shall validate all accesses and restrict them to what is absolutely required +by 3A components. + +V4L2 Compatibility Layer +------------------------ + +The project shall support traditional V4L2 application through an additional +libcamera wrapper library. The wrapper library shall trap all accesses to +camera devices through `LD_PRELOAD`, and route them through libcamera to +emulate a high-level V4L2 camera device. It shall expose camera device +features on a best-effort basis, and aim for the level of features +traditionally available from a UVC camera designed for video conferencing. + +Android Camera HAL v3 Compatibility +----------------------------------- + +The library API shall expose all the features required to implement an +Android Camera HAL v3 on top of libcamera. Some features of the HAL may be +omitted as long as they can be implemented separately in the HAL, such as +JPEG encoding, or YUV reprocessing. diff --git a/Documentation/index.rst b/Documentation/index.rst index 52ddc494..95c80b4e 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -18,8 +18,10 @@ Camera Sensor Model Developer Guide Environment variables + Feature Requirements IPA Writer's guide Lens driver requirements + libcamera Architecture Pipeline Handler Writer's Guide Python Bindings Sensor driver requirements diff --git a/Documentation/libcamera_architecture.rst b/Documentation/libcamera_architecture.rst new file mode 100644 index 00000000..1258db23 --- /dev/null +++ b/Documentation/libcamera_architecture.rst @@ -0,0 +1,138 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. include:: documentation-contents.rst + +libcamera Architecture +====================== + +:: + + ---------------------------< libcamera Public API >--------------------------- + ^ ^ + | | + v v + +-------------+ +-------------------------------------------------+ + | Camera | | Camera Device | + | Devices | | +---------------------------------------------+ | + | Manager | | | Device-Agnostic | | + +-------------+ | | | | + ^ | | +------------------------+ | + | | | | ~~~~~~~~~~~~~~~~~~~~~ | + | | | | { +---------------+ } | + | | | | } | ////Image//// | { | + | | | | <-> | /Processing// | } | + | | | | } | /Algorithms// | { | + | | | | { +---------------+ } | + | | | | ~~~~~~~~~~~~~~~~~~~~~ | + | | | | ======================== | + | | | | +---------------+ | + | | | | | //Pipeline/// | | + | | | | <-> | ///Handler/// | | + | | | | | ///////////// | | + | | +--------------------+ +---------------+ | + | | Device-Specific | + | +-------------------------------------------------+ + | ^ ^ + | | | + v v v + +--------------------------------------------------------------------+ + | Helpers and Support Classes | + | +-------------+ +-------------+ +-------------+ +-------------+ | + | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | + | | Support | | Allocator | | IPC | | Manager | | + | +-------------+ +-------------+ +-------------+ +-------------+ | + | +-------------+ +-------------+ | + | | Pipeline | | ... | | + | | Runner | | | | + | +-------------+ +-------------+ | + +--------------------------------------------------------------------+ + + /// Device-Specific Components + ~~~ Sandboxing + +While offering a unified API towards upper layers, and presenting +itself as a single library, libcamera isn't monolithic. It exposes +multiple components through its public API, is built around a set of +separate helpers internally, uses device-specific components and can +load dynamic plugins. + +Camera Devices Manager + The Camera Devices Manager provides a view of available cameras + in the system. It performs cold enumeration and runtime camera + management, and supports a hotplug notification mechanism in its + public API. + + To avoid the cost associated with cold enumeration of all devices + at application start, and to arbitrate concurrent access to camera + devices, the Camera Devices Manager could later be split to a + separate service, possibly with integration in platform-specific + device management. + +Camera Device + The Camera Device represents a camera device to upper layers. It + exposes full control of the device through the public API, and is + thus the highest level object exposed by libcamera. + + Camera Device instances are created by the Camera Devices + Manager. An optional function to create new instances could be exposed + through the public API to speed up initialization when the upper + layer knows how to directly address camera devices present in the + system. + +Pipeline Handler + The Pipeline Handler manages complex pipelines exposed by the kernel drivers + through the Media Controller and V4L2 APIs. It abstracts pipeline handling to + hide device-specific details to the rest of the library, and implements both + pipeline configuration based on stream configuration, and pipeline runtime + execution and scheduling when needed by the device. + + This component is device-specific and is part of the libcamera code base. As + such it is covered by the same free software license as the rest of libcamera + and needs to be contributed upstream by device vendors. The Pipeline Handler + lives in the same process as the rest of the library, and has access to all + helpers and kernel camera-related devices. + +Image Processing Algorithms + Together with the hardware image processing and hardware statistics + collection, the Image Processing Algorithms implement 3A (Auto-Exposure, + Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU + and interact with the kernel camera devices to control hardware image + processing based on the parameters supplied by upper layers, closing the + control loop of the ISP. + + This component is device-specific and is loaded as an external plugin. It can + be part of the libcamera code base, in which case it is covered by the same + license, or provided externally as an open-source or closed-source component. + + The component is sandboxed and can only interact with libcamera through + internal APIs specifically marked as such. In particular it will have no + direct access to kernel camera devices, and all its accesses to image and + metadata will be mediated by dmabuf instances explicitly passed to the + component. The component must be prepared to run in a process separate from + the main libcamera process, and to have a very restricted view of the system, + including no access to networking APIs and limited access to file systems. + + The sandboxing mechanism isn't defined by libcamera. One example + implementation will be provided as part of the project, and platforms vendors + will be able to provide their own sandboxing mechanism as a plugin. + + libcamera should provide a basic implementation of Image Processing + Algorithms, to serve as a reference for the internal API. Device vendors are + expected to provide a full-fledged implementation compatible with their + Pipeline Handler. One goal of the libcamera project is to create an + environment in which the community will be able to compete with the + closed-source vendor binaries and develop a high quality open source + implementation. + +Helpers and Support Classes + While Pipeline Handlers are device-specific, implementations are expected to + share code due to usage of identical APIs towards the kernel camera drivers + and the Image Processing Algorithms. This includes without limitation handling + of the MC and V4L2 APIs, buffer management through dmabuf, and pipeline + discovery, configuration and scheduling. Such code will be factored out to + helpers when applicable. + + Other parts of libcamera will also benefit from factoring code out to + self-contained support classes, even if such code is present only once in the + code base, in order to keep the source code clean and easy to read. This + should be the case for instance for plugin management. diff --git a/Documentation/meson.build b/Documentation/meson.build index 79135b6f..54788d6d 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -131,6 +131,7 @@ if sphinx.found() 'docs.rst', 'documentation-contents.rst', 'environment_variables.rst', + 'feature_requirements.rst', 'guides/application-developer.rst', 'guides/introduction.rst', 'guides/ipa.rst', @@ -138,6 +139,7 @@ if sphinx.found() 'guides/tracing.rst', 'index.rst', 'lens_driver_requirements.rst', + 'libcamera_architecture.rst', 'python-bindings.rst', 'sensor_driver_requirements.rst', 'software-isp-benchmarking.rst', From patchwork Mon Aug 19 16:09:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20954 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DE234BDB13 for ; Mon, 19 Aug 2024 16:09:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 03B35633C1; Mon, 19 Aug 2024 18:09:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="iFX4TQKw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 168E4633C1 for ; Mon, 19 Aug 2024 18:09:37 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B11915A5; Mon, 19 Aug 2024 18:08:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083715; bh=TrDc01eO9c+sVGG5O8dzL1QpNFMM5onIzd0IOK2lLR4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iFX4TQKw7jz1uccpsQBP8nlUsqH6ZxC2MEoYAKGBBGgnrCYMRe83V8roTJn36jOJO 0xe2dfuAS6Oxt93YcgUt7BvP7dQkrzdwwR0aN/1u4sRqNJ3O+AukJhyR2qFnfu/4wv gb+39TIxvc4p4OuCbH2a+TmgNxBe42Sr/Xb4XMQk= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally Subject: [PATCH v3 5/7] Documentation: Remove libcamera architecture from introduction.rst Date: Mon, 19 Aug 2024 17:09:19 +0100 Message-Id: <20240819160921.468981-6-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libcamera Architecture section of the introduction is largely a duplicate of the section broken out from docs.rst. Remove it from the introduction.rst file and consolidate anything that wasn't duplicated into libcamera_architecture.rst. Take the opportunity to expand the list of Platform Support which is now a bit out of date. Signed-off-by: Daniel Scally --- Changes since v2: - Reworked to change libcamera architecture section instead of camera stack. DDropped R-b - Updated the list of supported platforms on the new libcamera architecture page Changes since v1: - None Documentation/guides/introduction.rst | 156 +------------- Documentation/libcamera_architecture.rst | 261 ++++++++++++----------- 2 files changed, 141 insertions(+), 276 deletions(-) diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst index d1e67654..12d1b7d4 100644 --- a/Documentation/guides/introduction.rst +++ b/Documentation/guides/introduction.rst @@ -26,10 +26,8 @@ desirable results from the camera. .. _Media Controller: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/mediactl/media-controller.html -In this developers guide, we will explore the internal `Architecture`_ of -the libcamera library with its components. The current `Platform Support`_ is -detailed, as well as an overview of the `Licensing`_ requirements of the -project. +In this developers guide the `Licensing`_ requirements of the project are +detailed. This introduction is followed by a walkthrough tutorial to newcomers wishing to support a new platform with the `Pipeline Handler Writers Guide`_ and for those @@ -41,156 +39,6 @@ provides a tutorial of the key APIs exposed by libcamera. .. TODO: Correctly link to the other articles of the guide -Architecture ------------- - -While offering a unified API towards upper layers, and presenting itself as a -single library, libcamera isn't monolithic. It exposes multiple components -through its public API and is built around a set of separate helpers internally. -Hardware abstractions are handled through the use of device-specific components -where required and dynamically loadable plugins are used to separate image -processing algorithms from the core libcamera codebase. - -:: - - --------------------------< libcamera Public API >--------------------------- - ^ ^ - | | - v v - +-------------+ +---------------------------------------------------+ - | Camera | | Camera Device | - | Manager | | +-----------------------------------------------+ | - +-------------+ | | Device-Agnostic | | - ^ | | | | - | | | +--------------------------+ | - | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | - | | | | { +-----------------+ } | - | | | | } | //// Image //// | { | - | | | | <-> | / Processing // | } | - | | | | } | / Algorithms // | { | - | | | | { +-----------------+ } | - | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | - | | | | ========================== | - | | | | +-----------------+ | - | | | | | // Pipeline /// | | - | | | | <-> | /// Handler /// | | - | | | | | /////////////// | | - | | +--------------------+ +-----------------+ | - | | Device-Specific | - | +---------------------------------------------------+ - | ^ ^ - | | | - v v v - +--------------------------------------------------------------------+ - | Helpers and Support Classes | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | - | | Support | | Allocator | | IPC | | Manager | | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | +-------------+ +-------------+ | - | | Pipeline | | ... | | - | | Runner | | | | - | +-------------+ +-------------+ | - +--------------------------------------------------------------------+ - - /// Device-Specific Components - ~~~ Sandboxing - - -Camera Manager - The Camera Manager enumerates cameras and instantiates Pipeline Handlers to - manage each Camera that libcamera supports. The Camera Manager supports - hotplug detection and notification events when supported by the underlying - kernel devices. - - There is only ever one instance of the Camera Manager running per application. - Each application's instance of the Camera Manager ensures that only a single - application can take control of a camera device at once. - - Read the `Camera Manager API`_ documentation for more details. - -.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html - -Camera Device - The Camera class represents a single item of camera hardware that is capable - of producing one or more image streams, and provides the API to interact with - the underlying device. - - If a system has multiple instances of the same hardware attached, each has its - own instance of the camera class. - - The API exposes full control of the device to upper layers of libcamera through - the public API, making it the highest level object libcamera exposes, and the - object that all other API operations interact with from configuration to - capture. - - Read the `Camera API`_ documentation for more details. - -.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html - -Pipeline Handler - The Pipeline Handler manages the complex pipelines exposed by the kernel - drivers through the Media Controller and V4L2 APIs. It abstracts pipeline - handling to hide device-specific details from the rest of the library, and - implements both pipeline configuration based on stream configuration, and - pipeline runtime execution and scheduling when needed by the device. - - The Pipeline Handler lives in the same process as the rest of the library, and - has access to all helpers and kernel camera-related devices. - - Hardware abstraction is handled by device specific Pipeline Handlers which are - derived from the Pipeline Handler base class allowing commonality to be shared - among the implementations. - - Derived pipeline handlers create Camera device instances based on the devices - they detect and support on the running system, and are responsible for - managing the interactions with a camera device. - - More details can be found in the `PipelineHandler API`_ documentation, and the - `Pipeline Handler Writers Guide`_. - -.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html - -Image Processing Algorithms - An image processing algorithm (IPA) component is a loadable plugin that - implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other - algorithms. - - The algorithms run on the CPU and interact with the camera devices through the - Pipeline Handler to control hardware image processing based on the parameters - supplied by upper layers, maintaining state and closing the control loop - of the ISP. - - The component is sandboxed and can only interact with libcamera through the - API provided by the Pipeline Handler and an IPA has no direct access to kernel - camera devices. - - Open source IPA modules built with libcamera can be run in the same process - space as libcamera, however external IPA modules are run in a separate process - from the main libcamera process. IPA modules have a restricted view of the - system, including no access to networking APIs and limited access to file - systems. - - IPA modules are only required for platforms and devices with an ISP controlled - by the host CPU. Camera sensors which have an integrated ISP are not - controlled through the IPA module. - -Platform Support ----------------- - -The library currently supports the following hardware platforms specifically -with dedicated pipeline handlers: - - - Intel IPU3 (ipu3) - - Rockchip RK3399 (rkisp1) - - RaspberryPi 3 and 4 (rpi/vc4) - -Furthermore, generic platform support is provided for the following: - - - USB video device class cameras (uvcvideo) - - iMX7, Allwinner Sun6i (simple) - - Virtual media controller driver for test use cases (vimc) - Licensing --------- diff --git a/Documentation/libcamera_architecture.rst b/Documentation/libcamera_architecture.rst index 1258db23..699a1347 100644 --- a/Documentation/libcamera_architecture.rst +++ b/Documentation/libcamera_architecture.rst @@ -5,134 +5,151 @@ libcamera Architecture ====================== +While offering a unified API towards upper layers, and presenting itself as a +single library, libcamera isn't monolithic. It exposes multiple components +through its public API and is built around a set of separate helpers internally. +Hardware abstractions are handled through the use of device-specific components +where required and dynamically loadable plugins are used to separate image +processing algorithms from the core libcamera codebase. + :: - ---------------------------< libcamera Public API >--------------------------- - ^ ^ - | | - v v - +-------------+ +-------------------------------------------------+ - | Camera | | Camera Device | - | Devices | | +---------------------------------------------+ | - | Manager | | | Device-Agnostic | | - +-------------+ | | | | - ^ | | +------------------------+ | - | | | | ~~~~~~~~~~~~~~~~~~~~~ | - | | | | { +---------------+ } | - | | | | } | ////Image//// | { | - | | | | <-> | /Processing// | } | - | | | | } | /Algorithms// | { | - | | | | { +---------------+ } | - | | | | ~~~~~~~~~~~~~~~~~~~~~ | - | | | | ======================== | - | | | | +---------------+ | - | | | | | //Pipeline/// | | - | | | | <-> | ///Handler/// | | - | | | | | ///////////// | | - | | +--------------------+ +---------------+ | - | | Device-Specific | - | +-------------------------------------------------+ - | ^ ^ - | | | - v v v - +--------------------------------------------------------------------+ - | Helpers and Support Classes | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | - | | Support | | Allocator | | IPC | | Manager | | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | +-------------+ +-------------+ | - | | Pipeline | | ... | | - | | Runner | | | | - | +-------------+ +-------------+ | - +--------------------------------------------------------------------+ - - /// Device-Specific Components - ~~~ Sandboxing - -While offering a unified API towards upper layers, and presenting -itself as a single library, libcamera isn't monolithic. It exposes -multiple components through its public API, is built around a set of -separate helpers internally, uses device-specific components and can -load dynamic plugins. - -Camera Devices Manager - The Camera Devices Manager provides a view of available cameras - in the system. It performs cold enumeration and runtime camera - management, and supports a hotplug notification mechanism in its - public API. - - To avoid the cost associated with cold enumeration of all devices - at application start, and to arbitrate concurrent access to camera - devices, the Camera Devices Manager could later be split to a - separate service, possibly with integration in platform-specific - device management. + --------------------------< libcamera Public API >--------------------------- + ^ ^ + | | + v v + +-------------+ +---------------------------------------------------+ + | Camera | | Camera Device | + | Manager | | +-----------------------------------------------+ | + +-------------+ | | Device-Agnostic | | + ^ | | | | + | | | +--------------------------+ | + | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | + | | | | { +-----------------+ } | + | | | | } | //// Image //// | { | + | | | | <-> | / Processing // | } | + | | | | } | / Algorithms // | { | + | | | | { +-----------------+ } | + | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | + | | | | ========================== | + | | | | +-----------------+ | + | | | | | // Pipeline /// | | + | | | | <-> | /// Handler /// | | + | | | | | /////////////// | | + | | +--------------------+ +-----------------+ | + | | Device-Specific | + | +---------------------------------------------------+ + | ^ ^ + | | | + v v v + +--------------------------------------------------------------------+ + | Helpers and Support Classes | + | +-------------+ +-------------+ +-------------+ +-------------+ | + | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | + | | Support | | Allocator | | IPC | | Manager | | + | +-------------+ +-------------+ +-------------+ +-------------+ | + | +-------------+ +-------------+ | + | | Pipeline | | ... | | + | | Runner | | | | + | +-------------+ +-------------+ | + +--------------------------------------------------------------------+ + + /// Device-Specific Components + ~~~ Sandboxing + + +Camera Manager + The Camera Manager enumerates cameras and instantiates Pipeline Handlers to + manage each Camera that libcamera supports. The Camera Manager supports + hotplug detection and notification events when supported by the underlying + kernel devices. + + There is only ever one instance of the Camera Manager running per application. + Each application's instance of the Camera Manager ensures that only a single + application can take control of a camera device at once. + + Read the `Camera Manager API`_ documentation for more details. + +.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html Camera Device - The Camera Device represents a camera device to upper layers. It - exposes full control of the device through the public API, and is - thus the highest level object exposed by libcamera. + The Camera class represents a single item of camera hardware that is capable + of producing one or more image streams, and provides the API to interact with + the underlying device. + + If a system has multiple instances of the same hardware attached, each has its + own instance of the camera class. + + The API exposes full control of the device to upper layers of libcamera through + the public API, making it the highest level object libcamera exposes, and the + object that all other API operations interact with from configuration to + capture. - Camera Device instances are created by the Camera Devices - Manager. An optional function to create new instances could be exposed - through the public API to speed up initialization when the upper - layer knows how to directly address camera devices present in the - system. + Read the `Camera API`_ documentation for more details. + +.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html Pipeline Handler - The Pipeline Handler manages complex pipelines exposed by the kernel drivers - through the Media Controller and V4L2 APIs. It abstracts pipeline handling to - hide device-specific details to the rest of the library, and implements both - pipeline configuration based on stream configuration, and pipeline runtime - execution and scheduling when needed by the device. - - This component is device-specific and is part of the libcamera code base. As - such it is covered by the same free software license as the rest of libcamera - and needs to be contributed upstream by device vendors. The Pipeline Handler - lives in the same process as the rest of the library, and has access to all - helpers and kernel camera-related devices. + The Pipeline Handler manages the complex pipelines exposed by the kernel + drivers through the Media Controller and V4L2 APIs. It abstracts pipeline + handling to hide device-specific details from the rest of the library, and + implements both pipeline configuration based on stream configuration, and + pipeline runtime execution and scheduling when needed by the device. + + The Pipeline Handler lives in the same process as the rest of the library, and + has access to all helpers and kernel camera-related devices. + + Hardware abstraction is handled by device specific Pipeline Handlers which are + derived from the Pipeline Handler base class allowing commonality to be shared + among the implementations. + + Derived pipeline handlers create Camera device instances based on the devices + they detect and support on the running system, and are responsible for + managing the interactions with a camera device. + + More details can be found in the `PipelineHandler API`_ documentation, and the + :doc:`Pipeline Handler Writers Guide `. + +.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html Image Processing Algorithms - Together with the hardware image processing and hardware statistics - collection, the Image Processing Algorithms implement 3A (Auto-Exposure, - Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU - and interact with the kernel camera devices to control hardware image - processing based on the parameters supplied by upper layers, closing the - control loop of the ISP. - - This component is device-specific and is loaded as an external plugin. It can - be part of the libcamera code base, in which case it is covered by the same - license, or provided externally as an open-source or closed-source component. - - The component is sandboxed and can only interact with libcamera through - internal APIs specifically marked as such. In particular it will have no - direct access to kernel camera devices, and all its accesses to image and - metadata will be mediated by dmabuf instances explicitly passed to the - component. The component must be prepared to run in a process separate from - the main libcamera process, and to have a very restricted view of the system, - including no access to networking APIs and limited access to file systems. - - The sandboxing mechanism isn't defined by libcamera. One example - implementation will be provided as part of the project, and platforms vendors - will be able to provide their own sandboxing mechanism as a plugin. - - libcamera should provide a basic implementation of Image Processing - Algorithms, to serve as a reference for the internal API. Device vendors are - expected to provide a full-fledged implementation compatible with their - Pipeline Handler. One goal of the libcamera project is to create an - environment in which the community will be able to compete with the - closed-source vendor binaries and develop a high quality open source - implementation. - -Helpers and Support Classes - While Pipeline Handlers are device-specific, implementations are expected to - share code due to usage of identical APIs towards the kernel camera drivers - and the Image Processing Algorithms. This includes without limitation handling - of the MC and V4L2 APIs, buffer management through dmabuf, and pipeline - discovery, configuration and scheduling. Such code will be factored out to - helpers when applicable. - - Other parts of libcamera will also benefit from factoring code out to - self-contained support classes, even if such code is present only once in the - code base, in order to keep the source code clean and easy to read. This - should be the case for instance for plugin management. + An image processing algorithm (IPA) component is a loadable plugin that + implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other + algorithms. + + The algorithms run on the CPU and interact with the camera devices through the + Pipeline Handler to control hardware image processing based on the parameters + supplied by upper layers, maintaining state and closing the control loop + of the ISP. + + The component is sandboxed and can only interact with libcamera through the + API provided by the Pipeline Handler and an IPA has no direct access to kernel + camera devices. + + Open source IPA modules built with libcamera can be run in the same process + space as libcamera, however external IPA modules are run in a separate process + from the main libcamera process. IPA modules have a restricted view of the + system, including no access to networking APIs and limited access to file + systems. + + IPA modules are only required for platforms and devices with an ISP controlled + by the host CPU. Camera sensors which have an integrated ISP are not + controlled through the IPA module. + +Platform Support +---------------- + +The library currently supports the following hardware platforms specifically +with dedicated pipeline handlers: + + - Arm Mali-C55 + - Intel IPU3 (ipu3) + - NXP i.MX8MP (imx8-isi and rkisp1) + - Rockchip RK3399 (rkisp1) + - RaspberryPi 3, 4 and zero (rpi/vc4) + +Furthermore, generic platform support is provided for the following: + + - USB video device class cameras (uvcvideo) + - iMX7, IPU6, Allwinner Sun6i (simple) + - Virtual media controller driver for test use cases (vimc) From patchwork Mon Aug 19 16:09:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20955 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EDCCDC32A9 for ; Mon, 19 Aug 2024 16:09:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AEC4F633CC; Mon, 19 Aug 2024 18:09:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FH2p29ZE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 613F7633B3 for ; Mon, 19 Aug 2024 18:09:37 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 00B235B3; Mon, 19 Aug 2024 18:08:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083716; bh=1E3oikp8Svp2DWntfJc3sZSwxYHrksZg1Jac8DJejRE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FH2p29ZEXTtXQx0NdzzofamcGrSC6gsI0yKb4rc68BuXbyJb8uXSODW15TTOrYscF x/CcpeDxCvYjYprVhBjFdZi2hXHMQgcm6/BeXRCLlNbDNTsFvSa371bjs4ebapJ49j mjmnPjSrvF/FbrHKybJkJwGkPCDdMcwEHfQwPdH4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally Subject: [PATCH v3 6/7] Documentation: Rework docs.rst into introduction.rst Date: Mon, 19 Aug 2024 17:09:20 +0100 Message-Id: <20240819160921.468981-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" docs.rst is the landing page for the documentation from the libcamera website, but isn't particularly introductory. Move much of the content from guides/introduction.rst to docs.rst, which will serve as the new introductory page. Remove guides/introduction.rst. Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes since v2: - Renamed docs.rst to introduction.rst - Switched to use graphviz extension instead of including an SVG file directly. - Quite a bit of re-wording - Kept the camera stack section instead of libcamera architecture Changes since v1: - Removed deleted file from meson's docs_sources array. Documentation/conf.py | 3 + Documentation/documentation-contents.rst | 1 - Documentation/guides/introduction.rst | 62 ----------- Documentation/index.rst | 3 +- Documentation/{docs.rst => introduction.rst} | 110 ++++++++++++++++++- Documentation/mali-c55.dot | 23 ++++ Documentation/meson.build | 4 +- 7 files changed, 133 insertions(+), 73 deletions(-) delete mode 100644 Documentation/guides/introduction.rst rename Documentation/{docs.rst => introduction.rst} (54%) create mode 100644 Documentation/mali-c55.dot diff --git a/Documentation/conf.py b/Documentation/conf.py index 325f2759..089f114c 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -37,8 +37,11 @@ author = u'Kieran Bingham, Jacopo Mondi, Laurent Pinchart, Niklas Söderlund' # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ + 'sphinx.ext.graphviz' ] +graphviz_output_format = 'svg' + # Add any paths that contain templates here, relative to this directory. templates_path = [] diff --git a/Documentation/documentation-contents.rst b/Documentation/documentation-contents.rst index c7b78c43..c14389bc 100644 --- a/Documentation/documentation-contents.rst +++ b/Documentation/documentation-contents.rst @@ -9,7 +9,6 @@ * :doc:`/environment_variables` * :doc:`/feature_requirements` * :doc:`/guides/application-developer` - * :doc:`/guides/introduction` * :doc:`/guides/ipa` * :doc:`/guides/pipeline-handler` * :doc:`/guides/tracing` diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst deleted file mode 100644 index 12d1b7d4..00000000 --- a/Documentation/guides/introduction.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. SPDX-License-Identifier: CC-BY-SA-4.0 - -.. include:: ../documentation-contents.rst - -Developers guide to libcamera -============================= - -The Linux kernel handles multimedia devices through the 'Linux media' subsystem -and provides a set of APIs (application programming interfaces) known -collectively as V4L2 (`Video for Linux 2`_) and the `Media Controller`_ API -which provide an interface to interact and control media devices. - -Included in this subsystem are drivers for camera sensors, CSI2 (Camera -Serial Interface) receivers, and ISPs (Image Signal Processors) - -The usage of these drivers to provide a functioning camera stack is a -responsibility that lies in userspace which is commonly implemented separately -by vendors without a common architecture or API for application developers. - -libcamera provides a complete camera stack for Linux based systems to abstract -functionality desired by camera application developers and process the -configuration of hardware and image control algorithms required to obtain -desirable results from the camera. - -.. _Video for Linux 2: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/v4l2.html -.. _Media Controller: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/mediactl/media-controller.html - - -In this developers guide the `Licensing`_ requirements of the project are -detailed. - -This introduction is followed by a walkthrough tutorial to newcomers wishing to -support a new platform with the `Pipeline Handler Writers Guide`_ and for those -looking to make use of the libcamera native API an `Application Writers Guide`_ -provides a tutorial of the key APIs exposed by libcamera. - -.. _Pipeline Handler Writers Guide: pipeline-handler.html -.. _Application Writers Guide: application-developer.html - -.. TODO: Correctly link to the other articles of the guide - -Licensing ---------- - -The libcamera core, is covered by the `LGPL-2.1-or-later`_ license. Pipeline -Handlers are a part of the libcamera code base and need to be contributed -upstream by device vendors. IPA modules included in libcamera are covered by a -free software license, however third-parties may develop IPA modules outside of -libcamera and distribute them under a closed-source license, provided they do -not include source code from the libcamera project. - -The libcamera project itself contains multiple libraries, applications and -utilities. Licenses are expressed through SPDX tags in text-based files that -support comments, and through the .reuse/dep5 file otherwise. A copy of all -licenses are stored in the LICENSES directory, and a full summary of the -licensing used throughout the project can be found in the COPYING.rst document. - -Applications which link dynamically against libcamera and use only the public -API are an independent work of the authors and have no license restrictions -imposed upon them from libcamera. - -.. _LGPL-2.1-or-later: https://spdx.org/licenses/LGPL-2.1-or-later.html diff --git a/Documentation/index.rst b/Documentation/index.rst index 95c80b4e..3a790352 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -10,13 +10,12 @@ :caption: Contents: Home - Docs + Introduction Contribute Getting Started Application Writer's Guide Camera Sensor Model - Developer Guide Environment variables Feature Requirements IPA Writer's guide diff --git a/Documentation/docs.rst b/Documentation/introduction.rst similarity index 54% rename from Documentation/docs.rst rename to Documentation/introduction.rst index 1f0d12c0..a4994e3c 100644 --- a/Documentation/docs.rst +++ b/Documentation/introduction.rst @@ -14,12 +14,88 @@ Documentation API -API -=== - -The libcamera API is extensively documented using Doxygen. The :ref:`API -nightly build ` contains the most up-to-date API documentation, built from -the latest master branch. +What is libcamera? +================== + +libcamera is an open source complex camera support library for Linux, Android +and ChromeOS. The library interfaces with Linux kernel device drivers and +provides an intuitive API to developers in order to simplify the complexity +involved in capturing images from complex cameras on Linux systems. + +What is a "complex camera"? +=========================== + +A modern "camera" tends to infact be several different pieces of hardware which +must all be controlled together in order to produce and capture images of +appropriate quality. A hardware pipeline typically consists of a camera sensor +that captures raw frames and transmits them on a bus, a receiver that decodes +the bus signals, and an image signal processor that processes raw frames to +produce usable images in a standard format. The Linux kernel handles these multimedia devices through the +'Linux media' subsystem and provides a set of application programming interfaces +known collectively as V4L2 (`Video for Linux 2`_) and the `Media Controller`_ +APIs which provide an interface to interact and control media devices. + +.. _Video for Linux 2: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/v4l2.html +.. _Media Controller: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/mediactl/media-controller.html + +Included in this subsystem are drivers for camera sensors, CSI2 (Camera +Serial Interface) receivers, and ISPs (Image Signal Processors). + +The usage of these drivers to provide a functioning camera stack is a +responsibility that lies in userspace which is commonly implemented separately +by vendors without a common architecture or API for application developers. This +adds a lot of complexity to the task, particularly when considering that the +differences in hardware pipelines and their representation in the kernel's APIs +often necessitate bespoke handling. + +What is libcamera for? +====================== + +libcamera provides a complete camera stack for Linux-based systems to abstract +the configuration of hardware and image control algorithms required to obtain +desirable results from the camera through the kernel's APIs, reducing those +operations to a simple and consistent method for developers. In short instead of +having to deal with this: + +.. graphviz:: mali-c55.dot + +You can instead simply deal with this: + +.. code-block:: python + + >>> import libcamera as lc + >>> camera_manager = lc.CameraManager.singleton() + [0:15:59.582029920] [504] INFO Camera camera_manager.cpp:313 libcamera v0.3.0+182-01e57380 + >>> for camera in camera_manager.cameras: + ... print(f' - {camera.id}') + ... + - mali-c55 tpg + - imx415 1-001a + +And the library handles the rest for you. These documentary pages give more +information on the internal workings of libcamera (and the kernel camera stack +that lies behind it) as well as guidance on using libcamera in an application or +extending the library with support for your hardware (through the pipeline +handler and IPA module writer's guides). + +How should I use it? +==================== + +There are a few ways you might want to use libcamera, depending on your +application. It's always possible to use the library directly of course, and you +can find detailed information on how to do so in the +:doc:`application writer's guide `. It may be more +appropriate to use one of the frameworks with libcamera support. For example an +application powering an embedded media device incorporating capture, encoding +and streaming of both video and audio might benefit from using `GStreamer`_, for +which libcamera provides a plugin. Similarly an application for user-facing +devices like a laptop would likely benefit accessing cameras through the XDG +camera portal and `pipewire`_, which brings the advantages of resource sharing +(multiple applications accessing the stream at the same time) and access +control. + +.. _GStreamer: https://gstreamer.freedesktop.org/ +.. _pipewire: https://pipewire.org/ Camera Stack ============ @@ -126,3 +202,25 @@ Native libcamera API followed in the :doc:`Application writer's guide ` .. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html + +Licensing +========= + +The libcamera core, is covered by the `LGPL-2.1-or-later`_ license. Pipeline +Handlers are a part of the libcamera code base and need to be contributed +upstream by device vendors. IPA modules included in libcamera are covered by a +free software license, however third-parties may develop IPA modules outside of +libcamera and distribute them under a closed-source license, provided they do +not include source code from the libcamera project. + +The libcamera project itself contains multiple libraries, applications and +utilities. Licenses are expressed through SPDX tags in text-based files that +support comments, and through the .reuse/dep5 file otherwise. A copy of all +licenses are stored in the LICENSES directory, and a full summary of the +licensing used throughout the project can be found in the COPYING.rst document. + +Applications which link dynamically against libcamera and use only the public +API are an independent work of the authors and have no license restrictions +imposed upon them from libcamera. + +.. _LGPL-2.1-or-later: https://spdx.org/licenses/LGPL-2.1-or-later.html diff --git a/Documentation/mali-c55.dot b/Documentation/mali-c55.dot new file mode 100644 index 00000000..0b59a488 --- /dev/null +++ b/Documentation/mali-c55.dot @@ -0,0 +1,23 @@ +digraph board { + rankdir=TB + n00000001 [label="{{} | mali-c55 tpg\n/dev/v4l-subdev0 | { 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port0 -> n00000003:port0 [style=dashed] + n00000003 [label="{{ 0 | 4} | mali-c55 isp\n/dev/v4l-subdev1 | { 1 | 2 | 3}}", shape=Mrecord, style=filled, fillcolor=green] + n00000003:port1 -> n00000009:port0 [style=bold] + n00000003:port2 -> n00000009:port2 [style=bold] + n00000003:port1 -> n0000000d:port0 [style=bold] + n00000003:port3 -> n0000001c + n00000009 [label="{{ 0 | 2} | mali-c55 resizer fr\n/dev/v4l-subdev2 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000009:port1 -> n00000010 + n0000000d [label="{{ 0} | mali-c55 resizer ds\n/dev/v4l-subdev3 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n0000000d:port1 -> n00000014 + n00000010 [label="mali-c55 fr\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n00000014 [label="mali-c55 ds\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000018 [label="mali-c55 3a params\n/dev/video2", shape=box, style=filled, fillcolor=yellow] + n00000018 -> n00000003:port4 + n0000001c [label="mali-c55 3a stats\n/dev/video3", shape=box, style=filled, fillcolor=yellow] + n00000030 [label="{{ 0} | lte-csi2-rx\n/dev/v4l-subdev4 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000030:port1 -> n00000003:port0 + n00000035 [label="{{} | imx415 1-001a\n/dev/v4l-subdev5 | { 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000035:port0 -> n00000030:port0 [style=bold] +} diff --git a/Documentation/meson.build b/Documentation/meson.build index 54788d6d..36ffae23 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -128,18 +128,18 @@ if sphinx.found() 'coding-style.rst', 'conf.py', 'contributing.rst', - 'docs.rst', 'documentation-contents.rst', 'environment_variables.rst', 'feature_requirements.rst', 'guides/application-developer.rst', - 'guides/introduction.rst', 'guides/ipa.rst', 'guides/pipeline-handler.rst', 'guides/tracing.rst', 'index.rst', + 'introduction.rst', 'lens_driver_requirements.rst', 'libcamera_architecture.rst', + 'mali-c55.dot', 'python-bindings.rst', 'sensor_driver_requirements.rst', 'software-isp-benchmarking.rst', From patchwork Mon Aug 19 16:09:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20956 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C54C5BDB13 for ; Mon, 19 Aug 2024 16:09:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0A355633CF; Mon, 19 Aug 2024 18:09:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Srw13up4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A6646633C2 for ; Mon, 19 Aug 2024 18:09:37 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 403CA5A5; Mon, 19 Aug 2024 18:08:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1724083716; bh=5rjo2xdKLu6VuPTmo2St4bV/3eS8o3vPgomy2OfWJGY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Srw13up4OKKpo0jxe909TfPLEAJyBIIsfk6IFfezK6g1pEHXObOQc3ut2H2HJICgP 5JbrxlDE7HQZEyRl9EbDu2Y8Ms5ZeJo61L6og888OIJ6nr+6t60cONqJwdXUbipcVW 0IZTD58UZZVHZ7l9c01oCV0UmYR2zFL9pF8PfbXU= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Laurent Pinchart Subject: [PATCH v3 7/7] Documentation: Rework index.rst Date: Mon, 19 Aug 2024 17:09:21 +0100 Message-Id: <20240819160921.468981-8-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240819160921.468981-1-dan.scally@ideasonboard.com> References: <20240819160921.468981-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" index.rst is the page that becomes index.html, but currently just has some blurb. Although this page will only be seen if viewing the docs as built from the libcamera tree it'd be better if it were more of an introductory page. Include the content of docs.rst to improve it. As we're no longer including the content from README.rst the labels that enabled that can be dropped. With this change whether viewing the documentation as built in the libcamera tree or on the Docs page of the website, the landing content will be the same. The CSS for the documentation's theme currently hides the toctree from the generated body in html, as it's already displayed on every page via the theme's CSS. This change reorders the page such that the CSS that hides the toctree no longer works - update the CSS to retain the current behaviour. Reviewed-by: Laurent Pinchart Signed-off-by: Daniel Scally --- Changes since v2: - None Changes since v1: - Removed superfluous comments - Expanded commit message Documentation/index.rst | 11 ++++++----- Documentation/theme/static/css/theme.css | 2 +- README.rst | 3 --- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/Documentation/index.rst b/Documentation/index.rst index 3a790352..bea40660 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -1,16 +1,12 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 -.. Front page matter is defined in the project README file. -.. include:: ../README.rst - :start-after: .. section-begin-libcamera - :end-before: .. section-end-libcamera +.. include:: introduction.rst .. toctree:: :maxdepth: 1 :caption: Contents: Home - Introduction Contribute Getting Started @@ -26,3 +22,8 @@ Sensor driver requirements SoftwareISP Benchmarking Tracing guide + +.. toctree:: + :hidden: + + introduction \ No newline at end of file diff --git a/Documentation/theme/static/css/theme.css b/Documentation/theme/static/css/theme.css index 2b1ed095..a6d43195 100644 --- a/Documentation/theme/static/css/theme.css +++ b/Documentation/theme/static/css/theme.css @@ -283,7 +283,7 @@ div#signature { font-size: 12px; } -#libcamera div.toctree-wrapper { +#licensing div.toctree-wrapper { height: 0px; margin: 0px; padding: 0px; diff --git a/README.rst b/README.rst index 92f16502..4068c6cc 100644 --- a/README.rst +++ b/README.rst @@ -1,7 +1,5 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 -.. section-begin-libcamera - =========== libcamera =========== @@ -22,7 +20,6 @@ open-source-friendly while still protecting vendor core IP. libcamera was born out of that collaboration and will offer modern camera support to Linux-based systems, including traditional Linux distributions, ChromeOS and Android. -.. section-end-libcamera .. section-begin-getting-started Getting Started