Patch Detail
Show a patch.
GET /api/patches/9338/?format=api
{ "id": 9338, "url": "https://patchwork.libcamera.org/api/patches/9338/?format=api", "web_url": "https://patchwork.libcamera.org/patch/9338/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20200820134751.278033-2-kieran.bingham@ideasonboard.com>", "date": "2020-08-20T13:47:49", "name": "[libcamera-devel,v4,1/3] Documentation: Guides: Developers Guide to Libcamera", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "57f65a763b8cf8289d38dfa2e1a18fa276c7da5f", "submitter": { "id": 4, "url": "https://patchwork.libcamera.org/api/people/4/?format=api", "name": "Kieran Bingham", "email": "kieran.bingham@ideasonboard.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/9338/mbox/", "series": [ { "id": 1225, "url": "https://patchwork.libcamera.org/api/series/1225/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=1225", "date": "2020-08-20T13:47:48", "name": "Developer Guides", "version": 4, "mbox": "https://patchwork.libcamera.org/series/1225/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/9338/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/9338/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C04C4BE173\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu, 20 Aug 2020 13:47:59 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 791A961FE7;\n\tThu, 20 Aug 2020 15:47:59 +0200 (CEST)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4F49760381\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 20 Aug 2020 15:47:57 +0200 (CEST)", "from Q.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net\n\t[86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B17C9329;\n\tThu, 20 Aug 2020 15:47:56 +0200 (CEST)" ], "Authentication-Results": "lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"AXm5BLm+\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597931277;\n\tbh=chTvLguGK5lwpgNpV5CZqy8olyZPydGled1PQlfSzPY=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=AXm5BLm++Tqf1bOtFxO2UqFqc09hF5ThYwHdK1pdTWak9W58bGzY5Y51eECHvZbbb\n\tPA9rv7srNG93INg831NN3vhyRFdBlzJEfg3yRQ/llUXi21kqivoU4Y+OCdfk9BQj8R\n\t2QnkY5nmq44lB145hLE/ibBgBvQ2tI7ZE+JFb/bM=", "From": "Kieran Bingham <kieran.bingham@ideasonboard.com>", "To": "libcamera devel <libcamera-devel@lists.libcamera.org>,\n\tChris Ward <chris@gregariousmammal.com>, Jacopo Mondi <jacopo@jmondi.org>", "Date": "Thu, 20 Aug 2020 14:47:49 +0100", "Message-Id": "<20200820134751.278033-2-kieran.bingham@ideasonboard.com>", "X-Mailer": "git-send-email 2.25.1", "In-Reply-To": "<20200820134751.278033-1-kieran.bingham@ideasonboard.com>", "References": "<20200820134751.278033-1-kieran.bingham@ideasonboard.com>", "MIME-Version": "1.0", "Subject": "[libcamera-devel] [PATCH v4 1/3] Documentation: Guides: Developers\n\tGuide to Libcamera", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"utf-8\"", "Content-Transfer-Encoding": "base64", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "From: Chris Chinchilla <chris@gregariousmammal.com>\n\nCreate an introduction and overview for new developers to libcamera.\n\nProvide an overview of the Camera Stack, and Architecture of libcamera\nand introduce the main concepts of libcamera.\n\nSigned-off-by: Chris Chinchilla <chris@gregariousmammal.com>\n[Kieran: Rework/Reflow, add diagrams, licensing]\nSigned-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>\n\n---\n Documentation/guides/introduction.rst | 319 ++++++++++++++++++++++++++\n Documentation/index.rst | 2 +\n Documentation/meson.build | 1 +\n 3 files changed, 322 insertions(+)\n create mode 100644 Documentation/guides/introduction.rst", "diff": "diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst\nnew file mode 100644\nindex 000000000000..f34d2cf7cbdc\n--- /dev/null\n+++ b/Documentation/guides/introduction.rst\n@@ -0,0 +1,319 @@\n+.. SPDX-License-Identifier: CC-BY-SA-4.0\n+\n+Developers guide to libcamera\n+=============================\n+\n+The Linux kernel handles multimedia devices through the 'Linux media' subsystem\n+and provides a set of APIs (application programming interfaces) known\n+collectively as V4L2 (`Video for Linux 2`_) and the `Media Controller`_ API\n+which provide an interface to interact and control media devices.\n+\n+Included in this subsystem are drivers for camera sensors, CSI2 (Camera\n+Serial Interface) recievers, and ISPs (Image Signal Processors)\n+\n+The usage of these drivers to provide a functioning camera stack is a\n+responsibility that lies in userspace which is commonly implemented separately\n+by vendors without a common architecture or API for application developers.\n+\n+libcamera provides a complete camera stack for Linux based systems to abstract\n+functionality desired by camera application developers and process the\n+configuration of hardware and image control algorithms requried to obtain\n+desireable results from the camera.\n+\n+.. _Video for Linux 2: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/v4l2.html\n+.. _Media Controller: https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/mediactl/media-controller.html\n+\n+\n+In this developers guide, we will explore the `Camera Stack`_ and how it is\n+can be visualised at a high level, and explore the internal `Architecture`_ of\n+the libcamera library with its components. The current `Platform Support`_ is\n+detailed, as well as an overview of the `Licensing`_ requirements of the\n+project.\n+\n+This introduction is followed by a walkthrough tutorial to newcomers wishing to\n+support a new platform with the `Pipeline Handler Writers Guide`_ and for those\n+looking to make use of the libcamera native API an `Application Writers Guide`_\n+provides a tutorial of the key APIs exposed by libcamera.\n+\n+.. _Pipeline Handler Writers Guide: pipeline-handler.html\n+.. _Application Writers Guide: application-developer.html\n+\n+.. TODO: Correctly link to the other articles of the guide\n+\n+Camera Stack\n+------------\n+\n+The libcamera library is implemented in userspace, and makes use of underlying\n+kernel drivers that directly interact with hardware.\n+\n+Applications can make use of libcamera through the native `libcamera API`_'s or\n+through an adaptation layer integrating libcamera into a larger framework.\n+\n+.. _libcamera API: https://www.libcamera.org/api-html/index.html\n+\n+::\n+\n+ Application Layer\n+ / +--------------+ +--------------+ +--------------+ +--------------+\n+ | | Native | | Framework | | Native | | Android |\n+ | | V4L2 | | Application | | libcamera | | Camera |\n+ | | Application | | (gstreamer) | | Application | | Framework |\n+ \\ +--------------+ +--------------+ +--------------+ +--------------+\n+\n+ ^ ^ ^ ^\n+ | | | |\n+ | | | |\n+ v v | v\n+ Adaptation Layer |\n+ / +--------------+ +--------------+ | +--------------+\n+ | | V4L2 | | gstreamer | | | Android |\n+ | | Compatability| | element | | | Camera |\n+ | | (preload) | |(libcamerasrc)| | | HAL |\n+ \\ +--------------+ +--------------+ | +--------------+\n+ |\n+ ^ ^ | ^\n+ | | | |\n+ | | | |\n+ v v v v\n+ libcamera Framework\n+ / +--------------------------------------------------------------------+\n+ | | |\n+ | | libcamera |\n+ | | |\n+ \\ +--------------------------------------------------------------------+\n+\n+ ^ ^ ^\n+ Userspace | | |\n+ --------------------- | ---------------- | ---------------- | ---------------\n+ Kernel | | |\n+ v v v\n+\n+ +-----------+ +-----------+ +-----------+\n+ | Media | <--> | Video | <--> | V4L2 |\n+ | Device | | Device | | Subdev |\n+ +-----------+ +-----------+ +-----------+\n+\n+The camera stack comprises of four software layers. From bottom to top:\n+\n+* The kernel drivers control the camera hardware and expose a low-level\n+ interface to userspace through the Linux kernel V4L2 family of APIs\n+ (Media Controller API, V4L2 Video Device API and V4L2 Subdev API).\n+\n+* The libcamera framework is the core part of the stack. It handles all control\n+ of the camera devices in its core component, libcamera, and exposes a native\n+ C++ API to upper layers.\n+\n+* The libcamera adaptation layer is an umbrella term designating the components\n+ that interface to libcamera in other frameworks. Notable examples are the V4L2\n+ compatibility layer, the gstreamer libcamera element, and the Android camera\n+ HAL implementation based on libcamera which are provided as a part of the\n+ libcamera project.\n+\n+* The applications and upper level frameworks are based on the libcamera\n+ framework or libcamera adaptation, and are outside of the scope of the\n+ libcamera project, however example native applications (cam, qcam) are\n+ provided for testing.\n+\n+\n+V4L2 Compatibility Layer\n+ V4L2 compatibility is achieved through a shared library that traps all\n+ accesses to camera devices and routes them to libcamera to emulate high-level\n+ V4L2 camera devices. It is injected in a process address space through\n+ `LD_PRELOAD` and is completely transparent for applications.\n+\n+ The compatibility layer exposes camera device features on a best-effort basis,\n+ and aims for the level of features traditionally available from a UVC camera\n+ designed for video conferencing.\n+\n+Android Camera HAL\n+ Camera support for Android is achieved through a generic Android camera HAL\n+ implementation on top of libcamera. The HAL implements features required by\n+ Android and out of scope from libcamera, such as JPEG encoding support.\n+\n+ This component is used to provide support for ChromeOS platforms\n+\n+GStreamer element (gstlibcamerasrc)\n+ A `GStreamer element`_ is provided to allow capture from libcamera supported\n+ devices through GStreamer pipelines, and connect to other elements for further\n+ processing.\n+\n+ Development of this element is ongoing and is limited to a single stream.\n+\n+Native libcamera API\n+ Applications can make use of the libcamera API directly using the C++\n+ API. An example application and walkthrough using the libcamera API can be\n+ followed in the `Application Writers Guide`_\n+\n+.. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html\n+\n+Architecture\n+------------\n+\n+While offering a unified API towards upper layers, and presenting itself as a\n+single library, libcamera isn’t monolithic. It exposes multiple components\n+through its public API and is built around a set of separate helpers internally.\n+Hardware abstractractions are handled through the use of device-specific\n+components where required and dynamically loadable plugins are used to separate\n+image processing algorithms from the core libcamera codebase.\n+\n+::\n+\n+ --------------------------< libcamera Public API >---------------------------\n+ ^ ^\n+ | |\n+ v v\n+ +-------------+ +---------------------------------------------------+\n+ | Camera | | Camera Device |\n+ | Manager | | +-----------------------------------------------+ |\n+ +-------------+ | | Device-Agnostic | |\n+ ^ | | | |\n+ | | | +--------------------------+ |\n+ | | | | ~~~~~~~~~~~~~~~~~~~~~~~ |\n+ | | | | { +-----------------+ } |\n+ | | | | } | //// Image //// | { |\n+ | | | | <-> | / Processing // | } |\n+ | | | | } | / Algorithms // | { |\n+ | | | | { +-----------------+ } |\n+ | | | | ~~~~~~~~~~~~~~~~~~~~~~~ |\n+ | | | | ========================== |\n+ | | | | +-----------------+ |\n+ | | | | | // Pipeline /// | |\n+ | | | | <-> | /// Handler /// | |\n+ | | | | | /////////////// | |\n+ | | +--------------------+ +-----------------+ |\n+ | | Device-Specific |\n+ | +---------------------------------------------------+\n+ | ^ ^\n+ | | |\n+ v v v\n+ +--------------------------------------------------------------------+\n+ | Helpers and Support Classes |\n+ | +-------------+ +-------------+ +-------------+ +-------------+ |\n+ | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | |\n+ | | Support | | Allocator | | IPC | | Manager | |\n+ | +-------------+ +-------------+ +-------------+ +-------------+ |\n+ | +-------------+ +-------------+ |\n+ | | Pipeline | | ... | |\n+ | | Runner | | | |\n+ | +-------------+ +-------------+ |\n+ +--------------------------------------------------------------------+\n+\n+ /// Device-Specific Components\n+ ~~~ Sandboxing\n+\n+\n+Camera Manager\n+ The Camera Manager enumerates cameras and instantiates Pipeline Handlers to\n+ manage each Camera that libcamera supports. The Camera Manager supports\n+ hotplug detection and notification events when supported by the underlying\n+ kernel devices.\n+\n+ There is only ever one instance of the Camera Manager running per application.\n+ Each application's instance of the Camera Manager ensures that only a single\n+ application can take control of a camera device at once.\n+\n+ Read the `Camera Manager API`_ documentation for more details.\n+\n+.. _Camera Manager API: http://libcamera.org/api-html/classlibcamera_1_1CameraManager.html\n+\n+Camera Device\n+ The Camera class represents a single item of camera hardware that is capable\n+ of producing one or more image streams, and provides the API to interact with\n+ the underlying device.\n+\n+ If a system has multiple instances of the same hardware attached, each has it's\n+ own instance of the camera class.\n+\n+ The API exposes full control of the device to upper layers of libcamera through\n+ the public API, making it the highest level object libcamera exposes, and the\n+ object that all other API operations interact with from configuration to\n+ capture.\n+\n+ Read the `Camera API`_ documentation for more details.\n+\n+.. _Camera API: http://libcamera.org/api-html/classlibcamera_1_1Camera.html\n+\n+Pipeline Handler\n+ The Pipeline Handler manages the complex pipelines exposed by the kernel\n+ drivers through the Media Controller and V4L2 APIs. It abstracts pipeline\n+ handling to hide device-specific details from the rest of the library, and\n+ implements both pipeline configuration based on stream configuration, and\n+ pipeline runtime execution and scheduling when needed by the device.\n+\n+ The Pipeline Handler lives in the same process as the rest of the library, and\n+ has access to all helpers and kernel camera-related devices.\n+\n+ Hardware abstraction is handled by device specific Pipeline Handlers which are\n+ derived from the Pipeline Handler base class allowing commonality to be shared\n+ among the implementations.\n+\n+ Derived pipeline handlers create Camera device instances based on the devices\n+ they detect and support on the running system, and are responsible for\n+ managing the interactions with a camera device.\n+\n+ More details can be found in the `PipelineHandler API`_ documentation, and the\n+ `Pipeline Handler Writers Guide`_.\n+\n+.. _PipelineHandler API: http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html\n+\n+Image Processing Algorithms\n+ An image processing algorithm (IPA) component is a loadable plugin that\n+ implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other\n+ algorithms.\n+\n+ The algorithms run on the CPU and interact with the camera devices through the\n+ Pipeline Handler to control hardware image processing based on the parameters\n+ supplied by upper layers, maintaining state and closing the control loop\n+ of the ISP.\n+\n+ The component is sandboxed and can only interact with libcamera through the\n+ API provided by the Pipeline Handler and an IPA has no direct access to kernel\n+ camera devices.\n+\n+ Open source IPA modules built with libcamera can be run in the same process\n+ space as libcamera, however external IPA modules are run in a separate process\n+ from the main libcamera process. IPA modules have a restricted view of the\n+ system, including no access to networking APIs and limited access to file\n+ systems.\n+\n+ IPA modules are only required for platforms and devices with an ISP controlled\n+ by the host CPU. Camera sensors which have an integrated ISP are not\n+ controlled through the IPA module.\n+\n+Platform Support\n+----------------\n+\n+The library currently supports the following hardware platforms specifically\n+with dedicated pipeline handlers:\n+\n+ - Intel IPU3 (ipu3)\n+ - Rockchip RK3399 (rkisp1)\n+ - RaspberryPi 3 and 4 (raspberrypi)\n+\n+Furthermore, generic platform support is provided for the following:\n+\n+ - USB video device class cameras (uvcvideo)\n+ - iMX7, Allwinner Sun6i (simple)\n+ - Virtual media controller driver for test use cases (vimc)\n+\n+Licensing\n+---------\n+\n+The libcamera core, is covered by the `LGPL-2.1-or-later`_ license. Pipeline\n+Handlers are a part of the libcamera code base and need to be contributed\n+upstream by device vendors. IPA modules included in libcamera are covered by a\n+free software license, however third-parties may develop IPA modules outside of\n+libcamera and distribute them under a closed-source license, provided they do\n+not include source code from the libcamera project.\n+\n+The libcamera project itself contains multiple libraries, applications and\n+utilities. Licenses are expressed through SPDX tags in text-based files that\n+support comments, and through the .reuse/dep5 file otherwise. A copy of all\n+licenses are stored in the LICENSES directory, and a full summary of the\n+licensing used throughout the project can be found in the COPYING.rst document.\n+\n+Applications which link dynamically against libcamera and use only the public\n+API are an independent work of the authors and have no license restrictions\n+imposed upon them from libcamera.\n+\n+.. _LGPL-2.1-or-later: https://spdx.org/licenses/LGPL-2.1-or-later.html\n\\ No newline at end of file\ndiff --git a/Documentation/index.rst b/Documentation/index.rst\nindex 4e746bb17c4a..cfcfd388699b 100644\n--- a/Documentation/index.rst\n+++ b/Documentation/index.rst\n@@ -12,3 +12,5 @@\n Home <self>\n Docs <docs>\n Contribute <contributing>\n+\n+ Developer Guide <guides/introduction>\ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex 6d9a397cf1a3..dd7ae700af11 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -52,6 +52,7 @@ if sphinx.found()\n 'contributing.rst',\n 'docs.rst',\n 'index.rst',\n+ 'guides/introduction.rst',\n ]\n \n release = 'release=v' + libcamera_git_version\n", "prefixes": [ "libcamera-devel", "v4", "1/3" ] }