Patch Detail
Show a patch.
GET /api/1.1/patches/45/?format=api
{ "id": 45, "url": "https://patchwork.libcamera.org/api/1.1/patches/45/?format=api", "web_url": "https://patchwork.libcamera.org/patch/45/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20181213142327.4455-3-laurent.pinchart@ideasonboard.com>", "date": "2018-12-13T14:23:27", "name": "[libcamera-devel,3/3] Documentation: Add architecture documentation", "commit_ref": "58a31afc0c25e8f634b24c61e6d89ac2d3267c7e", "pull_url": null, "state": "accepted", "archived": false, "hash": "696ef741bf5fcdf79486ef39654d24155f2f80cb", "submitter": { "id": 2, "url": "https://patchwork.libcamera.org/api/1.1/people/2/?format=api", "name": "Laurent Pinchart", "email": "laurent.pinchart@ideasonboard.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/45/mbox/", "series": [ { "id": 21, "url": "https://patchwork.libcamera.org/api/1.1/series/21/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=21", "date": "2018-12-13T14:23:25", "name": "[libcamera-devel,1/3] Documentation: Remove _static and _templates directory", "version": 1, "mbox": "https://patchwork.libcamera.org/series/21/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/45/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/45/checks/", "tags": {}, "headers": { "Return-Path": "<laurent.pinchart@ideasonboard.com>", "Received": [ "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B07360B1B\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 13 Dec 2018 15:22:44 +0100 (CET)", "from avalon.bb.dnainternet.fi\n\t(dfj612ybrt5fhg77mgycy-3.rev.dnainternet.fi\n\t[IPv6:2001:14ba:21f5:5b00:2e86:4862:ef6a:2804])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9385658E\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 13 Dec 2018 15:22:43 +0100 (CET)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1544710963;\n\tbh=QxvAKM6MylMaz5aOwZTmfazckLxq10hlPDbKQE65WyE=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=Fwl6EJgyjhgQFLMFiHxw9C5VKNk1PpmPfBOeym0tHvxFPmi+iyZ2dq55qctd+QAei\n\t9LjW2rVFWHD4PpbvRuVqDQqwvPX53KVhVxJbusvVTnR1o86pImQ2OimC5uWXX36VGw\n\tUzF09auRkN89mKEz0xUisoymTyodc4oqjVQ5S6GY=", "From": "Laurent Pinchart <laurent.pinchart@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Date": "Thu, 13 Dec 2018 16:23:27 +0200", "Message-Id": "<20181213142327.4455-3-laurent.pinchart@ideasonboard.com>", "X-Mailer": "git-send-email 2.19.2", "In-Reply-To": "<20181213142327.4455-1-laurent.pinchart@ideasonboard.com>", "References": "<20181213142327.4455-1-laurent.pinchart@ideasonboard.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "Subject": "[libcamera-devel] [PATCH 3/3] Documentation: Add architecture\n\tdocumentation", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.23", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "X-List-Received-Date": "Thu, 13 Dec 2018 14:22:44 -0000" }, "content": "The documentation is copied mostly verbatim from the website, with small\nmodifications to the ascii art diagrams to make them compile.\n\nSigned-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n---\n Documentation/docs.rst | 383 ++++++++++++++++++++++++++++++++++++++\n Documentation/index.rst | 1 +\n Documentation/meson.build | 1 +\n 3 files changed, 385 insertions(+)\n create mode 100644 Documentation/docs.rst", "diff": "diff --git a/Documentation/docs.rst b/Documentation/docs.rst\nnew file mode 100644\nindex 000000000000..4bbcd9aa35bf\n--- /dev/null\n+++ b/Documentation/docs.rst\n@@ -0,0 +1,383 @@\n+*************\n+Documentation\n+*************\n+\n+Feature Requirements\n+====================\n+\n+Device enumeration\n+------------------\n+\n+The library shall support enumerating all camera devices available in the\n+system, including both fixed cameras and hotpluggable cameras. It shall\n+support cameras plugged and unplugged after the initialization of the\n+library, and shall offer a mechanism to notify applications of camera plug\n+and unplug.\n+\n+The following types of cameras shall be supported:\n+\n+* Internal cameras designed for point-and-shoot still image and video\n+ capture usage, either controlled directly by the CPU, or exposed through\n+ an internal USB bus as a UVC device.\n+\n+* External UVC cameras designed for video conferencing usage.\n+\n+Other types of camera, including analog cameras, depth cameras, thermal\n+cameras, external digital picture or movie cameras, are out of scope for\n+this project.\n+\n+A hardware device that includes independent camera sensors, such as front\n+and back sensors in a phone, shall be considered as multiple camera devices\n+for the purpose of this library.\n+\n+Independent Camera Devices\n+--------------------------\n+\n+When multiple cameras are present in the system and are able to operate\n+independently from each other, the library shall expose them as multiple\n+camera devices and support parallel operation without any additional usage\n+restriction apart from the limitations inherent to the hardware (such as\n+memory bandwidth, CPU usage or number of CSI-2 receivers for instance).\n+\n+Independent processes shall be able to use independent cameras devices\n+without interfering with each other. A single camera device shall be\n+usable by a single process at a time.\n+\n+Multiple streams support\n+------------------------\n+\n+The library shall support multiple video streams running in parallel\n+for each camera device, within the limits imposed by the system.\n+\n+Per frame controls\n+------------------\n+\n+The library shall support controlling capture parameters for each stream\n+on a per-frame basis, on a best effort basis based on the capabilities of the\n+hardware and underlying software stack (including kernel drivers and\n+firmware). It shall apply capture parameters to the frame they target, and\n+report the value of the parameters that have effectively been used for each\n+captured frame.\n+\n+When a camera device supports multiple streams, the library shall allow both\n+control of each stream independently, and control of multiple streams\n+together. Streams that are controlled together shall be synchronized. No\n+synchronization is required for streams controlled independently.\n+\n+Capability Enumeration\n+----------------------\n+\n+The library shall expose capabilities of each camera device in a way that\n+allows applications to discover those capabilities dynamically. Applications\n+shall be allowed to cache capabilities for as long as they are using the\n+library. If capabilities can change at runtime, the library shall offer a\n+mechanism to notify applications of such changes. Applications shall not\n+cache capabilities in long term storage between runs.\n+\n+Capabilities shall be discovered dynamically at runtime from the device when\n+possible, and may come, in part or in full, from platform configuration\n+data.\n+\n+Device Profiles\n+---------------\n+\n+The library may define different camera device profiles, each with a minimum\n+set of required capabilities. Applications may use those profiles to quickly\n+determine the level of features exposed by a device without parsing the full\n+list of capabilities. Camera devices may implement additional capabilities\n+on top of the minimum required set for the profile they expose.\n+\n+3A and Image Enhancement Algorithms\n+-----------------------------------\n+\n+The camera devices shall implement auto exposure, auto gain and auto white\n+balance. Camera devices that include a focus lens shall implement auto\n+focus. Additional image enhancement algorithms, such as noise reduction or\n+video stabilization, may be implemented.\n+\n+All algorithms may be implemented in hardware or firmware outside of the\n+library, or in software in the library. They shall all be controllable by\n+applications.\n+\n+The library shall be architectured to isolate the 3A and image enhancement\n+algorithms in a component with a documented API, respectively called the 3A\n+component and the 3A API. The 3A API shall be stable, and shall allow both\n+open-source and closed-source implementations of the 3A component.\n+\n+The library may include statically-linked open-source 3A components, and\n+shall support dynamically-linked open-source and closed-source 3A\n+components.\n+\n+Closed-source 3A Component Sandboxing\n+-------------------------------------\n+\n+For security purposes, it may be desired to run closed-source 3A components\n+in a separate process. The 3A API would in such a case be transported over\n+IPC. The 3A API shall make it possible to use any IPC mechanism that\n+supports passing file descriptors.\n+\n+The library may implement an IPC mechanism, and shall support third-party\n+platform-specific IPC mechanisms through the implementation of a\n+platform-specific 3A API wrapper. No modification to the library shall be\n+needed to use such third-party IPC mechanisms.\n+\n+The 3A component shall not directly access any device node on the system.\n+Such accesses shall instead be performed through the 3A API. The library\n+shall validate all accesses and restrict them to what is absolutely required\n+by 3A components.\n+\n+V4L2 Compatibility Layer\n+------------------------\n+\n+The project shall support traditional V4L2 application through an additional\n+libcamera wrapper library. The wrapper library shall trap all accesses to\n+camera devices through LD_PRELOAD, and route them through libcamera to\n+emulate a high-level V4L2 camera device. It shall expose camera device\n+features on a best-effort basis, and aim for the level of features\n+traditionally available from a UVC camera designed for video conferencing.\n+\n+Android Camera HAL v3 Compatibility\n+-----------------------------------\n+\n+The library API shall expose all the features required to implement an\n+Android Camera HAL v3 on top of libcamera. Some features of the HAL may be\n+omitted as long as they can be implemented separately in the HAL, such as\n+JPEG encoding, or YUV reprocessing.\n+\n+\n+Camera Stack\n+============\n+\n+::\n+\n+ | a c / +-------------+ +-------------+ +-------------+ +-------------+\n+ | p a | | Native | | Framework | | Native | | Android |\n+ | p t | | V4L2 | | Application | | libcamera | | Camera |\n+ | l i | | Application | | (gstreamer) | | Application | | Framework |\n+ | i o \\ +-------------+ +-------------+ +-------------+ +-------------+\n+ | n ^ ^ ^ ^\n+ | | | | |\n+ | l a | | | |\n+ | i d v v | v\n+ | b a / +-------------+ +-------------+ | +-------------+\n+ | c p | | V4L2 | | Camera | | | Android |\n+ | a t | | Compat. | | Framework | | | Camera |\n+ | m a | | | | (gstreamer) | | | HAL |\n+ | e t \\ +-------------+ +-------------+ | +-------------+\n+ | r i ^ ^ | ^\n+ | a o | | | |\n+ | n | | | |\n+ | / | ,................................................\n+ | | | ! : Language : !\n+ | l f | | ! : Bindings : !\n+ | i r | | ! : (optional) : !\n+ | b a | | \\...............................................'\n+ | c m | | | | |\n+ | a e | | | | |\n+ | m w | v v v v\n+ | e o | +----------------------------------------------------------------+\n+ | r r | | |\n+ | a k | | libcamera |\n+ | | | |\n+ | \\ +----------------------------------------------------------------+\n+ | ^ ^ ^\n+ | Userspace | | |\n+ | ------------------------ | ---------------- | ---------------- | ---------------\n+ | Kernel | | |\n+ | v v v\n+ | +-----------+ +-----------+ +-----------+\n+ | | Media | <--> | Video | <--> | V4L2 |\n+ | | Device | | Device | | Subdev |\n+ | +-----------+ +-----------+ +-----------+\n+\n+The camera stack comprises four software layers. From bottom to top:\n+\n+* The kernel drivers control the camera hardware and expose a\n+ low-level interface to userspace through the Linux kernel V4L2\n+ family of APIs (Media Controller API, V4L2 Video Device API and\n+ V4L2 Subdev API).\n+\n+* The libcamera framework is the core part of the stack. It\n+ handles all control of the camera devices in its core component,\n+ libcamera, and exposes a native C++ API to upper layers. Optional\n+ language bindings allow interfacing to libcamera from other\n+ programming languages.\n+\n+ Those components live in the same source code repository and\n+ all together constitute the libcamera framework.\n+\n+* The libcamera adaptation is an umbrella term designating the\n+ components that interface to libcamera in other frameworks.\n+ Notable examples are a V4L2 compatibility layer, a gstreamer\n+ libcamera element, and an Android camera HAL implementation based\n+ on libcamera.\n+\n+ Those components can live in the libcamera project source code\n+ in separate repositories, or move to their respective project's\n+ repository (for instance the gstreamer libcamera element).</p>\n+\n+* The applications and upper level frameworks are based on the\n+ libcamera framework or libcamera adaptation, and are outside of\n+ the scope of the libcamera project.\n+\n+\n+libcamera Architecture\n+======================\n+\n+::\n+\n+ | ---------------------------< libcamera Public API >---------------------------\n+ | ^ ^\n+ | | |\n+ | v v\n+ | +-------------+ +-------------------------------------------------+\n+ | | Camera | | Camera Device |\n+ | | Devices | | +---------------------------------------------+ |\n+ | | Manager | | | Device-Agnostic | |\n+ | +-------------+ | | | |\n+ | ^ | | +------------------------+ |\n+ | | | | | ~~~~~~~~~~~~~~~~~~~~~ |\n+ | | | | | { +---------------+ } |\n+ | | | | | } | ////Image//// | { |\n+ | | | | | <-> | /Processing// | } |\n+ | | | | | } | /Algorithms// | { |\n+ | | | | | { +---------------+ } |\n+ | | | | | ~~~~~~~~~~~~~~~~~~~~~ |\n+ | | | | | ======================== |\n+ | | | | | +---------------+ |\n+ | | | | | | //Pipeline/// | |\n+ | | | | | <-> | ///Handler/// | |\n+ | | | | | | ///////////// | |\n+ | | | +--------------------+ +---------------+ |\n+ | | | Device-Specific |\n+ | | +-------------------------------------------------+\n+ | | ^ ^\n+ | | | |\n+ | v v v\n+ | +--------------------------------------------------------------------+\n+ | | Helpers and Support Classes |\n+ | | +-------------+ +-------------+ +-------------+ +-------------+ |\n+ | | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | |\n+ | | | Support | | Allocator | | IPC | | Manager | |\n+ | | +-------------+ +-------------+ +-------------+ +-------------+ |\n+ | | +-------------+ +-------------+ |\n+ | | | Pipeline | | ... | |\n+ | | | Runner | | | |\n+ | | +-------------+ +-------------+ |\n+ | +--------------------------------------------------------------------+\n+ |\n+ | /// Device-Specific Components\n+ | ~~~ Sandboxing\n+\n+While offering a unified API towards upper layers, and presenting\n+itself as a single library, libcamera isn't monolithic. It exposes\n+multiple components through its public API, is built around a set of\n+separate helpers internally, uses device-specific components and can\n+load dynamic plugins.\n+\n+Camera Devices Manager\n+ The Camera Devices Manager provides a view of available cameras\n+ in the system. It performs cold enumeration and runtime camera\n+ management, and supports a hotplug notification mechanism in its\n+ public API.\n+\n+ To avoid the cost associated with cold enumeration of all devices\n+ at application start, and to arbitrate concurrent access to camera\n+ devices, the Camera Devices Manager could later be split to a\n+ separate service, possibly with integration in platform-specific\n+ device management.\n+\n+Camera Device\n+ The Camera Device represents a camera device to upper layers. It\n+ exposes full control of the device through the public API, and is\n+ thus the highest level object exposed by libcamera.\n+\n+ Camera Device instances are created by the Camera Devices\n+ Manager. An optional method to create new instances could be exposed\n+ through the public API to speed up initialization when the upper\n+ layer knows how to directly address camera devices present in the\n+ system.\n+\n+Pipeline Handler\n+ The Pipeline Handler manages complex pipelines exposed by the kernel drivers\n+ through the Media Controller and V4L2 APIs. It abstracts pipeline handling to\n+ hide device-specific details to the rest of the library, and implements both\n+ pipeline configuration based on stream configuration, and pipeline runtime\n+ execution and scheduling when needed by the device.\n+\n+ This component is device-specific and is part of the libcamera code base. As\n+ such it is covered by the same free software license as the rest of libcamera\n+ and needs to be contributed upstream by device vendors. The Pipeline Handler\n+ lives in the same process as the rest of the library, and has access to all\n+ helpers and kernel camera-related devices.</p>\n+\n+Image Processing Algorithms\n+ Together with the hardware image processing and hardware statistics\n+ collection, the Image Processing Algorithms implement 3A (Auto-Exposure,\n+ Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU\n+ and interact with the kernel camera devices to control hardware image\n+ processing based on the parameters supplied by upper layers, closing the\n+ control loop of the ISP.\n+\n+ This component is device-specific and is loaded as an external plugin. It can\n+ be part of the libcamera code base, in which case it is covered by the same\n+ license, or provided externally as an open-source or closed-source component.\n+\n+ The component is sandboxed and can only interact with libcamera through\n+ internal APIs specifically marked as such. In particular it will have no\n+ direct access to kernel camera devices, and all its accesses to image and\n+ metadata will be mediated by dmabuf instances explicitly passed to the\n+ component. The component must be prepared to run in a process separate from\n+ the main libcamera process, and to have a very restricted view of the system,\n+ including no access to networking APIs and limited access to file systems.\n+\n+ The sandboxing mechanism isn't defined by libcamera. One example\n+ implementation will be provided as part of the project, and platforms vendors\n+ will be able to provide their own sandboxing mechanism as a plugin.\n+\n+ libcamera should provide a basic implementation of Image Processing\n+ Algorithms, to serve as a reference for the internal API. Device vendors are\n+ expected to provide a full-fledged implementation compatible with their\n+ Pipeline Handler. One goal of the libcamera project is to create an\n+ environment in which the community will be able to compete with the\n+ closed-source vendor binaries and develop a high quality open source\n+ implementation.\n+\n+Helpers and Support Classes\n+ While Pipeline Handlers are device-specific, implementations are expected to\n+ share code due to usage of identical APIs towards the kernel camera drivers\n+ and the Image Processing Algorithms. This includes without limitation handling\n+ of the MC and V4L2 APIs, buffer management through dmabuf, and pipeline\n+ discovery, configuration and scheduling. Such code will be factored out to\n+ helpers when applicable.\n+\n+ Other parts of libcamera will also benefit from factoring code out to\n+ self-contained support classes, even if such code is present only once in the\n+ code base, in order to keep the source code clean and easy to read. This\n+ should be the case for instance for plugin management.\n+\n+\n+V4L2 Compatibility Layer\n+------------------------\n+\n+V4L2 compatibility is achieved through a shared library that traps all\n+accesses to camera devices and routes them to libcamera to emulate high-level\n+V4L2 camera devices. It is injected in a process address space through\n+LD_PRELOAD and is completely transparent for applications.\n+\n+The compatibility layer exposes camera device features on a best-effort basis,\n+and aims for the level of features traditionally available from a UVC camera\n+designed for video conferencing.\n+\n+\n+Android Camera HAL\n+------------------\n+\n+Camera support for Android is achieved through a generic Android\n+camera HAL implementation on top of libcamera. The HAL will implement internally\n+features required by Android and missing from libcamera, such as JPEG encoding\n+support.\n+\n+The Android camera HAL implementation will initially target the\n+LIMITED harware level, with support for the FULL level then being gradually\n+implemented.\ndiff --git a/Documentation/index.rst b/Documentation/index.rst\nindex ba6c6c6ea56d..44176f230c0f 100644\n--- a/Documentation/index.rst\n+++ b/Documentation/index.rst\n@@ -21,6 +21,7 @@ systems, including traditional Linux distributions, ChromeOS and Android.\n :caption: Contents:\n \n coding-style\n+ docs\n contributing\n \n \ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex 8264b5e6cdb1..dd43b2cbd401 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -44,6 +44,7 @@ if sphinx.found()\n 'coding-style.rst',\n 'conf.py',\n 'contributing.rst',\n+ 'docs.rst',\n 'index.rst',\n ]\n \n", "prefixes": [ "libcamera-devel", "3/3" ] }