Patch Detail
Show a patch.
GET /api/1.1/patches/20935/?format=api
{ "id": 20935, "url": "https://patchwork.libcamera.org/api/1.1/patches/20935/?format=api", "web_url": "https://patchwork.libcamera.org/patch/20935/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20240815082944.170109-4-dan.scally@ideasonboard.com>", "date": "2024-08-15T08:29:40", "name": "[v2,3/7] Documentation: Synchronise libcamera architecture details", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "b508e363c0a821dc230385a7051a430b329e1c2b", "submitter": { "id": 156, "url": "https://patchwork.libcamera.org/api/1.1/people/156/?format=api", "name": "Dan Scally", "email": "dan.scally@ideasonboard.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/20935/mbox/", "series": [ { "id": 4521, "url": "https://patchwork.libcamera.org/api/1.1/series/4521/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=4521", "date": "2024-08-15T08:29:37", "name": "Documentation improvements", "version": 2, "mbox": "https://patchwork.libcamera.org/series/4521/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/20935/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/20935/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7464DC323E\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu, 15 Aug 2024 08:30:13 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 033EC633C9;\n\tThu, 15 Aug 2024 10:30:11 +0200 (CEST)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 47011633BD\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 15 Aug 2024 10:30:04 +0200 (CEST)", "from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C9E21A34;\n\tThu, 15 Aug 2024 10:29:05 +0200 (CEST)" ], "Authentication-Results": "lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Iob4ucFE\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723710546;\n\tbh=LTdk8V4e0PVI+UwrcfcO98Tga//YtNXFv2kwJfo1iXk=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=Iob4ucFEwHd6d2munLprU8BQe6VPdPGF7AcLrszxCZSc8N8ffgNwSWFaVzmeEXDYf\n\twekjHZklJhSMGBs2H9WDTJRAOcs4HdlBEPoeEJD2wZm9kxGJxAxazAj/hRSfv/reoM\n\t7HMhfzD/HB69rKc8wY+PsVRFfCnvasSfg3dKk6/M=", "From": "Daniel Scally <dan.scally@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Cc": "Daniel Scally <dan.scally@ideasonboard.com>,\n\tLaurent Pinchart <laurent.pinchart@ideasonboard.com>", "Subject": "[PATCH v2 3/7] Documentation: Synchronise libcamera architecture\n\tdetails", "Date": "Thu, 15 Aug 2024 09:29:40 +0100", "Message-Id": "<20240815082944.170109-4-dan.scally@ideasonboard.com>", "X-Mailer": "git-send-email 2.34.1", "In-Reply-To": "<20240815082944.170109-1-dan.scally@ideasonboard.com>", "References": "<20240815082944.170109-1-dan.scally@ideasonboard.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "There are two near-duplicate instances of the libcamera architecture\ndetail in the Documentation, in docs.rst and guides/introduction.rst.\nThe latter is more up-to-date, so remove it from the introduction\nfile (which will soon be deprecated) and update the section in docs.\n\nReviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\nSigned-off-by: Daniel Scally <dan.scally@ideasonboard.com>\n---\nChanges since v1:\n\n\t- None\n\n Documentation/docs.rst | 221 ++++++++++++++------------\n Documentation/guides/introduction.rst | 137 +---------------\n 2 files changed, 121 insertions(+), 237 deletions(-)", "diff": "diff --git a/Documentation/docs.rst b/Documentation/docs.rst\nindex 5871961c..10f07a9e 100644\n--- a/Documentation/docs.rst\n+++ b/Documentation/docs.rst\n@@ -243,116 +243,135 @@ The camera stack comprises four software layers. From bottom to top:\n libcamera Architecture\n ======================\n \n+While offering a unified API towards upper layers, and presenting itself as a\n+single library, libcamera isn't monolithic. It exposes multiple components\n+through its public API and is built around a set of separate helpers internally.\n+Hardware abstractions are handled through the use of device-specific components\n+where required and dynamically loadable plugins are used to separate image\n+processing algorithms from the core libcamera codebase.\n+\n ::\n \n- ---------------------------< libcamera Public API >---------------------------\n- ^ ^\n- | |\n- v v\n- +-------------+ +-------------------------------------------------+\n- | Camera | | Camera Device |\n- | Devices | | +---------------------------------------------+ |\n- | Manager | | | Device-Agnostic | |\n- +-------------+ | | | |\n- ^ | | +------------------------+ |\n- | | | | ~~~~~~~~~~~~~~~~~~~~~ |\n- | | | | { +---------------+ } |\n- | | | | } | ////Image//// | { |\n- | | | | <-> | /Processing// | } |\n- | | | | } | /Algorithms// | { |\n- | | | | { +---------------+ } |\n- | | | | ~~~~~~~~~~~~~~~~~~~~~ |\n- | | | | ======================== |\n- | | | | +---------------+ |\n- | | | | | //Pipeline/// | |\n- | | | | <-> | ///Handler/// | |\n- | | | | | ///////////// | |\n- | | +--------------------+ +---------------+ |\n- | | Device-Specific |\n- | +-------------------------------------------------+\n- | ^ ^\n- | | |\n- v v v\n- +--------------------------------------------------------------------+\n- | Helpers and Support Classes |\n- | +-------------+ +-------------+ +-------------+ +-------------+ |\n- | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | |\n- | | Support | | Allocator | | IPC | | Manager | |\n- | +-------------+ +-------------+ +-------------+ +-------------+ |\n- | +-------------+ +-------------+ |\n- | | Pipeline | | ... | |\n- | | Runner | | | |\n- | +-------------+ +-------------+ |\n- +--------------------------------------------------------------------+\n-\n- /// Device-Specific Components\n- ~~~ Sandboxing\n-\n-While offering a unified API towards upper layers, and presenting\n-itself as a single library, libcamera isn't monolithic. It exposes\n-multiple components through its public API, is built around a set of\n-separate helpers internally, uses device-specific components and can\n-load dynamic plugins.\n-\n-Camera Devices Manager\n- The Camera Devices Manager provides a view of available cameras\n- in the system. It performs cold enumeration and runtime camera\n- management, and supports a hotplug notification mechanism in its\n- public API.\n-\n- To avoid the cost associated with cold enumeration of all devices\n- at application start, and to arbitrate concurrent access to camera\n- devices, the Camera Devices Manager could later be split to a\n- separate service, possibly with integration in platform-specific\n- device management.\n+ --------------------------< libcamera Public API >---------------------------\n+ ^ ^\n+ | |\n+ v v\n+ +-------------+ +---------------------------------------------------+\n+ | Camera | | Camera Device |\n+ | Manager | | +-----------------------------------------------+ |\n+ +-------------+ | | Device-Agnostic | |\n+ ^ | | | |\n+ | | | +--------------------------+ |\n+ | | | | ~~~~~~~~~~~~~~~~~~~~~~~ |\n+ | | | | { +-----------------+ } |\n+ | | | | } | //// Image //// | { |\n+ | | | | <-> | / Processing // | } |\n+ | | | | } | / Algorithms // | { |\n+ | | | | { +-----------------+ } |\n+ | | | | ~~~~~~~~~~~~~~~~~~~~~~~ |\n+ | | | | ========================== |\n+ | | | | +-----------------+ |\n+ | | | | | // Pipeline /// | |\n+ | | | | <-> | /// Handler /// | |\n+ | | | | | /////////////// | |\n+ | | +--------------------+ +-----------------+ |\n+ | | Device-Specific |\n+ | +---------------------------------------------------+\n+ | ^ ^\n+ | | |\n+ v v v\n+ +--------------------------------------------------------------------+\n+ | Helpers and Support Classes |\n+ | +-------------+ +-------------+ +-------------+ +-------------+ |\n+ | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | |\n+ | | Support | | Allocator | | IPC | | Manager | |\n+ | +-------------+ +-------------+ +-------------+ +-------------+ |\n+ | +-------------+ +-------------+ |\n+ | | Pipeline | | ... | |\n+ | | Runner | | | |\n+ | +-------------+ +-------------+ |\n+ +--------------------------------------------------------------------+\n+\n+ /// Device-Specific Components\n+ ~~~ Sandboxing\n+\n+Camera Manager\n+ The Camera Manager enumerates cameras and instantiates Pipeline Handlers to\n+ manage each Camera that libcamera supports. The Camera Manager supports\n+ hotplug detection and notification events when supported by the underlying\n+ kernel devices.\n+\n+ There is only ever one instance of the Camera Manager running per application.\n+ Each application's instance of the Camera Manager ensures that only a single\n+ application can take control of a camera device at once.\n+\n+ Read the `Camera Manager API`_ documentation for more details.\n+\n+.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html\n \n Camera Device\n- The Camera Device represents a camera device to upper layers. It\n- exposes full control of the device through the public API, and is\n- thus the highest level object exposed by libcamera.\n+ The Camera class represents a single item of camera hardware that is capable\n+ of producing one or more image streams, and provides the API to interact with\n+ the underlying device.\n+\n+ If a system has multiple instances of the same hardware attached, each has its\n+ own instance of the camera class.\n+\n+ The API exposes full control of the device to upper layers of libcamera through\n+ the public API, making it the highest level object libcamera exposes, and the\n+ object that all other API operations interact with from configuration to\n+ capture.\n \n- Camera Device instances are created by the Camera Devices\n- Manager. An optional function to create new instances could be exposed\n- through the public API to speed up initialization when the upper\n- layer knows how to directly address camera devices present in the\n- system.\n+ Read the `Camera API`_ documentation for more details.\n+\n+.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html\n \n Pipeline Handler\n- The Pipeline Handler manages complex pipelines exposed by the kernel drivers\n- through the Media Controller and V4L2 APIs. It abstracts pipeline handling to\n- hide device-specific details to the rest of the library, and implements both\n- pipeline configuration based on stream configuration, and pipeline runtime\n- execution and scheduling when needed by the device.\n-\n- This component is device-specific and is part of the libcamera code base. As\n- such it is covered by the same free software license as the rest of libcamera\n- and needs to be contributed upstream by device vendors. The Pipeline Handler\n- lives in the same process as the rest of the library, and has access to all\n- helpers and kernel camera-related devices.\n+ The Pipeline Handler manages the complex pipelines exposed by the kernel\n+ drivers through the Media Controller and V4L2 APIs. It abstracts pipeline\n+ handling to hide device-specific details from the rest of the library, and\n+ implements both pipeline configuration based on stream configuration, and\n+ pipeline runtime execution and scheduling when needed by the device.\n+\n+ The Pipeline Handler lives in the same process as the rest of the library, and\n+ has access to all helpers and kernel camera-related devices.\n+\n+ Hardware abstraction is handled by device specific Pipeline Handlers which are\n+ derived from the Pipeline Handler base class allowing commonality to be shared\n+ among the implementations.\n+\n+ Derived pipeline handlers create Camera device instances based on the devices\n+ they detect and support on the running system, and are responsible for\n+ managing the interactions with a camera device.\n+\n+ More details can be found in the `PipelineHandler API`_ documentation, and the\n+ :doc:`Pipeline Handler Writers Guide <guides/pipeline-handler>`.\n+\n+.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html\n \n Image Processing Algorithms\n- Together with the hardware image processing and hardware statistics\n- collection, the Image Processing Algorithms implement 3A (Auto-Exposure,\n- Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU\n- and interact with the kernel camera devices to control hardware image\n- processing based on the parameters supplied by upper layers, closing the\n- control loop of the ISP.\n-\n- This component is device-specific and is loaded as an external plugin. It can\n- be part of the libcamera code base, in which case it is covered by the same\n- license, or provided externally as an open-source or closed-source component.\n-\n- The component is sandboxed and can only interact with libcamera through\n- internal APIs specifically marked as such. In particular it will have no\n- direct access to kernel camera devices, and all its accesses to image and\n- metadata will be mediated by dmabuf instances explicitly passed to the\n- component. The component must be prepared to run in a process separate from\n- the main libcamera process, and to have a very restricted view of the system,\n- including no access to networking APIs and limited access to file systems.\n-\n- The sandboxing mechanism isn't defined by libcamera. One example\n- implementation will be provided as part of the project, and platforms vendors\n- will be able to provide their own sandboxing mechanism as a plugin.\n+ An image processing algorithm (IPA) component is a loadable plugin that\n+ implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other\n+ algorithms.\n+\n+ The algorithms run on the CPU and interact with the camera devices through the\n+ Pipeline Handler to control hardware image processing based on the parameters\n+ supplied by upper layers, maintaining state and closing the control loop\n+ of the ISP.\n+\n+ The component is sandboxed and can only interact with libcamera through the\n+ API provided by the Pipeline Handler and an IPA has no direct access to kernel\n+ camera devices.\n+\n+ Open source IPA modules built with libcamera can be run in the same process\n+ space as libcamera, however external IPA modules are run in a separate process\n+ from the main libcamera process. IPA modules have a restricted view of the\n+ system, including no access to networking APIs and limited access to file\n+ systems.\n+\n+ IPA modules are only required for platforms and devices with an ISP controlled\n+ by the host CPU. Camera sensors which have an integrated ISP are not\n+ controlled through the IPA module.\n \n libcamera should provide a basic implementation of Image Processing\n Algorithms, to serve as a reference for the internal API. Device vendors are\ndiff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst\nindex 8368bd4a..1898d5fe 100644\n--- a/Documentation/guides/introduction.rst\n+++ b/Documentation/guides/introduction.rst\n@@ -27,8 +27,7 @@ desirable results from the camera.\n \n \n In this developers guide, we will explore the `Camera Stack`_ and how it is\n-can be visualised at a high level, and explore the internal `Architecture`_ of\n-the libcamera library with its components. The current `Platform Support`_ is\n+can be visualised at a high level. The current `Platform Support`_ is\n detailed, as well as an overview of the `Licensing`_ requirements of the\n project.\n \n@@ -148,140 +147,6 @@ Native libcamera API\n \n .. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html\n \n-Architecture\n-------------\n-\n-While offering a unified API towards upper layers, and presenting itself as a\n-single library, libcamera isn't monolithic. It exposes multiple components\n-through its public API and is built around a set of separate helpers internally.\n-Hardware abstractions are handled through the use of device-specific components\n-where required and dynamically loadable plugins are used to separate image\n-processing algorithms from the core libcamera codebase.\n-\n-::\n-\n- --------------------------< libcamera Public API >---------------------------\n- ^ ^\n- | |\n- v v\n- +-------------+ +---------------------------------------------------+\n- | Camera | | Camera Device |\n- | Manager | | +-----------------------------------------------+ |\n- +-------------+ | | Device-Agnostic | |\n- ^ | | | |\n- | | | +--------------------------+ |\n- | | | | ~~~~~~~~~~~~~~~~~~~~~~~ |\n- | | | | { +-----------------+ } |\n- | | | | } | //// Image //// | { |\n- | | | | <-> | / Processing // | } |\n- | | | | } | / Algorithms // | { |\n- | | | | { +-----------------+ } |\n- | | | | ~~~~~~~~~~~~~~~~~~~~~~~ |\n- | | | | ========================== |\n- | | | | +-----------------+ |\n- | | | | | // Pipeline /// | |\n- | | | | <-> | /// Handler /// | |\n- | | | | | /////////////// | |\n- | | +--------------------+ +-----------------+ |\n- | | Device-Specific |\n- | +---------------------------------------------------+\n- | ^ ^\n- | | |\n- v v v\n- +--------------------------------------------------------------------+\n- | Helpers and Support Classes |\n- | +-------------+ +-------------+ +-------------+ +-------------+ |\n- | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | |\n- | | Support | | Allocator | | IPC | | Manager | |\n- | +-------------+ +-------------+ +-------------+ +-------------+ |\n- | +-------------+ +-------------+ |\n- | | Pipeline | | ... | |\n- | | Runner | | | |\n- | +-------------+ +-------------+ |\n- +--------------------------------------------------------------------+\n-\n- /// Device-Specific Components\n- ~~~ Sandboxing\n-\n-\n-Camera Manager\n- The Camera Manager enumerates cameras and instantiates Pipeline Handlers to\n- manage each Camera that libcamera supports. The Camera Manager supports\n- hotplug detection and notification events when supported by the underlying\n- kernel devices.\n-\n- There is only ever one instance of the Camera Manager running per application.\n- Each application's instance of the Camera Manager ensures that only a single\n- application can take control of a camera device at once.\n-\n- Read the `Camera Manager API`_ documentation for more details.\n-\n-.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html\n-\n-Camera Device\n- The Camera class represents a single item of camera hardware that is capable\n- of producing one or more image streams, and provides the API to interact with\n- the underlying device.\n-\n- If a system has multiple instances of the same hardware attached, each has its\n- own instance of the camera class.\n-\n- The API exposes full control of the device to upper layers of libcamera through\n- the public API, making it the highest level object libcamera exposes, and the\n- object that all other API operations interact with from configuration to\n- capture.\n-\n- Read the `Camera API`_ documentation for more details.\n-\n-.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html\n-\n-Pipeline Handler\n- The Pipeline Handler manages the complex pipelines exposed by the kernel\n- drivers through the Media Controller and V4L2 APIs. It abstracts pipeline\n- handling to hide device-specific details from the rest of the library, and\n- implements both pipeline configuration based on stream configuration, and\n- pipeline runtime execution and scheduling when needed by the device.\n-\n- The Pipeline Handler lives in the same process as the rest of the library, and\n- has access to all helpers and kernel camera-related devices.\n-\n- Hardware abstraction is handled by device specific Pipeline Handlers which are\n- derived from the Pipeline Handler base class allowing commonality to be shared\n- among the implementations.\n-\n- Derived pipeline handlers create Camera device instances based on the devices\n- they detect and support on the running system, and are responsible for\n- managing the interactions with a camera device.\n-\n- More details can be found in the `PipelineHandler API`_ documentation, and the\n- `Pipeline Handler Writers Guide`_.\n-\n-.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html\n-\n-Image Processing Algorithms\n- An image processing algorithm (IPA) component is a loadable plugin that\n- implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other\n- algorithms.\n-\n- The algorithms run on the CPU and interact with the camera devices through the\n- Pipeline Handler to control hardware image processing based on the parameters\n- supplied by upper layers, maintaining state and closing the control loop\n- of the ISP.\n-\n- The component is sandboxed and can only interact with libcamera through the\n- API provided by the Pipeline Handler and an IPA has no direct access to kernel\n- camera devices.\n-\n- Open source IPA modules built with libcamera can be run in the same process\n- space as libcamera, however external IPA modules are run in a separate process\n- from the main libcamera process. IPA modules have a restricted view of the\n- system, including no access to networking APIs and limited access to file\n- systems.\n-\n- IPA modules are only required for platforms and devices with an ISP controlled\n- by the host CPU. Camera sensors which have an integrated ISP are not\n- controlled through the IPA module.\n-\n Platform Support\n ----------------\n \n", "prefixes": [ "v2", "3/7" ] }