| Message ID | 20240809145304.537551-4-dan.scally@ideasonboard.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
|
| Related | show |
Hi Dan, Thank you for the patch. On Fri, Aug 09, 2024 at 03:53:00PM +0100, Daniel Scally wrote: > There are two near-duplicate instances of the libcamera architecture > detail in the Documentation, in docs.rst and guides/introduction.rst. > The latter is more up-to-date, so remove it from the introduction > file (which will soon be deprecated) and update the section in docs. > > Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com> > --- > Documentation/docs.rst | 221 ++++++++++++++------------ > Documentation/guides/introduction.rst | 137 +--------------- > 2 files changed, 121 insertions(+), 237 deletions(-) I like the diffstat :-) > diff --git a/Documentation/docs.rst b/Documentation/docs.rst > index 5871961c..10f07a9e 100644 > --- a/Documentation/docs.rst > +++ b/Documentation/docs.rst > @@ -243,116 +243,135 @@ The camera stack comprises four software layers. From bottom to top: > libcamera Architecture > ====================== > > +While offering a unified API towards upper layers, and presenting itself as a > +single library, libcamera isn't monolithic. It exposes multiple components > +through its public API and is built around a set of separate helpers internally. > +Hardware abstractions are handled through the use of device-specific components > +where required and dynamically loadable plugins are used to separate image > +processing algorithms from the core libcamera codebase. > + > :: > > - ---------------------------< libcamera Public API >--------------------------- > - ^ ^ > - | | > - v v > - +-------------+ +-------------------------------------------------+ > - | Camera | | Camera Device | > - | Devices | | +---------------------------------------------+ | > - | Manager | | | Device-Agnostic | | > - +-------------+ | | | | > - ^ | | +------------------------+ | > - | | | | ~~~~~~~~~~~~~~~~~~~~~ | > - | | | | { +---------------+ } | > - | | | | } | ////Image//// | { | > - | | | | <-> | /Processing// | } | > - | | | | } | /Algorithms// | { | > - | | | | { +---------------+ } | > - | | | | ~~~~~~~~~~~~~~~~~~~~~ | > - | | | | ======================== | > - | | | | +---------------+ | > - | | | | | //Pipeline/// | | > - | | | | <-> | ///Handler/// | | > - | | | | | ///////////// | | > - | | +--------------------+ +---------------+ | > - | | Device-Specific | > - | +-------------------------------------------------+ > - | ^ ^ > - | | | > - v v v > - +--------------------------------------------------------------------+ > - | Helpers and Support Classes | > - | +-------------+ +-------------+ +-------------+ +-------------+ | > - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | > - | | Support | | Allocator | | IPC | | Manager | | > - | +-------------+ +-------------+ +-------------+ +-------------+ | > - | +-------------+ +-------------+ | > - | | Pipeline | | ... | | > - | | Runner | | | | > - | +-------------+ +-------------+ | > - +--------------------------------------------------------------------+ > - > - /// Device-Specific Components > - ~~~ Sandboxing > - > -While offering a unified API towards upper layers, and presenting > -itself as a single library, libcamera isn't monolithic. It exposes > -multiple components through its public API, is built around a set of > -separate helpers internally, uses device-specific components and can > -load dynamic plugins. > - > -Camera Devices Manager > - The Camera Devices Manager provides a view of available cameras > - in the system. It performs cold enumeration and runtime camera > - management, and supports a hotplug notification mechanism in its > - public API. > - > - To avoid the cost associated with cold enumeration of all devices > - at application start, and to arbitrate concurrent access to camera > - devices, the Camera Devices Manager could later be split to a > - separate service, possibly with integration in platform-specific > - device management. > + --------------------------< libcamera Public API >--------------------------- > + ^ ^ > + | | > + v v > + +-------------+ +---------------------------------------------------+ > + | Camera | | Camera Device | > + | Manager | | +-----------------------------------------------+ | > + +-------------+ | | Device-Agnostic | | > + ^ | | | | > + | | | +--------------------------+ | > + | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | > + | | | | { +-----------------+ } | > + | | | | } | //// Image //// | { | > + | | | | <-> | / Processing // | } | > + | | | | } | / Algorithms // | { | > + | | | | { +-----------------+ } | > + | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | > + | | | | ========================== | > + | | | | +-----------------+ | > + | | | | | // Pipeline /// | | > + | | | | <-> | /// Handler /// | | > + | | | | | /////////////// | | > + | | +--------------------+ +-----------------+ | > + | | Device-Specific | > + | +---------------------------------------------------+ > + | ^ ^ > + | | | > + v v v > + +--------------------------------------------------------------------+ > + | Helpers and Support Classes | > + | +-------------+ +-------------+ +-------------+ +-------------+ | > + | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | > + | | Support | | Allocator | | IPC | | Manager | | > + | +-------------+ +-------------+ +-------------+ +-------------+ | > + | +-------------+ +-------------+ | > + | | Pipeline | | ... | | > + | | Runner | | | | > + | +-------------+ +-------------+ | > + +--------------------------------------------------------------------+ > + > + /// Device-Specific Components > + ~~~ Sandboxing > + > +Camera Manager > + The Camera Manager enumerates cameras and instantiates Pipeline Handlers to > + manage each Camera that libcamera supports. The Camera Manager supports > + hotplug detection and notification events when supported by the underlying > + kernel devices. > + > + There is only ever one instance of the Camera Manager running per application. > + Each application's instance of the Camera Manager ensures that only a single > + application can take control of a camera device at once. > + > + Read the `Camera Manager API`_ documentation for more details. > + > +.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html It would be nice to avoid linking to an absolute URL, but that won't be easy to solve. As it's not a new issue, it's fine for now. > > Camera Device > - The Camera Device represents a camera device to upper layers. It > - exposes full control of the device through the public API, and is > - thus the highest level object exposed by libcamera. > + The Camera class represents a single item of camera hardware that is capable > + of producing one or more image streams, and provides the API to interact with > + the underlying device. > + > + If a system has multiple instances of the same hardware attached, each has its > + own instance of the camera class. > + > + The API exposes full control of the device to upper layers of libcamera through > + the public API, making it the highest level object libcamera exposes, and the > + object that all other API operations interact with from configuration to > + capture. > > - Camera Device instances are created by the Camera Devices > - Manager. An optional function to create new instances could be exposed > - through the public API to speed up initialization when the upper > - layer knows how to directly address camera devices present in the > - system. > + Read the `Camera API`_ documentation for more details. > + > +.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html > > Pipeline Handler > - The Pipeline Handler manages complex pipelines exposed by the kernel drivers > - through the Media Controller and V4L2 APIs. It abstracts pipeline handling to > - hide device-specific details to the rest of the library, and implements both > - pipeline configuration based on stream configuration, and pipeline runtime > - execution and scheduling when needed by the device. > - > - This component is device-specific and is part of the libcamera code base. As > - such it is covered by the same free software license as the rest of libcamera > - and needs to be contributed upstream by device vendors. The Pipeline Handler > - lives in the same process as the rest of the library, and has access to all > - helpers and kernel camera-related devices. > + The Pipeline Handler manages the complex pipelines exposed by the kernel > + drivers through the Media Controller and V4L2 APIs. It abstracts pipeline > + handling to hide device-specific details from the rest of the library, and > + implements both pipeline configuration based on stream configuration, and > + pipeline runtime execution and scheduling when needed by the device. > + > + The Pipeline Handler lives in the same process as the rest of the library, and > + has access to all helpers and kernel camera-related devices. > + > + Hardware abstraction is handled by device specific Pipeline Handlers which are > + derived from the Pipeline Handler base class allowing commonality to be shared > + among the implementations. > + > + Derived pipeline handlers create Camera device instances based on the devices > + they detect and support on the running system, and are responsible for > + managing the interactions with a camera device. > + > + More details can be found in the `PipelineHandler API`_ documentation, and the > + :doc:`Pipeline Handler Writers Guide <guides/pipeline-handler>`. > + > +.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html With the documentation for the public and internal APIs being split, this will be a dead link (once the copy on libcamera.org gets updated). It should be fixed in a separate patch. Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > Image Processing Algorithms > - Together with the hardware image processing and hardware statistics > - collection, the Image Processing Algorithms implement 3A (Auto-Exposure, > - Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU > - and interact with the kernel camera devices to control hardware image > - processing based on the parameters supplied by upper layers, closing the > - control loop of the ISP. > - > - This component is device-specific and is loaded as an external plugin. It can > - be part of the libcamera code base, in which case it is covered by the same > - license, or provided externally as an open-source or closed-source component. > - > - The component is sandboxed and can only interact with libcamera through > - internal APIs specifically marked as such. In particular it will have no > - direct access to kernel camera devices, and all its accesses to image and > - metadata will be mediated by dmabuf instances explicitly passed to the > - component. The component must be prepared to run in a process separate from > - the main libcamera process, and to have a very restricted view of the system, > - including no access to networking APIs and limited access to file systems. > - > - The sandboxing mechanism isn't defined by libcamera. One example > - implementation will be provided as part of the project, and platforms vendors > - will be able to provide their own sandboxing mechanism as a plugin. > + An image processing algorithm (IPA) component is a loadable plugin that > + implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other > + algorithms. > + > + The algorithms run on the CPU and interact with the camera devices through the > + Pipeline Handler to control hardware image processing based on the parameters > + supplied by upper layers, maintaining state and closing the control loop > + of the ISP. > + > + The component is sandboxed and can only interact with libcamera through the > + API provided by the Pipeline Handler and an IPA has no direct access to kernel > + camera devices. > + > + Open source IPA modules built with libcamera can be run in the same process > + space as libcamera, however external IPA modules are run in a separate process > + from the main libcamera process. IPA modules have a restricted view of the > + system, including no access to networking APIs and limited access to file > + systems. > + > + IPA modules are only required for platforms and devices with an ISP controlled > + by the host CPU. Camera sensors which have an integrated ISP are not > + controlled through the IPA module. > > libcamera should provide a basic implementation of Image Processing > Algorithms, to serve as a reference for the internal API. Device vendors are > diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst > index 8368bd4a..1898d5fe 100644 > --- a/Documentation/guides/introduction.rst > +++ b/Documentation/guides/introduction.rst > @@ -27,8 +27,7 @@ desirable results from the camera. > > > In this developers guide, we will explore the `Camera Stack`_ and how it is > -can be visualised at a high level, and explore the internal `Architecture`_ of > -the libcamera library with its components. The current `Platform Support`_ is > +can be visualised at a high level. The current `Platform Support`_ is > detailed, as well as an overview of the `Licensing`_ requirements of the > project. > > @@ -148,140 +147,6 @@ Native libcamera API > > .. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html > > -Architecture > ------------- > - > -While offering a unified API towards upper layers, and presenting itself as a > -single library, libcamera isn't monolithic. It exposes multiple components > -through its public API and is built around a set of separate helpers internally. > -Hardware abstractions are handled through the use of device-specific components > -where required and dynamically loadable plugins are used to separate image > -processing algorithms from the core libcamera codebase. > - > -:: > - > - --------------------------< libcamera Public API >--------------------------- > - ^ ^ > - | | > - v v > - +-------------+ +---------------------------------------------------+ > - | Camera | | Camera Device | > - | Manager | | +-----------------------------------------------+ | > - +-------------+ | | Device-Agnostic | | > - ^ | | | | > - | | | +--------------------------+ | > - | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | > - | | | | { +-----------------+ } | > - | | | | } | //// Image //// | { | > - | | | | <-> | / Processing // | } | > - | | | | } | / Algorithms // | { | > - | | | | { +-----------------+ } | > - | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | > - | | | | ========================== | > - | | | | +-----------------+ | > - | | | | | // Pipeline /// | | > - | | | | <-> | /// Handler /// | | > - | | | | | /////////////// | | > - | | +--------------------+ +-----------------+ | > - | | Device-Specific | > - | +---------------------------------------------------+ > - | ^ ^ > - | | | > - v v v > - +--------------------------------------------------------------------+ > - | Helpers and Support Classes | > - | +-------------+ +-------------+ +-------------+ +-------------+ | > - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | > - | | Support | | Allocator | | IPC | | Manager | | > - | +-------------+ +-------------+ +-------------+ +-------------+ | > - | +-------------+ +-------------+ | > - | | Pipeline | | ... | | > - | | Runner | | | | > - | +-------------+ +-------------+ | > - +--------------------------------------------------------------------+ > - > - /// Device-Specific Components > - ~~~ Sandboxing > - > - > -Camera Manager > - The Camera Manager enumerates cameras and instantiates Pipeline Handlers to > - manage each Camera that libcamera supports. The Camera Manager supports > - hotplug detection and notification events when supported by the underlying > - kernel devices. > - > - There is only ever one instance of the Camera Manager running per application. > - Each application's instance of the Camera Manager ensures that only a single > - application can take control of a camera device at once. > - > - Read the `Camera Manager API`_ documentation for more details. > - > -.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html > - > -Camera Device > - The Camera class represents a single item of camera hardware that is capable > - of producing one or more image streams, and provides the API to interact with > - the underlying device. > - > - If a system has multiple instances of the same hardware attached, each has its > - own instance of the camera class. > - > - The API exposes full control of the device to upper layers of libcamera through > - the public API, making it the highest level object libcamera exposes, and the > - object that all other API operations interact with from configuration to > - capture. > - > - Read the `Camera API`_ documentation for more details. > - > -.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html > - > -Pipeline Handler > - The Pipeline Handler manages the complex pipelines exposed by the kernel > - drivers through the Media Controller and V4L2 APIs. It abstracts pipeline > - handling to hide device-specific details from the rest of the library, and > - implements both pipeline configuration based on stream configuration, and > - pipeline runtime execution and scheduling when needed by the device. > - > - The Pipeline Handler lives in the same process as the rest of the library, and > - has access to all helpers and kernel camera-related devices. > - > - Hardware abstraction is handled by device specific Pipeline Handlers which are > - derived from the Pipeline Handler base class allowing commonality to be shared > - among the implementations. > - > - Derived pipeline handlers create Camera device instances based on the devices > - they detect and support on the running system, and are responsible for > - managing the interactions with a camera device. > - > - More details can be found in the `PipelineHandler API`_ documentation, and the > - `Pipeline Handler Writers Guide`_. > - > -.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html > - > -Image Processing Algorithms > - An image processing algorithm (IPA) component is a loadable plugin that > - implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other > - algorithms. > - > - The algorithms run on the CPU and interact with the camera devices through the > - Pipeline Handler to control hardware image processing based on the parameters > - supplied by upper layers, maintaining state and closing the control loop > - of the ISP. > - > - The component is sandboxed and can only interact with libcamera through the > - API provided by the Pipeline Handler and an IPA has no direct access to kernel > - camera devices. > - > - Open source IPA modules built with libcamera can be run in the same process > - space as libcamera, however external IPA modules are run in a separate process > - from the main libcamera process. IPA modules have a restricted view of the > - system, including no access to networking APIs and limited access to file > - systems. > - > - IPA modules are only required for platforms and devices with an ISP controlled > - by the host CPU. Camera sensors which have an integrated ISP are not > - controlled through the IPA module. > - > Platform Support > ---------------- >
diff --git a/Documentation/docs.rst b/Documentation/docs.rst index 5871961c..10f07a9e 100644 --- a/Documentation/docs.rst +++ b/Documentation/docs.rst @@ -243,116 +243,135 @@ The camera stack comprises four software layers. From bottom to top: libcamera Architecture ====================== +While offering a unified API towards upper layers, and presenting itself as a +single library, libcamera isn't monolithic. It exposes multiple components +through its public API and is built around a set of separate helpers internally. +Hardware abstractions are handled through the use of device-specific components +where required and dynamically loadable plugins are used to separate image +processing algorithms from the core libcamera codebase. + :: - ---------------------------< libcamera Public API >--------------------------- - ^ ^ - | | - v v - +-------------+ +-------------------------------------------------+ - | Camera | | Camera Device | - | Devices | | +---------------------------------------------+ | - | Manager | | | Device-Agnostic | | - +-------------+ | | | | - ^ | | +------------------------+ | - | | | | ~~~~~~~~~~~~~~~~~~~~~ | - | | | | { +---------------+ } | - | | | | } | ////Image//// | { | - | | | | <-> | /Processing// | } | - | | | | } | /Algorithms// | { | - | | | | { +---------------+ } | - | | | | ~~~~~~~~~~~~~~~~~~~~~ | - | | | | ======================== | - | | | | +---------------+ | - | | | | | //Pipeline/// | | - | | | | <-> | ///Handler/// | | - | | | | | ///////////// | | - | | +--------------------+ +---------------+ | - | | Device-Specific | - | +-------------------------------------------------+ - | ^ ^ - | | | - v v v - +--------------------------------------------------------------------+ - | Helpers and Support Classes | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | - | | Support | | Allocator | | IPC | | Manager | | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | +-------------+ +-------------+ | - | | Pipeline | | ... | | - | | Runner | | | | - | +-------------+ +-------------+ | - +--------------------------------------------------------------------+ - - /// Device-Specific Components - ~~~ Sandboxing - -While offering a unified API towards upper layers, and presenting -itself as a single library, libcamera isn't monolithic. It exposes -multiple components through its public API, is built around a set of -separate helpers internally, uses device-specific components and can -load dynamic plugins. - -Camera Devices Manager - The Camera Devices Manager provides a view of available cameras - in the system. It performs cold enumeration and runtime camera - management, and supports a hotplug notification mechanism in its - public API. - - To avoid the cost associated with cold enumeration of all devices - at application start, and to arbitrate concurrent access to camera - devices, the Camera Devices Manager could later be split to a - separate service, possibly with integration in platform-specific - device management. + --------------------------< libcamera Public API >--------------------------- + ^ ^ + | | + v v + +-------------+ +---------------------------------------------------+ + | Camera | | Camera Device | + | Manager | | +-----------------------------------------------+ | + +-------------+ | | Device-Agnostic | | + ^ | | | | + | | | +--------------------------+ | + | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | + | | | | { +-----------------+ } | + | | | | } | //// Image //// | { | + | | | | <-> | / Processing // | } | + | | | | } | / Algorithms // | { | + | | | | { +-----------------+ } | + | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | + | | | | ========================== | + | | | | +-----------------+ | + | | | | | // Pipeline /// | | + | | | | <-> | /// Handler /// | | + | | | | | /////////////// | | + | | +--------------------+ +-----------------+ | + | | Device-Specific | + | +---------------------------------------------------+ + | ^ ^ + | | | + v v v + +--------------------------------------------------------------------+ + | Helpers and Support Classes | + | +-------------+ +-------------+ +-------------+ +-------------+ | + | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | + | | Support | | Allocator | | IPC | | Manager | | + | +-------------+ +-------------+ +-------------+ +-------------+ | + | +-------------+ +-------------+ | + | | Pipeline | | ... | | + | | Runner | | | | + | +-------------+ +-------------+ | + +--------------------------------------------------------------------+ + + /// Device-Specific Components + ~~~ Sandboxing + +Camera Manager + The Camera Manager enumerates cameras and instantiates Pipeline Handlers to + manage each Camera that libcamera supports. The Camera Manager supports + hotplug detection and notification events when supported by the underlying + kernel devices. + + There is only ever one instance of the Camera Manager running per application. + Each application's instance of the Camera Manager ensures that only a single + application can take control of a camera device at once. + + Read the `Camera Manager API`_ documentation for more details. + +.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html Camera Device - The Camera Device represents a camera device to upper layers. It - exposes full control of the device through the public API, and is - thus the highest level object exposed by libcamera. + The Camera class represents a single item of camera hardware that is capable + of producing one or more image streams, and provides the API to interact with + the underlying device. + + If a system has multiple instances of the same hardware attached, each has its + own instance of the camera class. + + The API exposes full control of the device to upper layers of libcamera through + the public API, making it the highest level object libcamera exposes, and the + object that all other API operations interact with from configuration to + capture. - Camera Device instances are created by the Camera Devices - Manager. An optional function to create new instances could be exposed - through the public API to speed up initialization when the upper - layer knows how to directly address camera devices present in the - system. + Read the `Camera API`_ documentation for more details. + +.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html Pipeline Handler - The Pipeline Handler manages complex pipelines exposed by the kernel drivers - through the Media Controller and V4L2 APIs. It abstracts pipeline handling to - hide device-specific details to the rest of the library, and implements both - pipeline configuration based on stream configuration, and pipeline runtime - execution and scheduling when needed by the device. - - This component is device-specific and is part of the libcamera code base. As - such it is covered by the same free software license as the rest of libcamera - and needs to be contributed upstream by device vendors. The Pipeline Handler - lives in the same process as the rest of the library, and has access to all - helpers and kernel camera-related devices. + The Pipeline Handler manages the complex pipelines exposed by the kernel + drivers through the Media Controller and V4L2 APIs. It abstracts pipeline + handling to hide device-specific details from the rest of the library, and + implements both pipeline configuration based on stream configuration, and + pipeline runtime execution and scheduling when needed by the device. + + The Pipeline Handler lives in the same process as the rest of the library, and + has access to all helpers and kernel camera-related devices. + + Hardware abstraction is handled by device specific Pipeline Handlers which are + derived from the Pipeline Handler base class allowing commonality to be shared + among the implementations. + + Derived pipeline handlers create Camera device instances based on the devices + they detect and support on the running system, and are responsible for + managing the interactions with a camera device. + + More details can be found in the `PipelineHandler API`_ documentation, and the + :doc:`Pipeline Handler Writers Guide <guides/pipeline-handler>`. + +.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html Image Processing Algorithms - Together with the hardware image processing and hardware statistics - collection, the Image Processing Algorithms implement 3A (Auto-Exposure, - Auto-White Balance and Auto-Focus) and other algorithms. They run on the CPU - and interact with the kernel camera devices to control hardware image - processing based on the parameters supplied by upper layers, closing the - control loop of the ISP. - - This component is device-specific and is loaded as an external plugin. It can - be part of the libcamera code base, in which case it is covered by the same - license, or provided externally as an open-source or closed-source component. - - The component is sandboxed and can only interact with libcamera through - internal APIs specifically marked as such. In particular it will have no - direct access to kernel camera devices, and all its accesses to image and - metadata will be mediated by dmabuf instances explicitly passed to the - component. The component must be prepared to run in a process separate from - the main libcamera process, and to have a very restricted view of the system, - including no access to networking APIs and limited access to file systems. - - The sandboxing mechanism isn't defined by libcamera. One example - implementation will be provided as part of the project, and platforms vendors - will be able to provide their own sandboxing mechanism as a plugin. + An image processing algorithm (IPA) component is a loadable plugin that + implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other + algorithms. + + The algorithms run on the CPU and interact with the camera devices through the + Pipeline Handler to control hardware image processing based on the parameters + supplied by upper layers, maintaining state and closing the control loop + of the ISP. + + The component is sandboxed and can only interact with libcamera through the + API provided by the Pipeline Handler and an IPA has no direct access to kernel + camera devices. + + Open source IPA modules built with libcamera can be run in the same process + space as libcamera, however external IPA modules are run in a separate process + from the main libcamera process. IPA modules have a restricted view of the + system, including no access to networking APIs and limited access to file + systems. + + IPA modules are only required for platforms and devices with an ISP controlled + by the host CPU. Camera sensors which have an integrated ISP are not + controlled through the IPA module. libcamera should provide a basic implementation of Image Processing Algorithms, to serve as a reference for the internal API. Device vendors are diff --git a/Documentation/guides/introduction.rst b/Documentation/guides/introduction.rst index 8368bd4a..1898d5fe 100644 --- a/Documentation/guides/introduction.rst +++ b/Documentation/guides/introduction.rst @@ -27,8 +27,7 @@ desirable results from the camera. In this developers guide, we will explore the `Camera Stack`_ and how it is -can be visualised at a high level, and explore the internal `Architecture`_ of -the libcamera library with its components. The current `Platform Support`_ is +can be visualised at a high level. The current `Platform Support`_ is detailed, as well as an overview of the `Licensing`_ requirements of the project. @@ -148,140 +147,6 @@ Native libcamera API .. _GStreamer element: https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html -Architecture ------------- - -While offering a unified API towards upper layers, and presenting itself as a -single library, libcamera isn't monolithic. It exposes multiple components -through its public API and is built around a set of separate helpers internally. -Hardware abstractions are handled through the use of device-specific components -where required and dynamically loadable plugins are used to separate image -processing algorithms from the core libcamera codebase. - -:: - - --------------------------< libcamera Public API >--------------------------- - ^ ^ - | | - v v - +-------------+ +---------------------------------------------------+ - | Camera | | Camera Device | - | Manager | | +-----------------------------------------------+ | - +-------------+ | | Device-Agnostic | | - ^ | | | | - | | | +--------------------------+ | - | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | - | | | | { +-----------------+ } | - | | | | } | //// Image //// | { | - | | | | <-> | / Processing // | } | - | | | | } | / Algorithms // | { | - | | | | { +-----------------+ } | - | | | | ~~~~~~~~~~~~~~~~~~~~~~~ | - | | | | ========================== | - | | | | +-----------------+ | - | | | | | // Pipeline /// | | - | | | | <-> | /// Handler /// | | - | | | | | /////////////// | | - | | +--------------------+ +-----------------+ | - | | Device-Specific | - | +---------------------------------------------------+ - | ^ ^ - | | | - v v v - +--------------------------------------------------------------------+ - | Helpers and Support Classes | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | | MC & V4L2 | | Buffers | | Sandboxing | | Plugins | | - | | Support | | Allocator | | IPC | | Manager | | - | +-------------+ +-------------+ +-------------+ +-------------+ | - | +-------------+ +-------------+ | - | | Pipeline | | ... | | - | | Runner | | | | - | +-------------+ +-------------+ | - +--------------------------------------------------------------------+ - - /// Device-Specific Components - ~~~ Sandboxing - - -Camera Manager - The Camera Manager enumerates cameras and instantiates Pipeline Handlers to - manage each Camera that libcamera supports. The Camera Manager supports - hotplug detection and notification events when supported by the underlying - kernel devices. - - There is only ever one instance of the Camera Manager running per application. - Each application's instance of the Camera Manager ensures that only a single - application can take control of a camera device at once. - - Read the `Camera Manager API`_ documentation for more details. - -.. _Camera Manager API: https://libcamera.org/api-html/classlibcamera_1_1CameraManager.html - -Camera Device - The Camera class represents a single item of camera hardware that is capable - of producing one or more image streams, and provides the API to interact with - the underlying device. - - If a system has multiple instances of the same hardware attached, each has its - own instance of the camera class. - - The API exposes full control of the device to upper layers of libcamera through - the public API, making it the highest level object libcamera exposes, and the - object that all other API operations interact with from configuration to - capture. - - Read the `Camera API`_ documentation for more details. - -.. _Camera API: https://libcamera.org/api-html/classlibcamera_1_1Camera.html - -Pipeline Handler - The Pipeline Handler manages the complex pipelines exposed by the kernel - drivers through the Media Controller and V4L2 APIs. It abstracts pipeline - handling to hide device-specific details from the rest of the library, and - implements both pipeline configuration based on stream configuration, and - pipeline runtime execution and scheduling when needed by the device. - - The Pipeline Handler lives in the same process as the rest of the library, and - has access to all helpers and kernel camera-related devices. - - Hardware abstraction is handled by device specific Pipeline Handlers which are - derived from the Pipeline Handler base class allowing commonality to be shared - among the implementations. - - Derived pipeline handlers create Camera device instances based on the devices - they detect and support on the running system, and are responsible for - managing the interactions with a camera device. - - More details can be found in the `PipelineHandler API`_ documentation, and the - `Pipeline Handler Writers Guide`_. - -.. _PipelineHandler API: https://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html - -Image Processing Algorithms - An image processing algorithm (IPA) component is a loadable plugin that - implements 3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) and other - algorithms. - - The algorithms run on the CPU and interact with the camera devices through the - Pipeline Handler to control hardware image processing based on the parameters - supplied by upper layers, maintaining state and closing the control loop - of the ISP. - - The component is sandboxed and can only interact with libcamera through the - API provided by the Pipeline Handler and an IPA has no direct access to kernel - camera devices. - - Open source IPA modules built with libcamera can be run in the same process - space as libcamera, however external IPA modules are run in a separate process - from the main libcamera process. IPA modules have a restricted view of the - system, including no access to networking APIs and limited access to file - systems. - - IPA modules are only required for platforms and devices with an ISP controlled - by the host CPU. Camera sensors which have an integrated ISP are not - controlled through the IPA module. - Platform Support ----------------
There are two near-duplicate instances of the libcamera architecture detail in the Documentation, in docs.rst and guides/introduction.rst. The latter is more up-to-date, so remove it from the introduction file (which will soon be deprecated) and update the section in docs. Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com> --- Documentation/docs.rst | 221 ++++++++++++++------------ Documentation/guides/introduction.rst | 137 +--------------- 2 files changed, 121 insertions(+), 237 deletions(-)