{"id":8551,"url":"https://patchwork.libcamera.org/api/1.1/patches/8551/?format=json","web_url":"https://patchwork.libcamera.org/patch/8551/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200702104542.19998-1-chris@gregariousmammal.com>","date":"2020-07-02T10:45:42","name":"[libcamera-devel,v2] Create libcamera overview document and glossary","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"d2849984563214292b84ae879df94ea087ad11ca","submitter":{"id":55,"url":"https://patchwork.libcamera.org/api/1.1/people/55/?format=json","name":"Chris Chinchilla","email":"chris@gregariousmammal.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/8551/mbox/","series":[{"id":1071,"url":"https://patchwork.libcamera.org/api/1.1/series/1071/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1071","date":"2020-07-02T10:45:42","name":"[libcamera-devel,v2] Create libcamera overview document and glossary","version":2,"mbox":"https://patchwork.libcamera.org/series/1071/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/8551/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/8551/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E011ABFFE2\n\tfor ;\n\tThu, 2 Jul 2020 10:45:55 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AA807603B4;\n\tThu, 2 Jul 2020 12:45:55 +0200 (CEST)","from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com\n\t[IPv6:2a00:1450:4864:20::52a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8AD5D603B3\n\tfor ;\n\tThu, 2 Jul 2020 12:45:54 +0200 (CEST)","by mail-ed1-x52a.google.com with SMTP id a8so21921124edy.1\n\tfor ;\n\tThu, 02 Jul 2020 03:45:54 -0700 (PDT)","from localhost.localdomain (p54ac54f4.dip0.t-ipconnect.de.\n\t[84.172.84.244]) by smtp.gmail.com with ESMTPSA id\n\tv5sm6598834ejj.61.2020.07.02.03.45.52\n\t(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 02 Jul 2020 03:45:53 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gregariousmammal.com\n\theader.i=@gregariousmammal.com header.b=\"OQGhkzZO\"; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gregariousmammal.com; s=google;\n\th=from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=vlYAanhGYkT1uDpZPCuiOzWFVDiMYT/XmAs2ApXweto=;\n\tb=OQGhkzZO/S53WyyFVvypWBppFc35cDzQII1Q08KmJCw9ZNiic3BulCp8NhVl1TYCzO\n\tgHoNH4WhAGAUPXtySW0x6rEle/XlzZCCwYK0Nmj1qpY0P0hqXDrjSrkuhvJ0ay/XvX6R\n\tA3vUW2FAHW3IWwUpa1V3POgpPEGkHiRT01NZeByXZD8yqmRbgC1DTKFx/U7MST3ZHVTD\n\tHrvKDFCQ1rlYcGnMJiSm06Ikfs8gFjichUxtuUx9234mM8aQ+UQKo6j5KXH+4pbyEJUY\n\tMHG95eUzuH+smHojeVH3QIugBIlj8FkI03fGV+5hGCKni9J8Rw9ZujALcZvZ/OBNqnVv\n\tAwOg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=vlYAanhGYkT1uDpZPCuiOzWFVDiMYT/XmAs2ApXweto=;\n\tb=kJ7FHPt4A8KDYci8r94UFbyPKoYY87qT14vwKqOuKGoi4lYMChpxU0JKPjGth5KxbP\n\tFHdEdfmYSSTUaJStQxPzMS9hCud5sy7E5X/CtHlo0A9HFYnU+sS4H1wmS8fqScT+vKkH\n\td1AOSVQX0OLk471hYydpNBYIeSowDDkcjnrPlZ3bYo8eqdxW9LccTKeIxmIIXubePaCK\n\tbIbnHZYAZM3BmcGKUdiANHRgvPuxGr7R1eZz4kpCxGOGemyqc3qUIPMbWu2WLNEpMXOT\n\tj2SQriuwjFZTDjcQ4E9cX4EgOpgU21XDGCBhRQ9nVApe9ooNEvfH1fLS3GS7XXLjfOdE\n\tQUsg==","X-Gm-Message-State":"AOAM532iZfc+xXhkopmSv29Cge2TsxGNnO141+kFzwNJ92NpF8ftNrK8\n\tSHMjGEi54SskD+tzrQCU92HWnNPK/T8=","X-Google-Smtp-Source":"ABdhPJwDNAJOfo5AKhaU6KzqszgX8Z/Rs6Qr7lCn7wVXMhx66YmB0dWfuSdh5U06KEJVpt4LJzERFw==","X-Received":"by 2002:a50:bec2:: with SMTP id e2mr35546573edk.3.1593686753694; \n\tThu, 02 Jul 2020 03:45:53 -0700 (PDT)","From":"chris@gregariousmammal.com","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 2 Jul 2020 12:45:42 +0200","Message-Id":"<20200702104542.19998-1-chris@gregariousmammal.com>","X-Mailer":"git-send-email 2.27.0","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v2] Create libcamera overview document and\n\tglossary","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Chris Chinchilla ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Chris Chinchilla \n\nCreates a libcamera architecture overview guide and glossary.\n\nSigned-off-by: Chris Chinchilla \n---\n Documentation/introduction.rst | 197 +++++++++++++++++++++++++++++++++\n 1 file changed, 197 insertions(+)\n create mode 100644 Documentation/introduction.rst","diff":"diff --git a/Documentation/introduction.rst b/Documentation/introduction.rst\nnew file mode 100644\nindex 0000000..cf914c7\n--- /dev/null\n+++ b/Documentation/introduction.rst\n@@ -0,0 +1,197 @@\n+An introduction to libcamera\n+============================\n+\n+The Video for Linux 2 (`V4L2 `_) API provides kernel drivers for devices that provide and manipulate\n+images and video. However, Linux was missing a convenient way for application developers to take \n+advantage of these kernel drivers in userspace. Vendors of devices that provide image input sources \n+referred to as \"image signal processors\" (ISPs) sometimes contribute open-source V4L2 drivers. \n+However, ISPs vary so much, and it's a hard task for developers to write portable ISP-based\n+applications.\n+\n+The libcamera library aims to fill this gap by providing a complete userspace camera \n+stack for Linux-based systems that supports a wide variety of ISPs, including systems with multiple ISPs attached.\n+\n+The library currently supports:\n+\n+- Hardware\n+\n+ - Intel IPU3\n+ - Rockchip RK3399\n+ - RaspberryPi 3 and 4\n+ - USB video device class (UVC) cameras\n+ - Virtual media controller (vimc) driver\n+\n+- Software\n+\n+ - ChromeOS support through an Android HAL3 adaptation layer\n+ - Image processing algorithms\n+ - A V4L2 compatibility layer for existing V4L2 based applications\n+ - A gstreamer element for gstreamer pipeline based applications.\n+\n+The library provides a public API for managing ISPs, frame capture, video streams, frame and \n+request metadata, events and callbacks, image processing, and more.\n+\n+Where libcamera sits in the camera stack\n+----------------------------------------\n+\n+The libcamera library sits in userspace, just on top of the kernel drivers that directly interact \n+with hardware and the V4L2 family of APIs (Media Controller, V4L2 Video Device, and V4L2 sub-device APIs).\n+\n+When using libcamera in a camera stack, it is the core component, taking control of all camera \n+devices, and exposing a native C++ API to upper layers of the framework. Other language bindings are in development.\n+\n+Compatibility Layers\n+~~~~~~~~~~~~~~~~~~~~\n+\n+In a layer above the core framework are compatibility libraries to help existing applications and their developers.\n+\n+V4L2 Compatibility Layer\n+^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+To emulate high-level V4L2 camera devices and capture all access to camera devices, libcamera uses \n+a shared library. The shared library is transparent to libcamera-based applications and injected \n+into a process address space using dynamic linking with \"LD_PRELOAD \".\n+\n+The compatibility layer exposes camera device features on a best-effort basis and aims for the \n+level of features traditionally available from a UVC camera designed for video conferencing.\n+\n+Android Camera HAL\n+^^^^^^^^^^^^^^^^^^\n+\n+The libcamera library offers Android camera support through a generic `hardware abstraction layer (HAL) `_ implementation. \n+The HAL focuses on supporting features required by Android that are missing from libcamera, such as JPEG encoding.\n+\n+The Android camera HAL implementation initially targets the \"LIMITED \"\n+`hardware level `_,\n+with support for the \"FULL \"level then gradually implemented.\n+\n+gstreamer element\n+^^^^^^^^^^^^^^^^^\n+\n+The library provides `a gstreamer element `_ that routes libcamera data for \n+further processing in a gstreamer pipeline.\n+\n+libcamera Architecture\n+----------------------\n+\n+The libcamera library exposes one unified API, but behind that is built from \n+re-usable components that provide hardware specific support and configuration \n+with a device agnostic API.\n+\n+Camera Manager\n+~~~~~~~~~~~~~~\n+\n+It enumerates cameras at runtime and instantiates a `Pipeline Handler`_ to manage each Camera \n+device that libcamera supports. The Camera Manager supports hotplug detection\n+and notification events when supported by the underlying kernel devices.\n+\n+There is only ever one instance of the Camera Manager running per application. Each application's \n+instance of the Camera Manager ensures that only a single application can take control of a camera \n+device at once.\n+\n+`Read the Camera Manager API documentation for more details `_\n+\n+Camera Device\n+~~~~~~~~~~~~~\n+\n+The camera class represents a single item of camera hardware that is capable of producing one or more image streams, and provides the API to interact with the underlying device. \n+\n+If a system has multiple instances of the same hardware attached, each has it's own instance of the camera class.\n+\n+The API exposes full control of the device to upper layers of libcamera through the public API, \n+making it the highest level object libcamera exposes, and the object that all other API operations \n+interact with from configuration to capture.\n+\n+`Read the Camera API documentation for more details `_\n+\n+Frame controls\n+^^^^^^^^^^^^^^\n+\n+Depending on the capabilities of a camera device and the hardware it is connected \n+to, libcamera supports controlling capture parameters for each stream on a per-frame basis.\n+\n+These controls include auto exposure, gain, brightness, contrast, lux, white balance, color \n+temperature, and saturation.\n+\n+`Read the Control API documentation for more details `_\n+\n+Pipeline Handler\n+~~~~~~~~~~~~~~~~\n+\n+Pipeline handlers are the abstraction layer for platform-specific hardware configuration. They \n+access and control hardware through the V4L2 and Media Controller kernel interfaces, and implement \n+an internal API to control the ISP and Capture components of a pipeline directly.\n+\n+Pipeline handlers' create' Camera device instances based on the devices they detect and support on \n+the running system.\n+\n+A pipeline handler manages most aspects of interacting with a camera device including:\n+\n+- frame controls\n+- pipelines and stream configuration\n+- the data the camera produces, and the buffers it needs to hold the data\n+- granting access to camera data\n+\n+Pipeline handlers form part of the libcamera codebase, and developers need to implement them for \n+complex hardware with an ISP that requires device-specific configurations.\n+\n+`Read the PipelineHandler API documentation for more details `_\n+\n+Image Processing Algorithms\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+An image processing algorithm (IPA) component is a private API that implements \n+3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) algorithms.\n+\n+Each supported camera device has its own IPA component that runs on the CPU and \n+interacts with the kernel camera devices to control hardware image processing \n+based on the parameters supplied by upper layers, and helps maintain state, \n+closing the control loop of the ISP.\n+\n+An IPA component can be part of the libcamera code base, in which case the same \n+license covers it, or provided externally as an open-source or closed-source component.\n+\n+The component is sandboxed and can only interact with libcamera through specifically \n+marked APIs. A component has no direct access to kernel camera devices, and dmabuf \n+instances explicitly passed to the component control its access to image data and \n+metadata. The component should run in a process separate from the main libcamera \n+process, and has a restricted view of the system, including no access to networking APIs\n+and limited access to file systems.\n+\n+While libcamera requires sandboxing, the implementation is platform-specific, and handled by \n+platform-specific plugins.\n+\n+To support this security and sandboxing model, libcamera provides an IPC mechanism \n+for an IPA and Pipeline Handler to communicate, but developers need to create the \n+API themselves. Platform vendors can also use any other IPC mechanism that supports \n+passing file descriptors.\n+\n+3A and Image Enhancement Algorithms\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Camera devices should implement auto exposure, auto gain, and auto white balance, \n+and those that include a focus lens should also implement autofocus.\n+\n+Device vendors implement the control methods required to control these algorithms in hardware or \n+firmware outside of libcamera, or in an IPA component.\n+\n+.. TODO: Add link to guide when completed\n+\n+Helpers and Support Classes\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+.. TODO: Feel like this is implied in other places of the doc\n+\n+To help developers create device-specific pipeline handlers and image processing \n+algorithms, libcamera provides helpers and support classes that sit on top of the \n+Media Controller and V4L2 APIs.\n+\n+Glossary\n+--------\n+\n+- **ISP**: Image Signal Processor.\n+- **Media controller API**: `The Linux Kernel API `_ that handles audio and video input and output.\n+- **V4L2**: `Video for Linux API version 2 `_. Device drivers and API for video capture on Linux.\n+- **UVC camera**: `USB Video Class `_ that describes devices capable of streaming video over the UVC protocol.\n+- **3A**: Common algorithms used by camera devices for auto-exposure, auto-white balance and auto-focus).\n+- **IPC**: `Inter-process communications based protocol `_.\n","prefixes":["libcamera-devel","v2"]}