Patch Detail
Show a patch.
GET /api/patches/8740/?format=api
{ "id": 8740, "url": "https://patchwork.libcamera.org/api/patches/8740/?format=api", "web_url": "https://patchwork.libcamera.org/patch/8740/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20200710103016.85455-1-chris@gregariousmammal.com>", "date": "2020-07-10T10:30:16", "name": "[libcamera-devel,v3] Create libcamera overview document and glossary", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "58afb731eeb2b183f0c8872fd3b0e9d2c745fd4f", "submitter": { "id": 55, "url": "https://patchwork.libcamera.org/api/people/55/?format=api", "name": "Chris Chinchilla", "email": "chris@gregariousmammal.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/8740/mbox/", "series": [ { "id": 1095, "url": "https://patchwork.libcamera.org/api/series/1095/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=1095", "date": "2020-07-10T10:30:16", "name": "[libcamera-devel,v3] Create libcamera overview document and glossary", "version": 3, "mbox": "https://patchwork.libcamera.org/series/1095/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/8740/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/8740/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 381FCBD790\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri, 10 Jul 2020 10:30:24 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0330A6118A;\n\tFri, 10 Jul 2020 12:30:24 +0200 (CEST)", "from mail-ed1-x536.google.com (mail-ed1-x536.google.com\n\t[IPv6:2a00:1450:4864:20::536])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3211E61186\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 10 Jul 2020 12:30:22 +0200 (CEST)", "by mail-ed1-x536.google.com with SMTP id d16so4232430edz.12\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 10 Jul 2020 03:30:22 -0700 (PDT)", "from localhost.localdomain\n\t(business-90-187-212-1.pool2.vodafone-ip.de. [90.187.212.1])\n\tby smtp.gmail.com with ESMTPSA id\n\to17sm3463592ejb.105.2020.07.10.03.30.20\n\t(version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);\n\tFri, 10 Jul 2020 03:30:20 -0700 (PDT)" ], "Authentication-Results": "lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gregariousmammal.com\n\theader.i=@gregariousmammal.com header.b=\"qqHDC2g9\"; \n\tdkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gregariousmammal.com; s=google;\n\th=from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=MhEx4ZV+ukCmmJIPE6LpT6c4d9ghZUWOH33mWjtI7nQ=;\n\tb=qqHDC2g9lVgN6buCTxZKhFd6MmGfba7kjRZcMyBNjqBoplKrB6WLdnBWSRntV6I9In\n\teIKURWbiyEesMzMOWoG0oFWYKmm8w9P2NEukc2b6ez0f3hvh7L7sdmyYwTDafg2hukoW\n\tGLZDtIlMqNVJRC59lE0s2/PVoMUNhog2kXXfw0FKLHrjhDzY2uCtRpAHxM/MsihtRa5D\n\tEWybhwbXmx2OALpGDMaLenyiHDzxPiRa6+z+hkUeTv5OSJWWcctkLzIIaLQnp9aAU0ev\n\todgO6fTj3s7tIGR8CDh8N9bOWlfLrEXx/y/9VKsS+UaVtF6e1A1D7qzuVfA82s3acAkc\n\t2TAg==", "X-Google-DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=MhEx4ZV+ukCmmJIPE6LpT6c4d9ghZUWOH33mWjtI7nQ=;\n\tb=thoj5gm5G1rioy8Dsf3IW5srtLbRxycAUxnSZDiiF0WOpTISGMZPI9bnU5vcKOLohm\n\tafwvl95/PkG+wdIR2nAV8vkRa/D2yJ+q9jJAbbn5gX3l7CR08xqUWxBxUMlKtBObE/JV\n\tJlNN7RuNU1LA+pcoieJPk63zBzxavmi2pyRb+J1PLULJ9ahcOtliA+Jw0UaC9CNqZgq0\n\tlhJq7Qab9reMwoOzqlRpEXWGqdpFyk4VKZLkYgEYa0k/i4J44gwUOVYRFD1M+n47rzc9\n\tMC4XAy8n1QWUjQm7XRggKXBGbAHOmagTtLRZwjn4EPFQ5OnxVLjhTS9Gb2U4DrT+50Tv\n\t15ag==", "X-Gm-Message-State": "AOAM532hMY5D+T4SeaB76cXunSVfC2WEqn792cAFIBFSK+OETSrB6jfV\n\tOkZ6RUKV2laS1bmUX18a7zArKf3WP3uwvQqE", "X-Google-Smtp-Source": "ABdhPJxCA166YNbbwbYJsgAmT9oxSPJMN/P/v38gOa4Dzu506QVEPR7YYZCkfKbjpa+nBP3X4+q0ew==", "X-Received": "by 2002:a50:f413:: with SMTP id\n\tr19mr80233139edm.17.1594377021245; \n\tFri, 10 Jul 2020 03:30:21 -0700 (PDT)", "From": "chris@gregariousmammal.com", "To": "libcamera-devel@lists.libcamera.org", "Date": "Fri, 10 Jul 2020 12:30:16 +0200", "Message-Id": "<20200710103016.85455-1-chris@gregariousmammal.com>", "X-Mailer": "git-send-email 2.27.0", "MIME-Version": "1.0", "Subject": "[libcamera-devel] [PATCH v3] Create libcamera overview document and\n\tglossary", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Cc": "Chris Chinchilla <chris@gregariousmammal.com>", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "From: Chris Chinchilla <chris@gregariousmammal.com>\n\nCreates a libcamera architecture overview guide and glossary.\n\nSigned-off-by: Chris Chinchilla <chris@gregariousmammal.com>\n---\n Documentation/introduction.rst | 198 +++++++++++++++++++++++++++++++++\n 1 file changed, 198 insertions(+)\n create mode 100644 Documentation/introduction.rst", "diff": "diff --git a/Documentation/introduction.rst b/Documentation/introduction.rst\nnew file mode 100644\nindex 0000000..1e87218\n--- /dev/null\n+++ b/Documentation/introduction.rst\n@@ -0,0 +1,198 @@\n+An introduction to libcamera\n+============================\n+\n+The Video for Linux 2 (`V4L2 <https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/v4l2.html>`_) API provides kernel drivers for devices that provide and manipulate\n+images and video. However, Linux was missing a convenient way for application developers to take \n+advantage of these kernel drivers in userspace. Vendors of devices that provide image input sources \n+referred to as \"image signal processors\" (ISPs) sometimes contribute open-source V4L2 drivers. \n+However, ISPs vary so much, and it's a hard task for developers to write portable ISP-based\n+applications.\n+\n+The libcamera library aims to fill this gap by providing a complete userspace camera \n+stack for Linux-based systems that supports a wide variety of ISPs, including systems with multiple ISPs attached.\n+\n+The library currently supports:\n+\n+- Hardware\n+\n+ - Intel IPU3\n+ - Rockchip RK3399\n+ - RaspberryPi 3 and 4\n+ - USB video device class (UVC) cameras\n+ - Virtual media controller (vimc) driver\n+\n+- Software\n+\n+ - Image processing algorithms\n+ - A V4L2 compatibility layer for existing V4L2 based applications\n+ - A gstreamer element for gstreamer pipeline based applications.\n+ - ChromeOS support through an Android HAL3 adaptation layer (does not have V4L2 or gstreamer support)\n+\n+The library provides a public API for managing ISPs, frame capture, video streams, frame and \n+request metadata, events and callbacks, image processing, and more.\n+\n+Where libcamera sits in the camera stack\n+----------------------------------------\n+\n+The libcamera library sits in userspace, just on top of the kernel drivers that directly interact \n+with hardware and the V4L2 family of APIs (Media Controller, V4L2 Video Device, and V4L2 sub-device APIs).\n+\n+When using libcamera in a camera stack, it is the core component, taking control of all camera \n+devices, and exposing a native C++ API to upper layers of the framework. Other language bindings are in development.\n+\n+Compatibility Layers\n+~~~~~~~~~~~~~~~~~~~~\n+\n+In a layer above the core framework are compatibility libraries to help existing applications and their developers.\n+\n+V4L2 Compatibility Layer\n+^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+To emulate high-level V4L2 camera devices and capture all access to camera devices, libcamera uses \n+a shared library. The shared library is transparent to libcamera-based applications and injected \n+into a process address space using dynamic linking with \"LD_PRELOAD \".\n+\n+The compatibility layer exposes camera device features on a best-effort basis and aims for the \n+level of features traditionally available from a UVC camera designed for video conferencing.\n+\n+Android and ChromeOS Camera HAL\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+ChromeOS uses the Android `hardware abstraction layer (HAL) <https://source.android.com/devices/camera/camera3_requests_hal>`_ to provide camera support, and libcamera uses that to offer support\n+for ChromeOS-based devices.\n+\n+The HAL focuses on supporting features required by Android that are missing from libcamera, such as\n+JPEG encoding.\n+\n+The HAL implementation initially targets the \"LIMITED \"\n+`hardware level <https://source.android.com/devices/camera/versioning#camera_api2>`_,\n+with support for the \"FULL \"level then gradually implemented.\n+\n+gstreamer element\n+^^^^^^^^^^^^^^^^^\n+\n+The library provides `a gstreamer element <https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html?gi-language=c>`_ that routes libcamera data for \n+further processing in a gstreamer pipeline.\n+\n+libcamera Architecture\n+----------------------\n+\n+The libcamera library exposes one unified API, but behind that is built from \n+re-usable components that provide hardware specific support and configuration \n+with a device agnostic API.\n+\n+Camera Manager\n+~~~~~~~~~~~~~~\n+\n+It enumerates cameras at runtime and instantiates a `Pipeline Handler`_ to manage each Camera \n+device that libcamera supports. The Camera Manager supports hotplug detection\n+and notification events when supported by the underlying kernel devices.\n+\n+There is only ever one instance of the Camera Manager running per application. Each application's \n+instance of the Camera Manager ensures that only a single application can take control of a camera \n+device at once.\n+\n+`Read the Camera Manager API documentation for more details <http://libcamera.org/api-html/classlibcamera_1_1CameraManager.html>`_\n+\n+Camera Device\n+~~~~~~~~~~~~~\n+\n+The camera class represents a single item of camera hardware that is capable of producing one or more image streams, and provides the API to interact with the underlying device. \n+\n+If a system has multiple instances of the same hardware attached, each has it's own instance of the camera class.\n+\n+The API exposes full control of the device to upper layers of libcamera through the public API, \n+making it the highest level object libcamera exposes, and the object that all other API operations \n+interact with from configuration to capture.\n+\n+`Read the Camera API documentation for more details <http://libcamera.org/api-html/classlibcamera_1_1Camera.html>`_\n+\n+Frame controls\n+^^^^^^^^^^^^^^\n+\n+Depending on the capabilities of a camera device and the hardware it is connected \n+to, libcamera supports controlling capture parameters for each stream on a per-frame basis.\n+\n+These controls include auto exposure, gain, brightness, contrast, lux, white balance, color \n+temperature, and saturation.\n+\n+`Read the Control API documentation for more details <http://libcamera.org/api-html/classlibcamera_1_1Control.html>`_\n+\n+Pipeline Handler\n+~~~~~~~~~~~~~~~~\n+\n+Pipeline handlers are the abstraction layer for platform-specific hardware configuration. They \n+access and control hardware through the V4L2 and Media Controller kernel interfaces, and implement \n+an internal API to control the ISP and Capture components of a pipeline directly.\n+\n+Pipeline handlers' create' Camera device instances based on the devices they detect and support on \n+the running system.\n+\n+A pipeline handler manages most aspects of interacting with a camera device including:\n+\n+- frame controls\n+- pipelines and stream configuration\n+- the data the camera produces, and the buffers it needs to hold the data\n+- granting access to camera data\n+\n+Pipeline handlers form part of the libcamera codebase, and developers need to implement them for \n+complex hardware with an ISP that requires device-specific configurations.\n+\n+`Read the PipelineHandler API documentation for more details <http://libcamera.org/api-html/classlibcamera_1_1PipelineHandler.html>`_\n+\n+Image Processing Algorithms\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+An image processing algorithm (IPA) component is a private API that implements \n+3A (Auto-Exposure, Auto-White Balance, and Auto-Focus) algorithms.\n+\n+Each supported camera device has its own IPA component that runs on the CPU and \n+interacts with the kernel camera devices to control hardware image processing \n+based on the parameters supplied by upper layers, and helps maintain state, \n+closing the control loop of the ISP.\n+\n+An IPA component can be part of the libcamera code base, in which case the same \n+license covers it, or provided externally as an open-source or closed-source component.\n+\n+The component is sandboxed and can only interact with libcamera through specifically \n+marked APIs. A component has no direct access to kernel camera devices, and dmabuf \n+instances explicitly passed to the component control its access to image data and \n+metadata. The component should run in a process separate from the main libcamera \n+process, and has a restricted view of the system, including no access to networking APIs\n+and limited access to file systems.\n+\n+While libcamera requires sandboxing, the implementation is platform-specific, and handled by \n+platform-specific plugins.\n+\n+To support this security and sandboxing model, libcamera provides an IPC mechanism \n+for an IPA and Pipeline Handler to communicate, but developers need to create the \n+API themselves. Platform vendors can also use any other IPC mechanism that supports \n+passing file descriptors.\n+\n+3A and Image Enhancement Algorithms\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Camera devices should implement auto exposure, auto gain, and auto white balance, \n+and those that include a focus lens should also implement autofocus.\n+\n+Device vendors implement the control methods required to control these algorithms in hardware or \n+firmware outside of libcamera, or in an IPA component.\n+\n+.. TODO: Add link to guide when completed\n+\n+Helpers and Support Classes\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+To help developers create device-specific pipeline handlers and image processing \n+algorithms, libcamera provides helpers and support classes that sit on top of the \n+Media Controller and V4L2 APIs.\n+\n+Glossary\n+--------\n+\n+- **ISP**: Image Signal Processor.\n+- **Media controller API**: `The Linux Kernel API <https://www.kernel.org/doc/html/latest/admin-guide/media/index.html>`_ that handles audio and video input and output.\n+- **V4L2**: `Video for Linux API version 2 <https://www.linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/v4l2.html>`_. Device drivers and API for video capture on Linux.\n+- **UVC camera**: `USB Video Class <https://en.wikipedia.org/wiki/USB_video_device_class>`_ that describes devices capable of streaming video over the UVC protocol.\n+- **3A**: Common algorithms used by camera devices for auto-exposure, auto-white balance and auto-focus).\n+- **IPC**: `Inter-process communications based protocol <https://en.wikipedia.org/wiki/Inter-process_communication>`_.\n", "prefixes": [ "libcamera-devel", "v3" ] }