[{"id":12085,"web_url":"https://patchwork.libcamera.org/comment/12085/","msgid":"<20200820154634.GT6593@pendragon.ideasonboard.com>","date":"2020-08-20T15:46:34","subject":"Re: [libcamera-devel] [PATCH v4 3/3] Documentation: Guides:\n\tApplication Writers Guide","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nThank you for the patch.\n\nOn Thu, Aug 20, 2020 at 02:47:51PM +0100, Kieran Bingham wrote:\n> From: Chris Chinchilla \n> \n> Provide a tutorial and walk through guide for writing an applications\n> with libcamera.\n> \n> Signed-off-by: Chris Chinchilla \n> [Reflow/Rework, update to mainline API]\n> Signed-off-by: Jacopo Mondi \n> [Further reworks and review]\n> Signed-off-by: Kieran Bingham \n> ---\n> .../guides/application-developer.rst | 644 ++++++++++++++++++\n> Documentation/index.rst | 1 +\n> Documentation/meson.build | 1 +\n> 3 files changed, 646 insertions(+)\n> create mode 100644 Documentation/guides/application-developer.rst\n> \n> diff --git a/Documentation/guides/application-developer.rst b/Documentation/guides/application-developer.rst\n> new file mode 100644\n> index 000000000000..14ab21550cbe\n> --- /dev/null\n> +++ b/Documentation/guides/application-developer.rst\n> @@ -0,0 +1,644 @@\n> +.. SPDX-License-Identifier: CC-BY-SA-4.0\n> +\n> +Using libcamera in a C++ application\n> +====================================\n> +\n> +This tutorial shows how to create a C++ application that uses libcamera to\n> +interface with a camera on a system, capture frames from it for 3 seconds, and\n> +write metadata about the frames to standard out.\n> +\n> +.. TODO: Check how much of the example code runs before camera start etc?\n> +\n> +Application skeleton\n> +--------------------\n> +\n> +Most of the code in this tutorial runs in the ``int main()`` function\n> +with a separate global function to handle events. The two functions need\n> +to share data, which are stored in global variables for simplicity. A\n> +production-ready application would organize the various objects created\n> +in classes, and the event handler would be a class member function to\n> +provide context data without requiring global variables.\n> +\n> +Use the following code snippets as the initial application skeleton.\n> +It already lists all the necessary includes directives and instructs the\n> +compiler to use the libcamera namespace, which gives access to the libcamera\n> +defined names and types without the need of prefixing them.\n> +\n> +.. code:: cpp\n> +\n> + #include \n> + #include \n> + #include \n> +\n> + #include \n> +\n> + using namespace libcamera;\n> +\n> + int main()\n> + {\n> + // Code to follow\n> +\n> + return 0;\n> + }\n> +\n> +Camera Manager\n> +--------------\n> +\n> +Every libcamera-based application needs an instance of a `CameraManager`_ that\n> +runs for the life of the application. When the Camera Manager starts, it\n> +enumerates all the cameras detected in the system. Behind the scenes, libcamera\n> +abstracts and manages the complex pipelines that kernel drivers expose through\n> +the `Linux Media Controller`_ and `Video for Linux`_ (V4L2) APIs, meaning that\n> +an application doesn’t need to handle device or driver specific details.\n> +\n> +.. _CameraManager: http://libcamera.org/api-html/classlibcamera_1_1CameraManager.html\n> +.. _Linux Media Controller: https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-controller-intro.html\n> +.. _Video for Linux: https://www.linuxtv.org/docs.php\n> +\n> +Before the ``int main()`` function, create a global shared pointer\n> +variable for the camera to support the event call back later:\n> +\n> +.. code:: cpp\n> +\n> + std::shared_ptr camera;\n> +\n> +Create a Camera Manager instance at the beginning of the main function, and then\n> +start it. An application should only create a single Camera Manager instance.\n> +\n> +.. code:: cpp\n> +\n> + CameraManager *cm = new CameraManager();\n> + cm->start();\n> +\n> +During the application initialization, the Camera Manager is started to\n> +enumerate all the supported devices and create cameras that the application can\n> +interact with.\n> +\n> +Once the camera manager is started, we can use it to iterate the available\n> +cameras in the system:\n> +\n> +.. code:: cpp\n> +\n> + for (auto const &camera : cm->cameras())\n> + std::cout << camera->id() << std::endl;\n> +\n> +Printing the camera id lists the machine-readable unique identifiers, so for\n> +example, the output on a Linux machine with a connected USB webcam is\n> +``\\_SB_.PCI0.XHC_.RHUB.HS08-8:1.0-5986:2115``.\n> +\n> +What libcamera considers a camera\n> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +The libcamera library considers any unique source of video frames, which usually\n> +correspond to a camera sensor, as a single camera device. Camera devices expose\n> +streams, which are obtained by processing data from the single image source and\n> +all share some basic properties such as the frame duration and the image\n> +exposure time, as they only depend by the image source configuration.\n> +\n> +Applications select one or multiple Camera devices they wish to operate on, and\n> +require frames from at least one of their Streams.\n> +\n> +Create and acquire a camera\n> +---------------------------\n> +\n> +This example application uses a single camera (the first enumerated one) that\n> +the Camera Manager reports as available to applications.\n> +\n> +Camera devices are stored by the CameraManager in a list accessible by index, or\n> +can be retrieved by name through the ``CameraManager::get()`` function. The\n> +code below retrieves the name of the first available camera and gets the camera\n> +by name from the Camera Manager.\n> +\n> +.. code:: cpp\n> +\n> + std::string cameraId = cm->cameras()[0]->id();\n> + camera = cm->get(cameraId);\n> +\n> + /*\n> + * Note that is equivalent to:\n> + * camera = cm->cameras()[0];\n> + */\n> +\n> +Once a camera has been selected an application needs to acquire an exclusive\n> +lock to it so no other application can use it.\n> +\n> +.. code:: cpp\n> +\n> + camera->acquire();\n> +\n> +Configure the camera\n> +--------------------\n> +\n> +Before the application can do anything with the camera, it needs to configure\n> +the image format and sizes of the streams it wants to capture frames from.\n> +\n> +Stream configurations are represented by instances of the\n> +``StreamConfiguration`` class, which are grouped together in a\n> +``CameraConfiguration`` object. Before an application can start setting its\n> +desired configuration, a ``CameraConfiguration`` instance needs to be generated\n> +from the ``Camera`` device using the ``Camera::generateConfiguration()``\n> +function.\n> +\n> +The libcamera library uses the ``StreamRole`` enumeration to define predefined\n> +ways an application intends to use a camera. The\n> +``Camera::generateConfiguration()`` function accepts a list of desired roles and\n> +generates a ``CameraConfiguration`` with the best stream parameters\n> +configuration for each of the requested roles. If the camera can handle the\n> +requested roles, it returns an initialized ``CameraConfiguration`` and a null\n> +pointer if it can't.\n> +\n> +It is possible for applications to generate an empty ``CameraConfiguration``\n> +instance by not providing any role. The desired configuration will have to be\n> +filled-in manually and manually validated.\n> +\n> +In the example application, create a new configuration variable and use the\n> +``Camera::generateConfiguration`` function to produce a ``CameraConfiguration``\n> +for the single ``StreamRole::Viewfinder`` role.\n> +\n> +.. code:: cpp\n> +\n> + std::unique_ptr config = camera->generateConfiguration( { StreamRole::Viewfinder } );\n> +\n> +The generated ``CameraConfiguration`` has a ``StreamConfiguration`` instance for\n> +each ``StreamRole`` the application requested. Each of these has a default size\n> +and format that the camera assigned, and a list of supported pixel formats and\n> +sizes.\n> +\n> +The code below accesses the first and only ``StreamConfiguration`` item in the\n> +``CameraConfiguration`` and outputs its parameters to standard output.\n> +\n> +.. code:: cpp\n> +\n> + StreamConfiguration &streamConfig = config->at(0);\n> + std::cout << \"Default viewfinder configuration is: \" << streamConfig.toString() << std::endl;\n> +\n> +This is expected to output something like:\n> +\n> + ``Default viewfinder configuration is: 1280x720-MJPEG``\n> +\n> +Change and validate the configuration\n> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +With an initialized ``CameraConfiguration``, an application can make changes to\n> +the parameters it contains, for example, to change the width and height, use the\n> +following code:\n> +\n> +.. code:: cpp\n> +\n> + streamConfig.size.width = 640;\n> + streamConfig.size.height = 480;\n> +\n> +If an application changes any parameters, it must validate the configuration\n> +before applying it to the camera using the ``CameraConfiguration::validate()``\n> +function. If the new values are not supported by the ``Camera`` device, the\n> +validation process adjusts the parameters to what it considers to be the closest\n> +supported values.\n> +\n> +The ``validate`` method returns a `Status`_ which applications shall check to\n> +see if the Pipeline Handler adjusted the configuration.\n> +\n> +.. _Status: http://libcamera.org/api-html/classlibcamera_1_1CameraConfiguration.html#a64163f21db2fe1ce0a6af5a6f6847744\n> +\n> +For example, the code above set the width and height to 640x480, but if the\n> +camera cannot produce an image that large, it might adjust the configuration to\n> +the supported size of 320x240 and return ``Adjusted`` as validation status\n> +result.\n> +\n> +If the configuration to validate cannot be adjusted to a set of supported\n> +values, the validation procedure fails and returns the ``Invalid`` status.\n> +\n> +For this example application, the code below prints the adjusted values to\n> +standard out.\n> +\n> +.. code:: cpp\n> +\n> + config->validate();\n> + std::cout << \"Validated viewfinder configuration is: \" << streamConfig.toString() << std::endl;\n> +\n> +For example, the output might be something like\n> +\n> + ``Validated viewfinder configuration is: 320x240-MJPEG``\n> +\n> +A validated ``CameraConfiguration`` can bet given to the ``Camera`` device to be\n> +applied to the system.\n> +\n> +.. code:: cpp\n> +\n> + camera->configure(config.get());\n> +\n> +If an application doesn’t first validate the configuration before calling\n> +``Camera::configure()``, there’s a chance that calling the function can fail, if\n> +the given configuration would have to be adjusted.\n> +\n> +Allocate FrameBuffers\n> +---------------------\n> +\n> +An application needs to reserve the memory that libcamera can write incoming\n> +frames and data to, and that the application can then read. The libcamera\n> +library uses ``FrameBuffer`` instances to represent memory buffers allocated in\n> +memory. An application should reserve enough memory for the frame size the\n> +streams need based on the configured image sizes and formats.\n> +\n> +The libcamera library consumes buffers provided by applications as\n> +``FrameBuffer`` instances, which makes libcamera a consumer of buffers exported\n> +by other devices (such as displays or video encoders), or allocated from an\n> +external allocator (such as ION on Android).\n> +\n> +In some situations, applications do not have any means to allocate or get hold\n> +of suitable buffers, for instance, when no other device is involved, or on Linux\n> +platforms that lack a centralized allocator. The ``FrameBufferAllocator`` class\n> +provides a buffer allocator an application can use in these situations.\n> +\n> +An application doesn’t have to use the default ``FrameBufferAllocator`` that\n> +libcamera provides. It can instead allocate memory manually and pass the buffers\n> +in ``Request``\\s (read more about ``Request`` in `the frame capture section\n> +<#frame-capture>`_ of this guide). The example in this guide covers using the\n> +``FrameBufferAllocator`` that libcamera provides.\n> +\n> +Using the libcamera ``FrameBufferAllocator``\n> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +Applications create a ``FrameBufferAllocator`` for a Camera and use it\n> +to allocate buffers for streams of a ``CameraConfiguration`` with the\n> +``allocate()`` function.\n> +\n> +The list of allocated buffers can be retrieved using the ``Stream`` instance\n> +as the parameter of the ``FrameBufferAllocator::buffers()`` function.\n> +\n> +.. code:: cpp\n> +\n> + FrameBufferAllocator *allocator = new FrameBufferAllocator(camera);\n> +\n> + for (StreamConfiguration &cfg : *config) {\n> + int ret = allocator->allocate(cfg.stream());\n> + if (ret < 0) {\n> + std::cerr << \"Can't allocate buffers\" << std::endl;\n> + return -ENOMEM;\n> + }\n> +\n> + unsigned int allocated = allocator->buffers(cfg.stream()).size();\n> + std::cout << \"Allocated \" << allocated << \" buffers for stream\" << std::endl;\n> + }\n> +\n> +Frame Capture\n> +~~~~~~~~~~~~~\n> +\n> +The libcamera library implements a streaming model based on per-frame requests.\n> +For each frame an application wants to capture it must queue a request for it to\n> +the camera. With libcamera, a ``Request`` is at least one ``Stream`` associated\n> +with a ``FrameBuffer`` representing the memory location where frames have to be\n> +stored.\n> +\n> +First, by using the ``Stream`` instance associated to each\n> +``StreamConfiguration``, retrieve the list of ``FrameBuffer``\\s created for it\n> +using the frame allocator. Then create a vector of requests to be submitted to\n> +the camera.\n> +\n> +.. code:: cpp\n> +\n> + Stream *stream = streamConfig.stream();\n> + const std::vector> &buffers = allocator->buffers(stream);\n> + std::vector requests;\n> +\n> +Proceed to fill the request vector by creating ``Request`` instances from the\n> +camera device, and associate a buffer for each of them for the ``Stream``.\n> +\n> +.. code:: cpp\n> +\n> + for (unsigned int i = 0; i < buffers.size(); ++i) {\n> + Request *request = camera->createRequest();\n> + if (!request)\n> + {\n> + std::cerr << \"Can't create request\" << std::endl;\n> + return -ENOMEM;\n> + }\n> +\n> + const std::unique_ptr &buffer = buffers[i];\n> + int ret = request->addBuffer(stream, buffer.get());\n> + if (ret < 0)\n> + {\n> + std::cerr << \"Can't set buffer for request\"\n> + << std::endl;\n> + return ret;\n> + }\n> +\n> + requests.push_back(request);\n> + }\n> +\n> +.. TODO: Controls\n> +\n> +.. TODO: A request can also have controls or parameters that you can apply to the image.\n> +\n> +Event handling and callbacks\n> +----------------------------\n> +\n> +The libcamera library uses the concept of `signals and slots` (similar to `Qt\n> +Signals and Slots`_) to connect events with callbacks to handle them.\n> +\n> +.. _signals and slots: http://libcamera.org/api-html/classlibcamera_1_1Signal.html#details\n> +.. _Qt Signals and Slots: https://doc.qt.io/qt-5/signalsandslots.html\n> +\n> +The ``Camera`` device emits two signals that applications can connect to in\n> +order to execute callbacks on frame completion events.\n> +\n> +The ``Camera::bufferCompleted`` signal notifies applications that a buffer with\n> +image data is available. Receiving notifications about the single buffer\n> +completion event allows applications to implement partial request completion\n> +support, and to inspect the buffer content before the request it is part of has\n> +fully completed.\n> +\n> +The ``Camera::requestCompleted`` signal notifies applications that a request\n> +has completed, which means all the buffers the request contains have now\n> +completed. Request completion notifications are always emitted in the same order\n> +as the requests have been queued to the camera.\n> +\n> +To receive the signals emission notifications, connect a slot function to the\n> +signal to handle it in the application code.\n> +\n> +.. code:: cpp\n> +\n> + camera->requestCompleted.connect(requestComplete);\n> +\n> +For this example application, only the ``Camera::requestCompleted`` signal gets\n> +handled and the matching ``requestComplete`` slot method outputs information\n> +about the FrameBuffer to standard output. This callback is typically where an\n> +application accesses the image data from the camera and does something with it.\n> +\n> +Signals operate in the libcamera ``CameraManager`` thread context, so it is\n> +important not to block the thread for a long time, as this blocks internal\n> +processing of the camera pipelines, and can affect realtime performance.\n> +\n> +Handle request completion events\n> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n> +\n> +Create the ``requestComplete`` function by matching the slot signature:\n> +\n> +.. code:: cpp\n> +\n> + static void requestComplete(Request *request)\n> + {\n> + // Code to follow\n> + }\n> +\n> +Request completion events can be emitted for requests which have been canceled,\n> +for example, by unexpected application shutdown. To avoid an application\n> +processing invalid image data, it’s worth checking that the request has\n> +completed successfully. The list of request completion statuses is available in\n> +the `Request::Status`_ class enum documentation.\n> +\n> +.. _Request::Status: https://www.libcamera.org/api-html/classlibcamera_1_1Request.html#a2209ba8d51af8167b25f6e3e94d5c45b\n> +\n> +.. code:: cpp\n> +\n> + if (request->status() == Request::RequestCancelled)\n> + return;\n> +\n> +If the ``Request`` has completed successfully, applications can access the\n> +completed buffers using the ``Request::buffers()`` function, which returns a map\n> +of ``FrameBuffer`` instances associated with the ``Stream`` that produced the\n> +images.\n> +\n> +.. code:: cpp\n> +\n> + const std::map &buffers = request->buffers();\n> +\n> +Iterating through the map allows applications to inspect each completed buffer\n> +in this request, and access the metadata associated to each frame.\n> +\n> +The metadata buffer contains information such the capture status, a timestamp,\n> +and the bytes used, as described in the `FrameMetadata`_ documentation.\n> +\n> +.. _FrameMetaData: http://libcamera.org/api-html/structlibcamera_1_1FrameMetadata.html\n> +\n> +.. code:: cpp\n> +\n> + for (auto bufferPair : buffers) {\n> + FrameBuffer *buffer = bufferPair.second;\n> + const FrameMetadata &metadata = buffer->metadata();\n> + }\n> +\n> +For this example application, inside the ``for`` loop from above, we ca print\n> +the Frame sequence number and details of the planes.\n> +\n> +.. code:: cpp\n> +\n> + std::cout << \" seq: \" << std::setw(6) << std::setfill('0') << metadata.sequence << \" bytesused: \";\n> +\n> + unsigned int nplane = 0;\n> + for (const FrameMetadata::Plane &plane : metadata.planes)\n> + {\n> + std::cout << plane.bytesused;\n> + if (++nplane < metadata.planes.size()) std::cout << \"/\";\n> + }\n> +\n> + std::cout << std::endl;\n> +\n> +The expected output shows each monotonically increasing frame sequence number\n> +and the bytes used by planes.\n> +\n> +.. code:: text\n> +\n> + seq: 000000 bytesused: 1843200\n> + seq: 000002 bytesused: 1843200\n> + seq: 000004 bytesused: 1843200\n> + seq: 000006 bytesused: 1843200\n> + seq: 000008 bytesused: 1843200\n> + seq: 000010 bytesused: 1843200\n> + seq: 000012 bytesused: 1843200\n> + seq: 000014 bytesused: 1843200\n> + seq: 000016 bytesused: 1843200\n> + seq: 000018 bytesused: 1843200\n> + seq: 000020 bytesused: 1843200\n> + seq: 000022 bytesused: 1843200\n> + seq: 000024 bytesused: 1843200\n> + seq: 000026 bytesused: 1843200\n> + seq: 000028 bytesused: 1843200\n> + seq: 000030 bytesused: 1843200\n> + seq: 000032 bytesused: 1843200\n> + seq: 000034 bytesused: 1843200\n> + seq: 000036 bytesused: 1843200\n> + seq: 000038 bytesused: 1843200\n> + seq: 000040 bytesused: 1843200\n> + seq: 000042 bytesused: 1843200\n> +\n> +A completed buffer contains of course image data which can be accessed through\n> +the per-plane dma-buf file descriptor transported by the ``FrameBuffer``\n> +instance. An example of how to write image data to disk is available in the\n> +`BufferWriter class`_ which is a part of the ``cam`` utility application in the\n> +libcamera repository.\n> +\n> +.. _BufferWriter class: https://git.linuxtv.org/libcamera.git/tree/src/cam/buffer_writer.cpp\n> +\n> +With the handling of this request completed, it is possible to re-use the\n> +buffers by adding them to a new ``Request`` instance with their matching\n> +streams, and finally, queue the new capture request to the camera device:\n> +\n> +.. code:: cpp\n> +\n> + request = camera->createRequest();\n> + if (!request)\n> + {\n> + std::cerr << \"Can't create request\" << std::endl;\n> + return;\n> + }\n> +\n> + for (auto it = buffers.begin(); it != buffers.end(); ++it)\n> + {\n> + Stream *stream = it->first;\n> + FrameBuffer *buffer = it->second;\n> +\n> + request->addBuffer(stream, buffer);\n> + }\n> +\n> + camera->queueRequest(request);\n> +\n> +Request queueing\n> +----------------\n> +\n> +The ``Camera`` device is now ready to receive frame capture requests and\n> +actually start delivering frames. In order to prepare for that, an application\n> +needs to first start the camera, and queue requests to it for them to be\n> +processed.\n> +\n> +In the main() function, just after having connected the\n> +``Camera::requestCompleted`` signal to the callback handler, start the camera\n> +and queue all the previously created requests.\n> +\n> +.. code:: cpp\n> +\n> + camera->start();\n> + for (Request *request : requests)\n> + camera->queueRequest(request);\n> +\n> +Start an event loop\n> +~~~~~~~~~~~~~~~~~~~\n> +\n> +The libcamera library needs an event loop to monitor and dispatch events\n> +generated by the video devices part of the capture pipeline. Libcamera provides\n\ns/Libcamera/libcamera/\n\n> +its own ``EventDispatcher`` class (inspired by the `Qt event system`_) to\n> +process and deliver events generated by ``EventNotifiers``.\n> +\n> +.. _Qt event system: https://doc.qt.io/qt-5/eventsandfilters.html\n> +\n> +The libcamera library implements this by creating instances of the\n> +``EventNotifier`` class, which models a file descriptor event source registered\n> +to an ``EventDispatcher``. Whenever the ``EventDispatcher`` detects an event on\n> +a notifier it is monitoring, it emits the notifier's\n> +``EventNotifier::activated`` signal. The libcamera components connect to the\n> +notifiers' signals and emit application visible events, such as the\n> +``Camera::bufferReady`` and ``Camera::requestCompleted`` signals.\n> +\n> +The code below retrieves a reference to the system-wide event dispatcher and for\n> +the a fixed duration of 3 seconds, processes all the events detected in the\n> +system.\n> +\n> +.. code:: cpp\n> +\n> + EventDispatcher *dispatcher = cm->eventDispatcher();\n> + Timer timer;\n> + timer.start(3000);\n> + while (timer.isRunning())\n> + dispatcher->processEvents();\n> +\n> +Clean up and stop the application\n> +---------------------------------\n> +\n> +The application is now finished with the camera and the resources the camera\n> +uses, so needs to do the following:\n> +\n> +- stop the camera\n> +- free the buffers in the FrameBufferAllocator and delete it\n> +- release the lock on the camera and reset the pointer to it\n> +- stop the camera manager\n> +\n> +.. code:: cpp\n> +\n> + camera->stop();\n> + allocator->free(stream);\n> + delete allocator;\n> + camera->release();\n> + camera.reset();\n> + cm->stop();\n> +\n> + return 0;\n> +\n> +Build and run instructions\n> +--------------------------\n> +\n> +To build the application, use the `Meson build system`_ which is also the\n> +official build system of the libcamera library.\n> +\n> +Make sure both ``meson`` and ``libcamera`` are installed in your system. Please\n> +refer to your distribution documentation to install meson and install the most\n> +recent version of libcamera from the git repository at `Linux TV`_. You would\n> +also need to install the ``pkg-config`` tool to correctly identify the\n> +libcamera.so object install location in the system.\n> +\n> +.. _Meson build system: https://mesonbuild.com/\n> +.. _Linux TV: https://git.linuxtv.org/libcamera.git/\n> +\n> +Dependencies\n> +~~~~~~~~~~~~\n> +\n> +The test application presented here depends on the libcamera library to be\n> +available in a path that meson can identify. The libcamera install procedure\n> +performed using the ``ninja install`` command may by default deploy the\n> +libcamera components in the ``/usr/local/lib`` path, or a package manager may\n> +install it to ``/usr/lib`` depending on your distribution. If meson is unable to\n> +find the location of the libcamera installation, you may need to instruct meson\n> +to look into a specific path when searching for ``libcamera.so`` by setting the\n> +``PKG_CONFIG_PATH`` environment variable to the right location.\n> +\n> +Adjust the following command to use the ``pkgconfig`` directory where libcamera\n> +has been installed in your system.\n> +\n> +.. code:: shell\n> +\n> + export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/\n> +\n> +Verify that ``pkg-config`` can identify the ``camera`` library with\n> +\n> +.. code:: shell\n> +\n> + $ pkg-config --libs --cflags camera\n> + -I/usr/local/include/libcamera -L/usr/local/lib -lcamera\n> +\n> +``meson`` can alternatively use ``cmake`` to locate packages, please refer to\n> +the ``meson`` documentation if you prefer to use it in place of ``pkgconfig``\n> +\n> +Build file\n> +~~~~~~~~~~\n> +\n> +With the dependencies correctly identified, prepare a ``meson.build`` build file\n> +to be placed in the same directory where the application lives. You can\n> +name your application as you like, but be sure to update the following snippet\n> +accordingly. In this example, the application file has been named\n> +``simple-cam.cpp``.\n> +\n> +.. code::\n> +\n> + project('simple-cam', 'cpp')\n> +\n> + simpler_cam = executable('simple-cam',\n> + 'simple-cam.cpp',\n> + dependencies: dependency('camera', required : true))\n> +\n> +The ``dependencies`` line instructs meson to ask ``pkgconfig`` (or ``cmake``) to\n> +locate the ``camera`` library, (libcamera without the lib prefix) which the test\n> +application will be dynamically linked against.\n> +\n> +With the build file in place, compile and run the application with:\n> +\n> +.. code:: shell\n> +\n> + $ meson build\n> + $ cd build\n> + $ ninja\n> + $ ./simple-cam\n> +\n> +It is possible to increase the library debug output by using environment\n> +variables which control the library log filtering system:\n> +\n> +.. code:: shell\n> +\n> + $ LIBCAMERA_LOG_LEVELS=0 ./simple-cam\n> diff --git a/Documentation/index.rst b/Documentation/index.rst\n> index fb391d2b6ebf..68b7ac06c506 100644\n> --- a/Documentation/index.rst\n> +++ b/Documentation/index.rst\n> @@ -15,3 +15,4 @@\n> \n> Developer Guide \n> Pipeline Handler Writers Guide \n> + Application Writers Guide \n\nSame comment, Writers or Writer's (including in the subject line, same\nfor patch 2/3) ?\n\nI would move this before the pipeline handler guide. Same in\nmeson.build.\n\nOther improvements can go on top.\n\nAcked-by: Laurent Pinchart \n\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 9f6c67071da9..6bff2e4a4912 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -54,6 +54,7 @@ if sphinx.found()\n> 'index.rst',\n> 'guides/introduction.rst',\n> 'guides/pipeline-handler.rst',\n> + 'guides/application-developer.rst',\n> ]\n> \n> release = 'release=v' + libcamera_git_version","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 35B79BD87C\n\tfor ;\n\tThu, 20 Aug 2020 15:46:55 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C4A6561B4F;\n\tThu, 20 Aug 2020 17:46:54 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DAE2760381\n\tfor ;\n\tThu, 20 Aug 2020 17:46:52 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 41AF023D;\n\tThu, 20 Aug 2020 17:46:52 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"bSsaHZRK\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597938412;\n\tbh=+IAzTb1bx8S3Dla2vkAA40v8F8szh/Vyi7aKN5hNU/o=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=bSsaHZRKhtMI+BLLSLbTWf4uvNzDkPp4zlFb/DJL7cbWHCgcEuCVpkO5qj54MP+11\n\tTf6tZkhfg506THGTNfaAPTRxFSmMnAFSAPrNHAaUEIM0qBywh3mTG7q3Bwm7d8a6r5\n\tNnx9V3mzrmwzDoOowbuUecgQhj6YvfevkzhYpP30=","Date":"Thu, 20 Aug 2020 18:46:34 +0300","From":"Laurent Pinchart ","To":"Kieran Bingham ","Message-ID":"<20200820154634.GT6593@pendragon.ideasonboard.com>","References":"<20200820134751.278033-1-kieran.bingham@ideasonboard.com>\n\t<20200820134751.278033-4-kieran.bingham@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200820134751.278033-4-kieran.bingham@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v4 3/3] Documentation: Guides:\n\tApplication Writers Guide","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Chris Ward ,\n\tlibcamera devel ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12086,"web_url":"https://patchwork.libcamera.org/comment/12086/","msgid":"<92f912d1-42be-f3eb-9bc7-7d3ec1c2ed13@ideasonboard.com>","date":"2020-08-20T15:55:12","subject":"Re: [libcamera-devel] [PATCH v4 3/3] Documentation: Guides:\n\tApplication Writers Guide","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Laurent,\n\nOn 20/08/2020 16:46, Laurent Pinchart wrote:\n> Hi Kieran,\n> \n> Thank you for the patch.\n> \n> On Thu, Aug 20, 2020 at 02:47:51PM +0100, Kieran Bingham wrote:\n>> From: Chris Chinchilla \n>>\n>> Provide a tutorial and walk through guide for writing an applications\n>> with libcamera.\n>>\n>> Signed-off-by: Chris Chinchilla \n>> [Reflow/Rework, update to mainline API]\n>> Signed-off-by: Jacopo Mondi \n>> [Further reworks and review]\n>> Signed-off-by: Kieran Bingham \n>> ---\n>> .../guides/application-developer.rst | 644 ++++++++++++++++++\n>> Documentation/index.rst | 1 +\n>> Documentation/meson.build | 1 +\n>> 3 files changed, 646 insertions(+)\n>> create mode 100644 Documentation/guides/application-developer.rst\n>>\n>> diff --git a/Documentation/guides/application-developer.rst b/Documentation/guides/application-developer.rst\n>> new file mode 100644\n>> index 000000000000..14ab21550cbe\n>> --- /dev/null\n>> +++ b/Documentation/guides/application-developer.rst\n>> @@ -0,0 +1,644 @@\n>> +.. SPDX-License-Identifier: CC-BY-SA-4.0\n>> +\n>> +Using libcamera in a C++ application\n>> +====================================\n>> +\n>> +This tutorial shows how to create a C++ application that uses libcamera to\n>> +interface with a camera on a system, capture frames from it for 3 seconds, and\n>> +write metadata about the frames to standard out.\n>> +\n>> +.. TODO: Check how much of the example code runs before camera start etc?\n>> +\n>> +Application skeleton\n>> +--------------------\n>> +\n>> +Most of the code in this tutorial runs in the ``int main()`` function\n>> +with a separate global function to handle events. The two functions need\n>> +to share data, which are stored in global variables for simplicity. A\n>> +production-ready application would organize the various objects created\n>> +in classes, and the event handler would be a class member function to\n>> +provide context data without requiring global variables.\n>> +\n>> +Use the following code snippets as the initial application skeleton.\n>> +It already lists all the necessary includes directives and instructs the\n>> +compiler to use the libcamera namespace, which gives access to the libcamera\n>> +defined names and types without the need of prefixing them.\n>> +\n>> +.. code:: cpp\n>> +\n>> + #include \n>> + #include \n>> + #include \n>> +\n>> + #include \n>> +\n>> + using namespace libcamera;\n>> +\n>> + int main()\n>> + {\n>> + // Code to follow\n>> +\n>> + return 0;\n>> + }\n>> +\n>> +Camera Manager\n>> +--------------\n>> +\n>> +Every libcamera-based application needs an instance of a `CameraManager`_ that\n>> +runs for the life of the application. When the Camera Manager starts, it\n>> +enumerates all the cameras detected in the system. Behind the scenes, libcamera\n>> +abstracts and manages the complex pipelines that kernel drivers expose through\n>> +the `Linux Media Controller`_ and `Video for Linux`_ (V4L2) APIs, meaning that\n>> +an application doesn’t need to handle device or driver specific details.\n>> +\n>> +.. _CameraManager: http://libcamera.org/api-html/classlibcamera_1_1CameraManager.html\n>> +.. _Linux Media Controller: https://www.kernel.org/doc/html/latest/media/uapi/mediactl/media-controller-intro.html\n>> +.. _Video for Linux: https://www.linuxtv.org/docs.php\n>> +\n>> +Before the ``int main()`` function, create a global shared pointer\n>> +variable for the camera to support the event call back later:\n>> +\n>> +.. code:: cpp\n>> +\n>> + std::shared_ptr camera;\n>> +\n>> +Create a Camera Manager instance at the beginning of the main function, and then\n>> +start it. An application should only create a single Camera Manager instance.\n>> +\n>> +.. code:: cpp\n>> +\n>> + CameraManager *cm = new CameraManager();\n>> + cm->start();\n>> +\n>> +During the application initialization, the Camera Manager is started to\n>> +enumerate all the supported devices and create cameras that the application can\n>> +interact with.\n>> +\n>> +Once the camera manager is started, we can use it to iterate the available\n>> +cameras in the system:\n>> +\n>> +.. code:: cpp\n>> +\n>> + for (auto const &camera : cm->cameras())\n>> + std::cout << camera->id() << std::endl;\n>> +\n>> +Printing the camera id lists the machine-readable unique identifiers, so for\n>> +example, the output on a Linux machine with a connected USB webcam is\n>> +``\\_SB_.PCI0.XHC_.RHUB.HS08-8:1.0-5986:2115``.\n>> +\n>> +What libcamera considers a camera\n>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n>> +\n>> +The libcamera library considers any unique source of video frames, which usually\n>> +correspond to a camera sensor, as a single camera device. Camera devices expose\n>> +streams, which are obtained by processing data from the single image source and\n>> +all share some basic properties such as the frame duration and the image\n>> +exposure time, as they only depend by the image source configuration.\n>> +\n>> +Applications select one or multiple Camera devices they wish to operate on, and\n>> +require frames from at least one of their Streams.\n>> +\n>> +Create and acquire a camera\n>> +---------------------------\n>> +\n>> +This example application uses a single camera (the first enumerated one) that\n>> +the Camera Manager reports as available to applications.\n>> +\n>> +Camera devices are stored by the CameraManager in a list accessible by index, or\n>> +can be retrieved by name through the ``CameraManager::get()`` function. The\n>> +code below retrieves the name of the first available camera and gets the camera\n>> +by name from the Camera Manager.\n>> +\n>> +.. code:: cpp\n>> +\n>> + std::string cameraId = cm->cameras()[0]->id();\n>> + camera = cm->get(cameraId);\n>> +\n>> + /*\n>> + * Note that is equivalent to:\n>> + * camera = cm->cameras()[0];\n>> + */\n>> +\n>> +Once a camera has been selected an application needs to acquire an exclusive\n>> +lock to it so no other application can use it.\n>> +\n>> +.. code:: cpp\n>> +\n>> + camera->acquire();\n>> +\n>> +Configure the camera\n>> +--------------------\n>> +\n>> +Before the application can do anything with the camera, it needs to configure\n>> +the image format and sizes of the streams it wants to capture frames from.\n>> +\n>> +Stream configurations are represented by instances of the\n>> +``StreamConfiguration`` class, which are grouped together in a\n>> +``CameraConfiguration`` object. Before an application can start setting its\n>> +desired configuration, a ``CameraConfiguration`` instance needs to be generated\n>> +from the ``Camera`` device using the ``Camera::generateConfiguration()``\n>> +function.\n>> +\n>> +The libcamera library uses the ``StreamRole`` enumeration to define predefined\n>> +ways an application intends to use a camera. The\n>> +``Camera::generateConfiguration()`` function accepts a list of desired roles and\n>> +generates a ``CameraConfiguration`` with the best stream parameters\n>> +configuration for each of the requested roles. If the camera can handle the\n>> +requested roles, it returns an initialized ``CameraConfiguration`` and a null\n>> +pointer if it can't.\n>> +\n>> +It is possible for applications to generate an empty ``CameraConfiguration``\n>> +instance by not providing any role. The desired configuration will have to be\n>> +filled-in manually and manually validated.\n>> +\n>> +In the example application, create a new configuration variable and use the\n>> +``Camera::generateConfiguration`` function to produce a ``CameraConfiguration``\n>> +for the single ``StreamRole::Viewfinder`` role.\n>> +\n>> +.. code:: cpp\n>> +\n>> + std::unique_ptr config = camera->generateConfiguration( { StreamRole::Viewfinder } );\n>> +\n>> +The generated ``CameraConfiguration`` has a ``StreamConfiguration`` instance for\n>> +each ``StreamRole`` the application requested. Each of these has a default size\n>> +and format that the camera assigned, and a list of supported pixel formats and\n>> +sizes.\n>> +\n>> +The code below accesses the first and only ``StreamConfiguration`` item in the\n>> +``CameraConfiguration`` and outputs its parameters to standard output.\n>> +\n>> +.. code:: cpp\n>> +\n>> + StreamConfiguration &streamConfig = config->at(0);\n>> + std::cout << \"Default viewfinder configuration is: \" << streamConfig.toString() << std::endl;\n>> +\n>> +This is expected to output something like:\n>> +\n>> + ``Default viewfinder configuration is: 1280x720-MJPEG``\n>> +\n>> +Change and validate the configuration\n>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n>> +\n>> +With an initialized ``CameraConfiguration``, an application can make changes to\n>> +the parameters it contains, for example, to change the width and height, use the\n>> +following code:\n>> +\n>> +.. code:: cpp\n>> +\n>> + streamConfig.size.width = 640;\n>> + streamConfig.size.height = 480;\n>> +\n>> +If an application changes any parameters, it must validate the configuration\n>> +before applying it to the camera using the ``CameraConfiguration::validate()``\n>> +function. If the new values are not supported by the ``Camera`` device, the\n>> +validation process adjusts the parameters to what it considers to be the closest\n>> +supported values.\n>> +\n>> +The ``validate`` method returns a `Status`_ which applications shall check to\n>> +see if the Pipeline Handler adjusted the configuration.\n>> +\n>> +.. _Status: http://libcamera.org/api-html/classlibcamera_1_1CameraConfiguration.html#a64163f21db2fe1ce0a6af5a6f6847744\n>> +\n>> +For example, the code above set the width and height to 640x480, but if the\n>> +camera cannot produce an image that large, it might adjust the configuration to\n>> +the supported size of 320x240 and return ``Adjusted`` as validation status\n>> +result.\n>> +\n>> +If the configuration to validate cannot be adjusted to a set of supported\n>> +values, the validation procedure fails and returns the ``Invalid`` status.\n>> +\n>> +For this example application, the code below prints the adjusted values to\n>> +standard out.\n>> +\n>> +.. code:: cpp\n>> +\n>> + config->validate();\n>> + std::cout << \"Validated viewfinder configuration is: \" << streamConfig.toString() << std::endl;\n>> +\n>> +For example, the output might be something like\n>> +\n>> + ``Validated viewfinder configuration is: 320x240-MJPEG``\n>> +\n>> +A validated ``CameraConfiguration`` can bet given to the ``Camera`` device to be\n>> +applied to the system.\n>> +\n>> +.. code:: cpp\n>> +\n>> + camera->configure(config.get());\n>> +\n>> +If an application doesn’t first validate the configuration before calling\n>> +``Camera::configure()``, there’s a chance that calling the function can fail, if\n>> +the given configuration would have to be adjusted.\n>> +\n>> +Allocate FrameBuffers\n>> +---------------------\n>> +\n>> +An application needs to reserve the memory that libcamera can write incoming\n>> +frames and data to, and that the application can then read. The libcamera\n>> +library uses ``FrameBuffer`` instances to represent memory buffers allocated in\n>> +memory. An application should reserve enough memory for the frame size the\n>> +streams need based on the configured image sizes and formats.\n>> +\n>> +The libcamera library consumes buffers provided by applications as\n>> +``FrameBuffer`` instances, which makes libcamera a consumer of buffers exported\n>> +by other devices (such as displays or video encoders), or allocated from an\n>> +external allocator (such as ION on Android).\n>> +\n>> +In some situations, applications do not have any means to allocate or get hold\n>> +of suitable buffers, for instance, when no other device is involved, or on Linux\n>> +platforms that lack a centralized allocator. The ``FrameBufferAllocator`` class\n>> +provides a buffer allocator an application can use in these situations.\n>> +\n>> +An application doesn’t have to use the default ``FrameBufferAllocator`` that\n>> +libcamera provides. It can instead allocate memory manually and pass the buffers\n>> +in ``Request``\\s (read more about ``Request`` in `the frame capture section\n>> +<#frame-capture>`_ of this guide). The example in this guide covers using the\n>> +``FrameBufferAllocator`` that libcamera provides.\n>> +\n>> +Using the libcamera ``FrameBufferAllocator``\n>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n>> +\n>> +Applications create a ``FrameBufferAllocator`` for a Camera and use it\n>> +to allocate buffers for streams of a ``CameraConfiguration`` with the\n>> +``allocate()`` function.\n>> +\n>> +The list of allocated buffers can be retrieved using the ``Stream`` instance\n>> +as the parameter of the ``FrameBufferAllocator::buffers()`` function.\n>> +\n>> +.. code:: cpp\n>> +\n>> + FrameBufferAllocator *allocator = new FrameBufferAllocator(camera);\n>> +\n>> + for (StreamConfiguration &cfg : *config) {\n>> + int ret = allocator->allocate(cfg.stream());\n>> + if (ret < 0) {\n>> + std::cerr << \"Can't allocate buffers\" << std::endl;\n>> + return -ENOMEM;\n>> + }\n>> +\n>> + unsigned int allocated = allocator->buffers(cfg.stream()).size();\n>> + std::cout << \"Allocated \" << allocated << \" buffers for stream\" << std::endl;\n>> + }\n>> +\n>> +Frame Capture\n>> +~~~~~~~~~~~~~\n>> +\n>> +The libcamera library implements a streaming model based on per-frame requests.\n>> +For each frame an application wants to capture it must queue a request for it to\n>> +the camera. With libcamera, a ``Request`` is at least one ``Stream`` associated\n>> +with a ``FrameBuffer`` representing the memory location where frames have to be\n>> +stored.\n>> +\n>> +First, by using the ``Stream`` instance associated to each\n>> +``StreamConfiguration``, retrieve the list of ``FrameBuffer``\\s created for it\n>> +using the frame allocator. Then create a vector of requests to be submitted to\n>> +the camera.\n>> +\n>> +.. code:: cpp\n>> +\n>> + Stream *stream = streamConfig.stream();\n>> + const std::vector> &buffers = allocator->buffers(stream);\n>> + std::vector requests;\n>> +\n>> +Proceed to fill the request vector by creating ``Request`` instances from the\n>> +camera device, and associate a buffer for each of them for the ``Stream``.\n>> +\n>> +.. code:: cpp\n>> +\n>> + for (unsigned int i = 0; i < buffers.size(); ++i) {\n>> + Request *request = camera->createRequest();\n>> + if (!request)\n>> + {\n>> + std::cerr << \"Can't create request\" << std::endl;\n>> + return -ENOMEM;\n>> + }\n>> +\n>> + const std::unique_ptr &buffer = buffers[i];\n>> + int ret = request->addBuffer(stream, buffer.get());\n>> + if (ret < 0)\n>> + {\n>> + std::cerr << \"Can't set buffer for request\"\n>> + << std::endl;\n>> + return ret;\n>> + }\n>> +\n>> + requests.push_back(request);\n>> + }\n>> +\n>> +.. TODO: Controls\n>> +\n>> +.. TODO: A request can also have controls or parameters that you can apply to the image.\n>> +\n>> +Event handling and callbacks\n>> +----------------------------\n>> +\n>> +The libcamera library uses the concept of `signals and slots` (similar to `Qt\n>> +Signals and Slots`_) to connect events with callbacks to handle them.\n>> +\n>> +.. _signals and slots: http://libcamera.org/api-html/classlibcamera_1_1Signal.html#details\n>> +.. _Qt Signals and Slots: https://doc.qt.io/qt-5/signalsandslots.html\n>> +\n>> +The ``Camera`` device emits two signals that applications can connect to in\n>> +order to execute callbacks on frame completion events.\n>> +\n>> +The ``Camera::bufferCompleted`` signal notifies applications that a buffer with\n>> +image data is available. Receiving notifications about the single buffer\n>> +completion event allows applications to implement partial request completion\n>> +support, and to inspect the buffer content before the request it is part of has\n>> +fully completed.\n>> +\n>> +The ``Camera::requestCompleted`` signal notifies applications that a request\n>> +has completed, which means all the buffers the request contains have now\n>> +completed. Request completion notifications are always emitted in the same order\n>> +as the requests have been queued to the camera.\n>> +\n>> +To receive the signals emission notifications, connect a slot function to the\n>> +signal to handle it in the application code.\n>> +\n>> +.. code:: cpp\n>> +\n>> + camera->requestCompleted.connect(requestComplete);\n>> +\n>> +For this example application, only the ``Camera::requestCompleted`` signal gets\n>> +handled and the matching ``requestComplete`` slot method outputs information\n>> +about the FrameBuffer to standard output. This callback is typically where an\n>> +application accesses the image data from the camera and does something with it.\n>> +\n>> +Signals operate in the libcamera ``CameraManager`` thread context, so it is\n>> +important not to block the thread for a long time, as this blocks internal\n>> +processing of the camera pipelines, and can affect realtime performance.\n>> +\n>> +Handle request completion events\n>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n>> +\n>> +Create the ``requestComplete`` function by matching the slot signature:\n>> +\n>> +.. code:: cpp\n>> +\n>> + static void requestComplete(Request *request)\n>> + {\n>> + // Code to follow\n>> + }\n>> +\n>> +Request completion events can be emitted for requests which have been canceled,\n>> +for example, by unexpected application shutdown. To avoid an application\n>> +processing invalid image data, it’s worth checking that the request has\n>> +completed successfully. The list of request completion statuses is available in\n>> +the `Request::Status`_ class enum documentation.\n>> +\n>> +.. _Request::Status: https://www.libcamera.org/api-html/classlibcamera_1_1Request.html#a2209ba8d51af8167b25f6e3e94d5c45b\n>> +\n>> +.. code:: cpp\n>> +\n>> + if (request->status() == Request::RequestCancelled)\n>> + return;\n>> +\n>> +If the ``Request`` has completed successfully, applications can access the\n>> +completed buffers using the ``Request::buffers()`` function, which returns a map\n>> +of ``FrameBuffer`` instances associated with the ``Stream`` that produced the\n>> +images.\n>> +\n>> +.. code:: cpp\n>> +\n>> + const std::map &buffers = request->buffers();\n>> +\n>> +Iterating through the map allows applications to inspect each completed buffer\n>> +in this request, and access the metadata associated to each frame.\n>> +\n>> +The metadata buffer contains information such the capture status, a timestamp,\n>> +and the bytes used, as described in the `FrameMetadata`_ documentation.\n>> +\n>> +.. _FrameMetaData: http://libcamera.org/api-html/structlibcamera_1_1FrameMetadata.html\n>> +\n>> +.. code:: cpp\n>> +\n>> + for (auto bufferPair : buffers) {\n>> + FrameBuffer *buffer = bufferPair.second;\n>> + const FrameMetadata &metadata = buffer->metadata();\n>> + }\n>> +\n>> +For this example application, inside the ``for`` loop from above, we ca print\n>> +the Frame sequence number and details of the planes.\n>> +\n>> +.. code:: cpp\n>> +\n>> + std::cout << \" seq: \" << std::setw(6) << std::setfill('0') << metadata.sequence << \" bytesused: \";\n>> +\n>> + unsigned int nplane = 0;\n>> + for (const FrameMetadata::Plane &plane : metadata.planes)\n>> + {\n>> + std::cout << plane.bytesused;\n>> + if (++nplane < metadata.planes.size()) std::cout << \"/\";\n>> + }\n>> +\n>> + std::cout << std::endl;\n>> +\n>> +The expected output shows each monotonically increasing frame sequence number\n>> +and the bytes used by planes.\n>> +\n>> +.. code:: text\n>> +\n>> + seq: 000000 bytesused: 1843200\n>> + seq: 000002 bytesused: 1843200\n>> + seq: 000004 bytesused: 1843200\n>> + seq: 000006 bytesused: 1843200\n>> + seq: 000008 bytesused: 1843200\n>> + seq: 000010 bytesused: 1843200\n>> + seq: 000012 bytesused: 1843200\n>> + seq: 000014 bytesused: 1843200\n>> + seq: 000016 bytesused: 1843200\n>> + seq: 000018 bytesused: 1843200\n>> + seq: 000020 bytesused: 1843200\n>> + seq: 000022 bytesused: 1843200\n>> + seq: 000024 bytesused: 1843200\n>> + seq: 000026 bytesused: 1843200\n>> + seq: 000028 bytesused: 1843200\n>> + seq: 000030 bytesused: 1843200\n>> + seq: 000032 bytesused: 1843200\n>> + seq: 000034 bytesused: 1843200\n>> + seq: 000036 bytesused: 1843200\n>> + seq: 000038 bytesused: 1843200\n>> + seq: 000040 bytesused: 1843200\n>> + seq: 000042 bytesused: 1843200\n>> +\n>> +A completed buffer contains of course image data which can be accessed through\n>> +the per-plane dma-buf file descriptor transported by the ``FrameBuffer``\n>> +instance. An example of how to write image data to disk is available in the\n>> +`BufferWriter class`_ which is a part of the ``cam`` utility application in the\n>> +libcamera repository.\n>> +\n>> +.. _BufferWriter class: https://git.linuxtv.org/libcamera.git/tree/src/cam/buffer_writer.cpp\n>> +\n>> +With the handling of this request completed, it is possible to re-use the\n>> +buffers by adding them to a new ``Request`` instance with their matching\n>> +streams, and finally, queue the new capture request to the camera device:\n>> +\n>> +.. code:: cpp\n>> +\n>> + request = camera->createRequest();\n>> + if (!request)\n>> + {\n>> + std::cerr << \"Can't create request\" << std::endl;\n>> + return;\n>> + }\n>> +\n>> + for (auto it = buffers.begin(); it != buffers.end(); ++it)\n>> + {\n>> + Stream *stream = it->first;\n>> + FrameBuffer *buffer = it->second;\n>> +\n>> + request->addBuffer(stream, buffer);\n>> + }\n>> +\n>> + camera->queueRequest(request);\n>> +\n>> +Request queueing\n>> +----------------\n>> +\n>> +The ``Camera`` device is now ready to receive frame capture requests and\n>> +actually start delivering frames. In order to prepare for that, an application\n>> +needs to first start the camera, and queue requests to it for them to be\n>> +processed.\n>> +\n>> +In the main() function, just after having connected the\n>> +``Camera::requestCompleted`` signal to the callback handler, start the camera\n>> +and queue all the previously created requests.\n>> +\n>> +.. code:: cpp\n>> +\n>> + camera->start();\n>> + for (Request *request : requests)\n>> + camera->queueRequest(request);\n>> +\n>> +Start an event loop\n>> +~~~~~~~~~~~~~~~~~~~\n>> +\n>> +The libcamera library needs an event loop to monitor and dispatch events\n>> +generated by the video devices part of the capture pipeline. Libcamera provides\n> \n> s/Libcamera/libcamera/\n> \n>> +its own ``EventDispatcher`` class (inspired by the `Qt event system`_) to\n>> +process and deliver events generated by ``EventNotifiers``.\n>> +\n>> +.. _Qt event system: https://doc.qt.io/qt-5/eventsandfilters.html\n>> +\n>> +The libcamera library implements this by creating instances of the\n>> +``EventNotifier`` class, which models a file descriptor event source registered\n>> +to an ``EventDispatcher``. Whenever the ``EventDispatcher`` detects an event on\n>> +a notifier it is monitoring, it emits the notifier's\n>> +``EventNotifier::activated`` signal. The libcamera components connect to the\n>> +notifiers' signals and emit application visible events, such as the\n>> +``Camera::bufferReady`` and ``Camera::requestCompleted`` signals.\n>> +\n>> +The code below retrieves a reference to the system-wide event dispatcher and for\n>> +the a fixed duration of 3 seconds, processes all the events detected in the\n>> +system.\n>> +\n>> +.. code:: cpp\n>> +\n>> + EventDispatcher *dispatcher = cm->eventDispatcher();\n>> + Timer timer;\n>> + timer.start(3000);\n>> + while (timer.isRunning())\n>> + dispatcher->processEvents();\n>> +\n>> +Clean up and stop the application\n>> +---------------------------------\n>> +\n>> +The application is now finished with the camera and the resources the camera\n>> +uses, so needs to do the following:\n>> +\n>> +- stop the camera\n>> +- free the buffers in the FrameBufferAllocator and delete it\n>> +- release the lock on the camera and reset the pointer to it\n>> +- stop the camera manager\n>> +\n>> +.. code:: cpp\n>> +\n>> + camera->stop();\n>> + allocator->free(stream);\n>> + delete allocator;\n>> + camera->release();\n>> + camera.reset();\n>> + cm->stop();\n>> +\n>> + return 0;\n>> +\n>> +Build and run instructions\n>> +--------------------------\n>> +\n>> +To build the application, use the `Meson build system`_ which is also the\n>> +official build system of the libcamera library.\n>> +\n>> +Make sure both ``meson`` and ``libcamera`` are installed in your system. Please\n>> +refer to your distribution documentation to install meson and install the most\n>> +recent version of libcamera from the git repository at `Linux TV`_. You would\n>> +also need to install the ``pkg-config`` tool to correctly identify the\n>> +libcamera.so object install location in the system.\n>> +\n>> +.. _Meson build system: https://mesonbuild.com/\n>> +.. _Linux TV: https://git.linuxtv.org/libcamera.git/\n>> +\n>> +Dependencies\n>> +~~~~~~~~~~~~\n>> +\n>> +The test application presented here depends on the libcamera library to be\n>> +available in a path that meson can identify. The libcamera install procedure\n>> +performed using the ``ninja install`` command may by default deploy the\n>> +libcamera components in the ``/usr/local/lib`` path, or a package manager may\n>> +install it to ``/usr/lib`` depending on your distribution. If meson is unable to\n>> +find the location of the libcamera installation, you may need to instruct meson\n>> +to look into a specific path when searching for ``libcamera.so`` by setting the\n>> +``PKG_CONFIG_PATH`` environment variable to the right location.\n>> +\n>> +Adjust the following command to use the ``pkgconfig`` directory where libcamera\n>> +has been installed in your system.\n>> +\n>> +.. code:: shell\n>> +\n>> + export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/\n>> +\n>> +Verify that ``pkg-config`` can identify the ``camera`` library with\n>> +\n>> +.. code:: shell\n>> +\n>> + $ pkg-config --libs --cflags camera\n>> + -I/usr/local/include/libcamera -L/usr/local/lib -lcamera\n>> +\n>> +``meson`` can alternatively use ``cmake`` to locate packages, please refer to\n>> +the ``meson`` documentation if you prefer to use it in place of ``pkgconfig``\n>> +\n>> +Build file\n>> +~~~~~~~~~~\n>> +\n>> +With the dependencies correctly identified, prepare a ``meson.build`` build file\n>> +to be placed in the same directory where the application lives. You can\n>> +name your application as you like, but be sure to update the following snippet\n>> +accordingly. In this example, the application file has been named\n>> +``simple-cam.cpp``.\n>> +\n>> +.. code::\n>> +\n>> + project('simple-cam', 'cpp')\n>> +\n>> + simpler_cam = executable('simple-cam',\n>> + 'simple-cam.cpp',\n>> + dependencies: dependency('camera', required : true))\n>> +\n>> +The ``dependencies`` line instructs meson to ask ``pkgconfig`` (or ``cmake``) to\n>> +locate the ``camera`` library, (libcamera without the lib prefix) which the test\n>> +application will be dynamically linked against.\n>> +\n>> +With the build file in place, compile and run the application with:\n>> +\n>> +.. code:: shell\n>> +\n>> + $ meson build\n>> + $ cd build\n>> + $ ninja\n>> + $ ./simple-cam\n>> +\n>> +It is possible to increase the library debug output by using environment\n>> +variables which control the library log filtering system:\n>> +\n>> +.. code:: shell\n>> +\n>> + $ LIBCAMERA_LOG_LEVELS=0 ./simple-cam\n>> diff --git a/Documentation/index.rst b/Documentation/index.rst\n>> index fb391d2b6ebf..68b7ac06c506 100644\n>> --- a/Documentation/index.rst\n>> +++ b/Documentation/index.rst\n>> @@ -15,3 +15,4 @@\n>> \n>> Developer Guide \n>> Pipeline Handler Writers Guide \n>> + Application Writers Guide \n> \n> Same comment, Writers or Writer's (including in the subject line, same\n> for patch 2/3) ?\n\nYes, I think the apostrophe is correct.\n\n> \n> I would move this before the pipeline handler guide. Same in\n> meson.build.\n\nMoved.\n\n> \n> Other improvements can go on top.\n> \n> Acked-by: Laurent Pinchart \n\nThanks.\n\n\n> \n>> diff --git a/Documentation/meson.build b/Documentation/meson.build\n>> index 9f6c67071da9..6bff2e4a4912 100644\n>> --- a/Documentation/meson.build\n>> +++ b/Documentation/meson.build\n>> @@ -54,6 +54,7 @@ if sphinx.found()\n>> 'index.rst',\n>> 'guides/introduction.rst',\n>> 'guides/pipeline-handler.rst',\n>> + 'guides/application-developer.rst',\n>> ]\n>> \n>> release = 'release=v' + libcamera_git_version\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3CBC7BE173\n\tfor ;\n\tThu, 20 Aug 2020 15:55:17 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BEB0762082;\n\tThu, 20 Aug 2020 17:55:16 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 39EC760381\n\tfor ;\n\tThu, 20 Aug 2020 17:55:16 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9A65FA17;\n\tThu, 20 Aug 2020 17:55:15 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"G7RmdoiN\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597938915;\n\tbh=2MKLGXcXkhbtFGaV+cvrbvsR3wbSf2NtFbPYX+U7KHo=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=G7RmdoiNuWtLj5Z5OpYMG1bPc55cjqvTHuuVKVRTk/tWrwlHRzRRgYqyfXXIUX5kP\n\twBIpQ2tNCFD1ZLqyM4tNcuEsCEMAbyWNK/dnmLMCEC+1oDYl6F51vkvQWb+Ba1NbZ0\n\t8+QEnNr22NXUofuSxEIHbv3tmZSgqgk27Q370LpU=","To":"Laurent Pinchart ","References":"<20200820134751.278033-1-kieran.bingham@ideasonboard.com>\n\t<20200820134751.278033-4-kieran.bingham@ideasonboard.com>\n\t<20200820154634.GT6593@pendragon.ideasonboard.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<92f912d1-42be-f3eb-9bc7-7d3ec1c2ed13@ideasonboard.com>","Date":"Thu, 20 Aug 2020 16:55:12 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200820154634.GT6593@pendragon.ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v4 3/3] Documentation: Guides:\n\tApplication Writers Guide","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"Chris Ward ,\n\tlibcamera devel ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]