[libcamera-devel,PATCH/RFC,00/12] Rework camera configuration to introduce negotiation of parameters

1217 mbox series

Hello,

Camera are currently configured in two steps, first by retrieving a
configuration for a set of stream usages with streamConfiguration(), and
then by configuring the camera with configureStreams(). The camera
configuration can be changed by the application between those two steps,
but there is no way for applications to find out what changes are
compatible with the camera, as the configureStreams() method either
accepts the configuration as-is, or rejects it completely without any
additional feedback. This leaves applications with no other option than
shooting in the dark.

This patch series reworks the camera configuration API to fix this
problems, and introduces a new concept: configuration negotiation
through validation.

Patch 01/12 is an unrelated fix for an issue noticed during development.
The next 5 patches refactor the camera configuration API by renaming the
confusing streamConfiguration() and configureStreams() methods to
generateConfiguration() and configure() (02/12), replacing stream usages
with plain stream roles (03/12, with a regression in the features
exposed by libcamera, but this gets fixed in the same series), and
further refactoring the APIs (04/12 and 05/12) and the pipeline handlers
(06/12).

Patch 07/12 introduces configuration negotiation. The core idea is to
introduce a new validate() method for the CameraConfiguration class that
validates and possibly updates the configuration to the closest
supported configuration. Applications can then create a configuration
(possibly from scratch, or using Camera::generateConfiguration() with a
set of stream roles), modify it, and then call validate() to request a
configuration update from libcamera. If the configuration is updated,
the application can then inspect it, decide to use the updated version,
or try again, until a configuration acceptable for both libcamera and
the application is found.

The validate() operation is implemented with the help of the pipeline
handlers that need to provide a subclass of CameraConfiguration. This
puts more burden on the pipeline handlers, but is offset by the fact
that the pipeline handler configure() method doesn't need to perform
validation anymore as the configuration is validated prior to
configure().

Patches 08/12 to 12/12 then add enumeration of supported stream formats.
This assists pipeline handlers for their implementation of the
validate() method by enumerating supported formats and sizes from a V4L2
device, and aims at eventually replacing the difficult to use FormatEnum
class. Patch 11/12 shows how StreamFormats can be used in the UVC
pipeline handler. As these patches have been developed by Niklas and not
reviewed by me yet I will refrain from commenting further on them in
this cover letter, but will instead proceed to reviewing them.

Laurent Pinchart (7):
  libcamera: camera: Fix std::ostringstream initialisation
  libcamera: camera: Rename configureStreams() and streamConfiguration()
  libcamera: Use stream roles directly instead of StreamUsage
  libcamera: Refactor the camera configuration storage and API
  libcamera: camera: Return a pointer from generateConfiguration()
  libcamera: pipeline: Move camera data classes to the top level scope
  libcamera: camera: Add a validation API to the CameraConfiguration
    class

Niklas Söderlund (5):
  libcamera: stream: Add StreamFormats
  test: stream: Add test for StreamFormat
  libcamera: v4l2_device: Add method to enumerate all discrete frame
    sizes
  libcamera: pipeline: uvcvideo: Validate format in
    UVCCameraConfiguration::validate()
  cam: Validate camera configuration

 include/libcamera/camera.h               |  41 +--
 include/libcamera/stream.h               |  67 +++--
 src/cam/main.cpp                         |  86 +++---
 src/libcamera/camera.cpp                 | 299 +++++++++----------
 src/libcamera/include/pipeline_handler.h |  10 +-
 src/libcamera/include/v4l2_device.h      |   4 +
 src/libcamera/pipeline/ipu3/ipu3.cpp     | 351 ++++++++++++++++-------
 src/libcamera/pipeline/rkisp1/rkisp1.cpp | 210 ++++++++++----
 src/libcamera/pipeline/uvcvideo.cpp      | 161 ++++++++---
 src/libcamera/pipeline/vimc.cpp          | 147 +++++++---
 src/libcamera/pipeline_handler.cpp       |  50 ++--
 src/libcamera/stream.cpp                 | 281 +++++++++++++-----
 src/libcamera/v4l2_device.cpp            |  51 ++++
 src/qcam/main_window.cpp                 |  11 +-
 src/qcam/main_window.h                   |   3 +-
 test/camera/capture.cpp                  |  35 ++-
 test/camera/configuration_default.cpp    |  20 +-
 test/camera/configuration_set.cpp        |  41 ++-
 test/camera/statemachine.cpp             |  38 ++-
 test/meson.build                         |   1 +
 test/stream/meson.build                  |  10 +
 test/stream/stream_formats.cpp           | 112 ++++++++
 22 files changed, 1406 insertions(+), 623 deletions(-)
 create mode 100644 test/stream/meson.build
 create mode 100644 test/stream/stream_formats.cpp