[libcamera-devel,v3,0/8] libcamera: Add DRM/KMS viewfinder display to cam

13195 mbox series

Hello,

This patch series extends the cam application to implement live
viewfinder display using a DRM/KMS device. Supported display features
are currently fairly limited (scaling and overlays are not supported for
instance), but should nonetheless be a good first step.

Patches 1/8 extends to EventLoop class to support file descriptor
events, needed by KMS. Patches 2/8 to 4/8 then refactor the cam
application and the BufferWriter class to introduce an abstract
FrameSink class. Patch 5/8 adds a set of classes to wrap the DRM/KMS API
(using libdrm). Patches 6/8 and 7/8 add a DRM/KMS frame sink
implementation, and patch 8/8 wires it up in the cam application.

I've kept patch 7/8 separate from 6/8 to outline in the history that
it's one of multiple options, and make it easier to switch to a
different implementation later if needed. Enabling the display is a
costly operation, so doing it at start() time would be best. However, we
would need a frame buffer to be displayed at that point, and none is
available yet from the camera. Patch 7/8 thus delays enabling the
display to the first frame. The operation isn't too costly from a
userspace point of view as the atomic request is committed
asynchronously (although it still requires two ioctl calls, one to
create the mode blob - but this could be moved to start() - and one to
commit the request). Another option would be to allocate an extra dumb
buffer and fill it with static data for the first frame.

I've tested this with the simple pipeline handler on an R-Car H3
Salvator-XS board, running

cam -c 'platform/vimc.0 Sensor B' \
        -s pixelformat=BGR888,width=1440,height=900 -C -D HDMI-A-1

with vimc as a source. As the R-Car DU requires DMA contiguous memory by
default (IOMMU support is enabled), a patch is required to use the vb2
dma-contig allocator in vimc. It can be found at [1].

The size of the stream has to match the size of the display, as scaling
isn't supported yet. As vimc requires the output size to be a multiple
of 3, this restricts the possible resolutions (we should *really* fix
this in the vimc driver).

Compared to v2, review comments have been taken into account. Please see
changelogs in individual patches for details.

Compared to v1, the frame sink implementation is now based on requests
instead of buffers, which should address Niklas' concerns.

[1] https://lore.kernel.org/linux-media/20210730001939.30769-1-laurent.pinchart+renesas@ideasonboard.com/T/#u

Laurent Pinchart (8):
  cam: event_loop: Add support for file descriptor events
  cam: Add FrameSink base class
  cam: Turn BufferWriter into a FrameSink
  cam: Rename BufferWriter to FileSink
  cam: Add DRM helper classes
  cam: Add KMS sink class
  cam: kms_sink: Enable display on first frame
  cam: Add support for viewfinder through DRM/KMS

 src/cam/buffer_writer.h                      |  31 -
 src/cam/camera_session.cpp                   |  94 ++-
 src/cam/camera_session.h                     |   6 +-
 src/cam/drm.cpp                              | 663 +++++++++++++++++++
 src/cam/drm.h                                | 331 +++++++++
 src/cam/event_loop.cpp                       |  44 ++
 src/cam/event_loop.h                         |  20 +
 src/cam/{buffer_writer.cpp => file_sink.cpp} |  49 +-
 src/cam/file_sink.h                          |  38 ++
 src/cam/frame_sink.cpp                       |  67 ++
 src/cam/frame_sink.h                         |  34 +
 src/cam/kms_sink.cpp                         | 297 +++++++++
 src/cam/kms_sink.h                           |  74 +++
 src/cam/main.cpp                             |   6 +
 src/cam/main.h                               |   1 +
 src/cam/meson.build                          |  17 +-
 16 files changed, 1718 insertions(+), 54 deletions(-)
 delete mode 100644 src/cam/buffer_writer.h
 create mode 100644 src/cam/drm.cpp
 create mode 100644 src/cam/drm.h
 rename src/cam/{buffer_writer.cpp => file_sink.cpp} (67%)
 create mode 100644 src/cam/file_sink.h
 create mode 100644 src/cam/frame_sink.cpp
 create mode 100644 src/cam/frame_sink.h
 create mode 100644 src/cam/kms_sink.cpp
 create mode 100644 src/cam/kms_sink.h