[libcamera-devel,v2,0/3] Add sensor mode selection to GStreamer
mbox series

Message ID 20230324181247.302586-1-nicolas@ndufresne.ca
Headers show
Series
  • Add sensor mode selection to GStreamer
Related show

Message

Nicolas Dufresne March 24, 2023, 6:12 p.m. UTC 
From: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Sensor mode selection is needed to achieve specific framerate, or select
specific field of view. The currently implemented method to do so it not
portable, but works for M2M ISP. It consist of creating a stream with Raw role
and forcing the width/height/format configuration that is unique to the wanted
mode. Enumerating the modes is done by doing a validation loop.

This has been tested on Raspberry Pi with an Arducam IMX477 sensor. This sensor
has four modes:

    Modes: 'SRGGB10_CSI2P' : 1332x990 [120.05 fps - (696, 528)/2664x1980 crop]
           'SRGGB12_CSI2P' : 2028x1080 [50.03 fps - (0, 440)/4056x2160 crop]
                             2028x1520 [40.01 fps - (0, 0)/4056x3040 crop]
                             4056x3040 [10.00 fps - (0, 0)/4056x3040 crop]

Only one of these can achieve 60 fps, though at the cost of a smaller field of
view. The pipeline manager does not take the framerate control into
consideration when selecting a mode, since you need a configuration in order to
set the control. It also does not know anything about the wanted field of view.

To solve this, two new properties are added. sensor-modes is a GstCaps
representation of all the modes. The sensor mode caps looks like:

  sensor-mode,width=1332,height=990,format=SRGGB10,framerate=[1/1, 120/1];
  sensor-mode,width=2028,height=1080,format=SRGGB12,framerate=[1/1, 50/1];
  ...

The user can select a mode with a filter set to sensor-mode. The sensor-mode
value can be the exact mode, or could be just something to match to. If an
application want to ensure a 60fps capable mode is selected, it
can do:

   libcamerasrc sensor-mode=sensor/mode,framerate=60/1 ! ...

Note that on cameras like UVC camera, where its not possible to have multiple
streams, sensor mode enumeration (which is extremely slow on these) is skipped,
and the sensor mode selection will have no effect.

Changes in v2:
  - Fixed missing merge conflicts
  - Retested with UVC and RPi

Nicolas Dufresne (3):
  gstreamer: Add sensor mode selection
  doc: gstreamer: Add missing queues
  doc: gstreamer: Document the sensor-mode(s) properties

 README.rst                           |  18 ++++-
 src/gstreamer/gstlibcamera-utils.cpp |   4 +
 src/gstreamer/gstlibcamerasrc.cpp    | 110 ++++++++++++++++++++++++++-
 3 files changed, 128 insertions(+), 4 deletions(-)