[libcamera-devel,v4,0/9] Add Bayer format support for RkISP1

17851 mbox series

Hello,

This patch series implements support for raw Bayer capture in the RkISP1
pipeline handler and IPA module. I have taken over Florian's v2, posted
v3, and addressed review comments in this v4, with further bug fixes.

The RkISP1 can capture raw frames by bypassing the ISP, feeding the
input data directly to the resizer (which has to be bypassed as well,
with a 1:1 scaling ratio). In this mode, the ISP doesn't consume
parameters or produce statistics, which prevents most algorithms from
running. The notable exception is the AGC algorithm which still needs to
run in order to set manual exposure time and analogue gain (automatic
mode is not available).

The series starts with a few drive-by fixes and cleanups. Patches 1/9
and 2/9 turn StreamRole into a scoped enum. These could be merged
independently from the rest of the series if a v5 is needed for the
remaining patches.

Patch 3/9 starts adding support for raw capture by addressing the IPA
side first, allowing algorithms to report if they support raw capture,
and skipping the ones that don't at runtime when capturing raw frames.
The next patch, 4/9, adds support for manual gain and exposure time in
the AGC algorithm. I have picked it up from the list (posted as "[PATCH
v3] ipa: rkisp1: Add support for manual gain and exposure"), in order to
implement raw support in AGC on top in patch 5/9.

Patches 6/9 and 7/9 then prepare the RkISP1 pipeline handler. Patch 6/9
is an improvement previously posted to the list (as "[RFC PATCH v2]
pipeline: rkisp1: Query the driver for formats") that I found useful,
and patch 7/9 a bug fix.

Finally, patches 8/9 and 3/9 implement raw capture support in the
pipeline handler. I have split this into runtime support (8/9) and
configuration support (9/9) to ease review. Patch 9/9 could possibly be
further split into three patches for configure(), validate() and
generateConfiguration(), but that's more complicated and I think that,
while relatively big, 13/13 is still reviewable.

The result has been tested on an NXP i.MX8MP with an IMX219 camera
sensor. I haven't noticed any regression with non-raw capture, and have
successfully captured raw frames in different resolutions by specifying
either a raw role or a raw pixel format on the cam command line. I have
also successfully tested manual gain and exposure with YUV formats using
a capture script.

Florian Sylvestre (2):
  pipeline: rkisp1: Support raw Bayer capture at runtime
  pipeline: rkisp1: Support raw Bayer capture configuration

Laurent Pinchart (5):
  libcamera: stream: Add operator<<() to print StreamRole as a string
  libcamera: stream: Turn StreamRole into scoped enumeration
  ipa: rkisp1: Support raw capture in IPA operations
  ipa: rkisp1: agc: Support raw capture
  pipeline: rkisp1: Fix stream size validation

Paul Elder (2):
  ipa: rkisp1: Add support for manual gain and exposure
  pipeline: rkisp1: Query the driver for formats

 include/libcamera/stream.h                    |   5 +-
 src/android/camera_capabilities.cpp           |   2 +-
 src/apps/lc-compliance/capture_test.cpp       |  17 +-
 src/gstreamer/gstlibcamerapad.cpp             |  20 +-
 src/ipa/rkisp1/algorithms/agc.cpp             | 100 +++++--
 src/ipa/rkisp1/algorithms/agc.h               |   6 +
 src/ipa/rkisp1/algorithms/algorithm.h         |  12 +-
 src/ipa/rkisp1/ipa_context.cpp                |   5 +
 src/ipa/rkisp1/ipa_context.h                  |  15 +-
 src/ipa/rkisp1/rkisp1.cpp                     |  65 ++++-
 src/libcamera/pipeline/imx8-isi/imx8-isi.cpp  |   8 +-
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      | 275 ++++++++++++------
 src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 255 ++++++++++++++--
 src/libcamera/pipeline/rkisp1/rkisp1_path.h   |  15 +-
 src/libcamera/stream.cpp                      |  19 ++
 15 files changed, 665 insertions(+), 154 deletions(-)