[v3,00/20] Add support for rkisp2

28120 mbox series

This series adds support for the rkisp2 pipeline handler and its IPA. It
supports versions of the ISP supported by the rkisp2 driver (hence the
name) in conjuction with the rkcif driver. The rkisp2 driver is posted
on the list (currently v3) [0] and currently supports the RK3588.

This pipeline handler implementation only supports memory-to-memory
mode. The RFC version had a patch [1] to support inline mode using shared
media graph [2] which I have removed in v2 to simplify things a little.

See the patch list below for the list of control algorithms that are
supported in the current version.

This series depends on:
- v2 of "rkisp1: pipeline rework for PFC" (patches 1~24 inclusive) [3]
I've merged these together into a base branch here [4] and I have a
branch including this series here [5].

On the kernel side, this series depends on:
- v3 of "media: rockchip: rkisp2: Add driver for ISP on Rk3588" [0]
Of which I have a branch here [6]

I've added the IPA before the pipeline handler to not break
compilation/execution and without having to replumb the pipeline
handler. Otherwise I think the patchlist below is self-explanatory. 

Compared to the RFC version, not much has changed specifically on the
libcamera side. Most of the changes are to match the changes in the
driver, which are:
- split DMA RX and main/self path pads on the ISP subdev
- no more crop rectangles on subdev, as all crop and scaling
  configuration has moved to parameters buffers
- use extensible stats

The one libcamera-specific change is that the ScalerCrop control has now
been implemented in a crop algorithm module. See the specific patch for
more details.

v3 is mainly to keep up with the kernel changes, especially since the
uapi header has changed.

[0] https://lore.kernel.org/linux-media/20260827-rkisp2-v3-0-e91ff0a604d6@ideasonboard.com/T/#t
[1] https://patchwork.libcamera.org/patch/27166/
[2] https://lore.kernel.org/all/20260619052637.1110672-1-paul.elder@ideasonboard.com/
[3] https://patchwork.libcamera.org/project/libcamera/list/?series=5849
[4] https://git.ideasonboard.com/epaul/libcamera/src/branch/epaul/dev/rkisp2/base
[5] https://git.ideasonboard.com/epaul/libcamera/src/branch/epaul/dev/rkisp2/upstream-v3
[6] https://git.ideasonboard.com/epaul/linux/commits/branch/epaul/v7.2-rc1/rk3588/rkisp2/upstream-v3

Paul Elder (20):
  utils: update-kernel-headers: Add rkisp2-config.h
  include: linux: rkisp2-config.h: update to v7.2-rc1
  ipa: rkisp2: params: Add rkisp2 extensible parameters wrapper
  ipa: rkisp2: stats: Add rkisp2 extensible statistics wrapper
  include: ipa: rkisp2: Add rkisp2 ipa interface
  ipa: rkisp2: Add rkisp2 ipa
  pipeline: rkisp2: Implement pipeline handler for rkisp2
  ipa: rkisp2: algo: bls: Implement black level subtraction
  ipa: rkisp2: algo: awb: Implement automatic white balance control
  ipa: rkisp2: algo: agc: Implement automatic gain control
  ipa: rkisp2: algo: ccm: Implement color correction matrix
  ipa: rkisp2: algo: csm: Implement color space conversion
  ipa: rkisp2: algo: goc: Implement gamma out correction
  ipa: rkisp2: algo: lsc: Implement lens shading correction
  ipa: rkisp2: algo: crop: Implement scaler crop
  utils: tuning: libtuning: image: Enable images missing some metadata
  utils: tuning: rkisp2.py: Add tuning script for rkisp2
  ipa: rkisp2: data: Add uncalibrated tuning file
  ipa: rkisp2: data: Add rudimentary tuning file for imx219
  ipa: rkisp2: data: Add tuning file for imx708

 Documentation/runtime_configuration.rst      |    9 +
 include/libcamera/ipa/meson.build            |    1 +
 include/libcamera/ipa/rkisp2.mojom           |   47 +
 include/linux/README                         |    2 +-
 include/linux/media/rockchip/rkisp2-config.h |  640 +++++++++
 include/linux/videodev2.h                    |    4 +
 meson.build                                  |    1 +
 meson_options.txt                            |    3 +-
 src/ipa/meson.build                          |    1 +
 src/ipa/rkisp2/algorithms/agc.cpp            |  479 +++++++
 src/ipa/rkisp2/algorithms/agc.h              |   58 +
 src/ipa/rkisp2/algorithms/algorithm.h        |   27 +
 src/ipa/rkisp2/algorithms/awb.cpp            |  211 +++
 src/ipa/rkisp2/algorithms/awb.h              |   56 +
 src/ipa/rkisp2/algorithms/bls.cpp            |  154 +++
 src/ipa/rkisp2/algorithms/bls.h              |   41 +
 src/ipa/rkisp2/algorithms/ccm.cpp            |  126 ++
 src/ipa/rkisp2/algorithms/ccm.h              |   57 +
 src/ipa/rkisp2/algorithms/crop.cpp           |   99 ++
 src/ipa/rkisp2/algorithms/crop.h             |   39 +
 src/ipa/rkisp2/algorithms/csm.cpp            |  177 +++
 src/ipa/rkisp2/algorithms/csm.h              |   34 +
 src/ipa/rkisp2/algorithms/goc.cpp            |  143 ++
 src/ipa/rkisp2/algorithms/goc.h              |   42 +
 src/ipa/rkisp2/algorithms/lsc.cpp            |  263 ++++
 src/ipa/rkisp2/algorithms/lsc.h              |   71 +
 src/ipa/rkisp2/algorithms/meson.build        |   13 +
 src/ipa/rkisp2/algorithms/module.h           |   29 +
 src/ipa/rkisp2/data/imx219.yaml              |  139 ++
 src/ipa/rkisp2/data/imx708.yaml              |  155 +++
 src/ipa/rkisp2/data/meson.build              |   12 +
 src/ipa/rkisp2/data/uncalibrated.yaml        |    7 +
 src/ipa/rkisp2/ipa_context.h                 |  211 +++
 src/ipa/rkisp2/meson.build                   |   30 +
 src/ipa/rkisp2/params.h                      |   85 ++
 src/ipa/rkisp2/rkisp2.cpp                    |  440 ++++++
 src/ipa/rkisp2/stats.h                       |   65 +
 src/libcamera/pipeline/rkisp2/meson.build    |    5 +
 src/libcamera/pipeline/rkisp2/rkisp2.cpp     | 1293 ++++++++++++++++++
 utils/tuning/libtuning/image.py              |   25 +-
 utils/tuning/rkisp2.py                       |   42 +
 utils/update-kernel-headers.sh               |    1 +
 42 files changed, 5330 insertions(+), 7 deletions(-)
 create mode 100644 include/libcamera/ipa/rkisp2.mojom
 create mode 100644 include/linux/media/rockchip/rkisp2-config.h
 create mode 100644 src/ipa/rkisp2/algorithms/agc.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/agc.h
 create mode 100644 src/ipa/rkisp2/algorithms/algorithm.h
 create mode 100644 src/ipa/rkisp2/algorithms/awb.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/awb.h
 create mode 100644 src/ipa/rkisp2/algorithms/bls.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/bls.h
 create mode 100644 src/ipa/rkisp2/algorithms/ccm.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/ccm.h
 create mode 100644 src/ipa/rkisp2/algorithms/crop.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/crop.h
 create mode 100644 src/ipa/rkisp2/algorithms/csm.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/csm.h
 create mode 100644 src/ipa/rkisp2/algorithms/goc.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/goc.h
 create mode 100644 src/ipa/rkisp2/algorithms/lsc.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/lsc.h
 create mode 100644 src/ipa/rkisp2/algorithms/meson.build
 create mode 100644 src/ipa/rkisp2/algorithms/module.h
 create mode 100644 src/ipa/rkisp2/data/imx219.yaml
 create mode 100644 src/ipa/rkisp2/data/imx708.yaml
 create mode 100644 src/ipa/rkisp2/data/meson.build
 create mode 100644 src/ipa/rkisp2/data/uncalibrated.yaml
 create mode 100644 src/ipa/rkisp2/ipa_context.h
 create mode 100644 src/ipa/rkisp2/meson.build
 create mode 100644 src/ipa/rkisp2/params.h
 create mode 100644 src/ipa/rkisp2/rkisp2.cpp
 create mode 100644 src/ipa/rkisp2/stats.h
 create mode 100644 src/libcamera/pipeline/rkisp2/meson.build
 create mode 100644 src/libcamera/pipeline/rkisp2/rkisp2.cpp
 create mode 100755 utils/tuning/rkisp2.py