[RFC,00/19] Add support for rkisp2

27159 mbox series

This series adds support for the rkisp2 pipeline handler and its IPA. It
supports versions of the ISP supported by the rkisp2 driver (hence the
name) in conjuction with the rkcif driver. The rkisp2 driver is posted
on the list in RFC form [5] and currently supports the RK3588.

This pipeline handler implementation only supports memory-to-memory
mode, and I've added a patch to support the the currently posted version
of the shared media graph [6]. I've also implemented a handful of
control algorithms (see patch list below).

This series depends on:
- v2 of "ipa: libipa: Introduce libipa algorithms" [0]
- v2 of "libipa: Re-work IPU3 IPA to use libipa algorithms" [1]
- v2 of "rkisp1: pipeline rework for PFC" (patches 1~24 inclusive) [2]
I've merged these together into a base branch here [3] and I have a
branch including this series here [4].

On the kernel side, this series depends on:
- rfc of "media: rockchip: rkisp2: Add driver for ISP on Rk3588" [5]
- rfc of "Add Shared Media Graph API" [6]
Of which I have a branch here [7]

I've added the IPA before the pipeline handler to not break
compilation/execution and without having to replumb the pipeline
handler. Otherwise I think the patchlist below is self-explanatory. 

[0] https://patchwork.libcamera.org/project/libcamera/list/?series=6013
[1] https://patchwork.libcamera.org/project/libcamera/list/?series=6022
[2] https://patchwork.libcamera.org/project/libcamera/list/?series=5849
[3] https://git.ideasonboard.com/epaul/libcamera/src/branch/epaul/dev/rkisp2/base
[4] https://git.ideasonboard.com/epaul/libcamera/src/branch/epaul/dev/rkisp2/upstream
[5] https://lore.kernel.org/all/20260424175853.638202-1-paul.elder@ideasonboard.com/
[6] https://lore.kernel.org/all/20260619052637.1110672-1-paul.elder@ideasonboard.com/
[7] https://gitlab.freedesktop.org/linux-media/users/epaul/-/tree/epaul/v7.1-rc1/rk3588/shared-mc/upstream

Paul Elder (19):
  utils: update-kernel-headers: Add rkisp2-config.h
  include: linux: rkisp2-config.h: update to v7.0
  ipa: rkisp2: params: Add rkisp2 extensible parameters wrapper
  include: ipa: rkisp2: Add rkisp2 ipa interface
  ipa: rkisp2: Add rkisp2 ipa
  pipeline: rkisp2: Implement pipeline handler for rkisp2
  pipeline: rkisp2: Support shared media graph
  ipa: rkisp2: algo: bls: Implement black level subtraction
  ipa: rkisp2: algo: awb: Implement automatic white balance control
  ipa: rkisp2: algo: agc: Implement automatic gain control
  ipa: rkisp2: algo: ccm: Implement color correction matrix
  ipa: rkisp2: algo: csm: Implement color space conversion
  ipa: rkisp2: algo: goc: Implement gamma out correction
  ipa: rkisp2: algo: lsc: Implement lens shading correction
  utils: tuning: libtuning: image: Enable images missing some metadata
  utils: tuning: rkisp2.py: Add tuning script for rkisp2
  ipa: rkisp2: data: Add uncalibrated tuning file
  ipa: rkisp2: data: Add rudimentary tuning file for imx219
  ipa: rkisp2: data: Add tuning file for imx708

 Documentation/runtime_configuration.rst   |    9 +
 include/libcamera/ipa/meson.build         |    1 +
 include/libcamera/ipa/rkisp2.mojom        |   47 +
 include/linux/README                      |    2 +-
 include/linux/rkisp2-config.h             |  562 +++++++++
 include/linux/videodev2.h                 |    4 +
 meson.build                               |    1 +
 meson_options.txt                         |    5 +-
 src/ipa/rkisp2/algorithms/agc.cpp         |  472 ++++++++
 src/ipa/rkisp2/algorithms/agc.h           |   58 +
 src/ipa/rkisp2/algorithms/algorithm.h     |   42 +
 src/ipa/rkisp2/algorithms/awb.cpp         |  174 +++
 src/ipa/rkisp2/algorithms/awb.h           |   56 +
 src/ipa/rkisp2/algorithms/bls.cpp         |  154 +++
 src/ipa/rkisp2/algorithms/bls.h           |   41 +
 src/ipa/rkisp2/algorithms/ccm.cpp         |  126 ++
 src/ipa/rkisp2/algorithms/ccm.h           |   57 +
 src/ipa/rkisp2/algorithms/csm.cpp         |  177 +++
 src/ipa/rkisp2/algorithms/csm.h           |   34 +
 src/ipa/rkisp2/algorithms/goc.cpp         |  143 +++
 src/ipa/rkisp2/algorithms/goc.h           |   42 +
 src/ipa/rkisp2/algorithms/lsc.cpp         |  263 ++++
 src/ipa/rkisp2/algorithms/lsc.h           |   71 ++
 src/ipa/rkisp2/algorithms/meson.build     |   12 +
 src/ipa/rkisp2/algorithms/module.h        |   28 +
 src/ipa/rkisp2/data/imx219.yaml           |  139 +++
 src/ipa/rkisp2/data/imx708.yaml           |  154 +++
 src/ipa/rkisp2/data/meson.build           |   12 +
 src/ipa/rkisp2/data/uncalibrated.yaml     |    7 +
 src/ipa/rkisp2/ipa_context.h              |  201 ++++
 src/ipa/rkisp2/meson.build                |   30 +
 src/ipa/rkisp2/params.h                   |   83 ++
 src/ipa/rkisp2/rkisp2.cpp                 |  429 +++++++
 src/libcamera/pipeline/rkisp2/meson.build |    5 +
 src/libcamera/pipeline/rkisp2/rkisp2.cpp  | 1330 +++++++++++++++++++++
 utils/tuning/libtuning/image.py           |   25 +-
 utils/tuning/rkisp2.py                    |   42 +
 utils/update-kernel-headers.sh            |    1 +
 38 files changed, 5031 insertions(+), 8 deletions(-)
 create mode 100644 include/libcamera/ipa/rkisp2.mojom
 create mode 100644 include/linux/rkisp2-config.h
 create mode 100644 src/ipa/rkisp2/algorithms/agc.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/agc.h
 create mode 100644 src/ipa/rkisp2/algorithms/algorithm.h
 create mode 100644 src/ipa/rkisp2/algorithms/awb.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/awb.h
 create mode 100644 src/ipa/rkisp2/algorithms/bls.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/bls.h
 create mode 100644 src/ipa/rkisp2/algorithms/ccm.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/ccm.h
 create mode 100644 src/ipa/rkisp2/algorithms/csm.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/csm.h
 create mode 100644 src/ipa/rkisp2/algorithms/goc.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/goc.h
 create mode 100644 src/ipa/rkisp2/algorithms/lsc.cpp
 create mode 100644 src/ipa/rkisp2/algorithms/lsc.h
 create mode 100644 src/ipa/rkisp2/algorithms/meson.build
 create mode 100644 src/ipa/rkisp2/algorithms/module.h
 create mode 100644 src/ipa/rkisp2/data/imx219.yaml
 create mode 100644 src/ipa/rkisp2/data/imx708.yaml
 create mode 100644 src/ipa/rkisp2/data/meson.build
 create mode 100644 src/ipa/rkisp2/data/uncalibrated.yaml
 create mode 100644 src/ipa/rkisp2/ipa_context.h
 create mode 100644 src/ipa/rkisp2/meson.build
 create mode 100644 src/ipa/rkisp2/params.h
 create mode 100644 src/ipa/rkisp2/rkisp2.cpp
 create mode 100644 src/libcamera/pipeline/rkisp2/meson.build
 create mode 100644 src/libcamera/pipeline/rkisp2/rkisp2.cpp
 create mode 100755 utils/tuning/rkisp2.py