[libcamera-devel,v3,00/13] libcamera: ipa: Add basic IPA support

2024 mbox series

Hi,

This series adds basic support for Image Processing Algorithms (IPA). It
extends the pipeline handler framework to optionally allow an IPA plugin
to prepare a parameters buffer and examine a statistics buffer as well
as applying V4L2 controls on a sensor. It's not mandatory for a pipeline
to have an IPA component, but if a pipeline handler supports an IPA it
becomes mandatory, if an IPA component can't be found that pipeline will
not register any cameras.

This series adds an Auto Exposure control and extends the rkisp1
pipeline with and IPA that implements a very basic control loop for
exposure time and gain. The series compared to v2 have been merged with 
the meta data series that was developed on top of this and now supports 
reporting meta data back to the application about the control loop 
state.

The series do not make use of serialization or other patches floating 
around to make isolation of the IPA easer. Some care have been taken to 
make the IPA interface simpler to "carry" over a C interface. But more 
work is needed as that work becomes more mature. If this is done on top 
or in a next version of this series is an ordering of what is ready 
first.

To be able to test this series using qcam two timer patches for the Qt 
event dispatcher posted separately. To test this series using cam no 
external patches are needed.

The series is more or less rewritten from v2 so a meaningful change log 
is hard to produce.

Niklas Söderlund (13):
  libcamera: pipeline: Move IPA from pipeline to camera data
  libcamera: controls: Add AeEnable
  libcamera: controls: Allow read only access to control values
  libcamera: request: Allow read only access to controls
  libcamera: request: Add IPAMetaData
  libcamera: pipeline: Add helper to process meta data coming from IPA
  libcamera: ipa: meson: Allow access to internal libcamera headers
  libcamera: ipa: Extend to support IPA interactions
  libcamera: timeline: Add a basic timeline implementation
  test: Add timeline test
  include: linux: Add rkisp1 kernel header and format definitions
  libcamera: ipa: rkisp1: Add basic control of auto exposure
  libcamera: pipeline: rkisp1: Attach to an IPA

 include/ipa/ipa_interface.h                |  21 +
 include/libcamera/control_ids.h            |   1 +
 include/libcamera/controls.h               |   1 +
 include/libcamera/request.h                |  13 +
 include/linux/rkisp1-config.h              | 816 +++++++++++++++++++++
 include/linux/videodev2.h                  |   4 +
 src/ipa/ipa_dummy.cpp                      |   6 +-
 src/ipa/ipa_rkisp1.cpp                     | 228 ++++++
 src/ipa/meson.build                        |  15 +-
 src/libcamera/controls.cpp                 |  52 +-
 src/libcamera/include/meson.build          |   1 +
 src/libcamera/include/pipeline_handler.h   |   7 +
 src/libcamera/include/timeline.h           |  71 ++
 src/libcamera/ipa_interface.cpp            | 106 +++
 src/libcamera/meson.build                  |   1 +
 src/libcamera/pipeline/rkisp1/meson.build  |   1 +
 src/libcamera/pipeline/rkisp1/rkisp1.cpp   | 263 ++++++-
 src/libcamera/pipeline/rkisp1/rkisp1.h     |  81 ++
 src/libcamera/pipeline/rkisp1/timeline.cpp |  56 ++
 src/libcamera/pipeline/vimc.cpp            |  17 +-
 src/libcamera/pipeline_handler.cpp         |  49 ++
 src/libcamera/proxy/ipa_proxy_linux.cpp    |  13 +-
 src/libcamera/request.cpp                  |  48 +-
 src/libcamera/timeline.cpp                 | 267 +++++++
 test/meson.build                           |   1 +
 test/timeline.cpp                          |  97 +++
 26 files changed, 2202 insertions(+), 34 deletions(-)
 create mode 100644 include/linux/rkisp1-config.h
 create mode 100644 src/ipa/ipa_rkisp1.cpp
 create mode 100644 src/libcamera/include/timeline.h
 create mode 100644 src/libcamera/pipeline/rkisp1/rkisp1.h
 create mode 100644 src/libcamera/pipeline/rkisp1/timeline.cpp
 create mode 100644 src/libcamera/timeline.cpp
 create mode 100644 test/timeline.cpp