[libcamera-devel,00/13] Raspberry Pi: Code refactoring
mbox series

Message ID 20230426131057.21550-1-naush@raspberrypi.com
Headers show
Series
  • Raspberry Pi: Code refactoring
Related show

Message

Naushir Patuck April 26, 2023, 1:10 p.m. UTC 
Hi,

The aim of this (rather large) series is to refactor the Raspberry Pi pipeline
handler and IPA code bases.  We are splitting out hardware specific functions
from general boilerplate tasks in both the pipeline handler and IPA. This would
make it easer for third-parties to implement their own VC4/BCM2835 based
pipeline handlers and IPAs should they wish to. This is done by introducing new
base classes (RPi::PipelineBase and RPi::IpaBase) for the common/boilerplate
code, and derived classes (PipelineHandlerVc4 and IpaVc4) for the HW specific
code.

Additionally, these changes also introduce a new directory structure for the
Raspberry Pi code:

./src/libcamera/pipeline/rpi/
  |- common (boilerplate/common code to all vc4 based pipeline handler implementations)
  |- vc4 (the vc4/bcm2835 HW specific bits of the pipeline handler)

./src/ipa/rpi/
  |- common (boilerplate/common code to all vc4 based IPA implementations)
  |- controller (controller/algorithm code)
  |- cam_helper (camera helpers for all our supported cameras)
  |- vc4 (the vc4/bcm2835 HW specific bits of the IPA)

The Raspberry Pi pipeline handler and IPA meson options have been renamed from
"raspberrypi" to "rpi/vc4".  To enable the Raspberry Pi pipeline handler and IPA
in the build, the following meson command needs to be used:

meson configure -Dpipelines=rpi/vc4 -Dipas=rpi/vc4

Some minor tidy ups, fixes and code improvement patches have also been included
in this series.

Thanks,
Naush

David Plowman (1):
  ipa: raspberrypi: agc: Move weights out of AGC

Naushir Patuck (12):
  meson: ipa: Add mapping for pipeline handler to mojom interface file
  libcamera: ipa_proxy: Allow a prefix for the configuration file
  pipeline: raspberrypi: Refactor and move pipeline handler code
  ipa: raspberrypi: Refactor and move IPA code
  pipeline: raspberrypi: rpi_stream: Set invalid buffer to id == 0
  pipeline: ipa: raspberrypi: Restructure the IPA mojom interface
  pipeline: ipa: raspberrypi: Replace IPA metadataReady() call
  ipa: raspberrypi: Introduce IpaBase class
  pipeline: raspberrypi: Make RPi::Stream::name() return const
    std::string &
  pipeline: raspberrypi: Introduce PipelineHandlerBase class
  pipeline: raspberrypi: Add stream flags to RPi::Stream
  pipeline: vc4: Connect/disconnect IPA and dequeue signals on
    start/stop

 Documentation/environment_variables.rst       |    4 +-
 Documentation/guides/introduction.rst         |    2 +-
 Documentation/guides/ipa.rst                  |   19 +-
 Documentation/guides/pipeline-handler.rst     |    2 +-
 include/libcamera/internal/ipa_proxy.h        |    3 +-
 include/libcamera/ipa/meson.build             |   36 +-
 include/libcamera/ipa/raspberrypi.mojom       |  129 +-
 meson.build                                   |    2 +-
 meson_options.txt                             |    4 +-
 src/ipa/meson.build                           |   11 +
 src/ipa/raspberrypi/meson.build               |   66 -
 src/ipa/{raspberrypi => rpi}/README.md        |    0
 .../cam_helper}/cam_helper.cpp                |    0
 .../cam_helper}/cam_helper.h                  |    2 +-
 .../cam_helper}/cam_helper_imx219.cpp         |    0
 .../cam_helper}/cam_helper_imx290.cpp         |    0
 .../cam_helper}/cam_helper_imx296.cpp         |    0
 .../cam_helper}/cam_helper_imx477.cpp         |    0
 .../cam_helper}/cam_helper_imx519.cpp         |    0
 .../cam_helper}/cam_helper_imx708.cpp         |    0
 .../cam_helper}/cam_helper_ov5647.cpp         |    0
 .../cam_helper}/cam_helper_ov9281.cpp         |    0
 .../cam_helper}/md_parser.h                   |    0
 .../cam_helper}/md_parser_smia.cpp            |    0
 src/ipa/rpi/cam_helper/meson.build            |   14 +
 .../common/ipa_base.cpp}                      | 1272 +++------
 src/ipa/rpi/common/ipa_base.h                 |  125 +
 src/ipa/rpi/common/meson.build                |    7 +
 .../controller/af_algorithm.h                 |    0
 .../controller/af_status.h                    |    0
 .../controller/agc_algorithm.h                |    3 +
 .../controller/agc_status.h                   |    0
 .../controller/algorithm.cpp                  |    0
 .../controller/algorithm.h                    |    0
 .../controller/alsc_status.h                  |    0
 .../controller/awb_algorithm.h                |    0
 .../controller/awb_status.h                   |    0
 .../controller/black_level_status.h           |    0
 .../controller/camera_mode.h                  |    0
 .../controller/ccm_algorithm.h                |    0
 .../controller/ccm_status.h                   |    0
 .../controller/contrast_algorithm.h           |    0
 .../controller/contrast_status.h              |    0
 .../controller/controller.cpp                 |    0
 .../controller/controller.h                   |    0
 .../controller/denoise_algorithm.h            |    0
 .../controller/denoise_status.h               |    0
 .../controller/device_status.cpp              |    0
 .../controller/device_status.h                |    0
 .../controller/dpc_status.h                   |    0
 .../controller/geq_status.h                   |    0
 .../controller/histogram.cpp                  |    0
 .../controller/histogram.h                    |    0
 .../controller/lux_status.h                   |    0
 src/ipa/rpi/controller/meson.build            |   22 +
 .../controller/metadata.h                     |    0
 .../controller/noise_status.h                 |    0
 .../controller/pdaf_data.h                    |    0
 .../{raspberrypi => rpi}/controller/pwl.cpp   |    0
 src/ipa/{raspberrypi => rpi}/controller/pwl.h |    0
 .../controller/region_stats.h                 |    0
 .../controller/rpi/af.cpp                     |    0
 .../{raspberrypi => rpi}/controller/rpi/af.h  |    0
 .../controller/rpi/agc.cpp                    |   27 +-
 .../{raspberrypi => rpi}/controller/rpi/agc.h |    1 +
 .../controller/rpi/alsc.cpp                   |    0
 .../controller/rpi/alsc.h                     |    0
 .../controller/rpi/awb.cpp                    |    0
 .../{raspberrypi => rpi}/controller/rpi/awb.h |    0
 .../controller/rpi/black_level.cpp            |    0
 .../controller/rpi/black_level.h              |    0
 .../controller/rpi/ccm.cpp                    |    0
 .../{raspberrypi => rpi}/controller/rpi/ccm.h |    0
 .../controller/rpi/contrast.cpp               |    0
 .../controller/rpi/contrast.h                 |    0
 .../controller/rpi/dpc.cpp                    |    0
 .../{raspberrypi => rpi}/controller/rpi/dpc.h |    0
 .../controller/rpi/focus.h                    |    0
 .../controller/rpi/geq.cpp                    |    0
 .../{raspberrypi => rpi}/controller/rpi/geq.h |    0
 .../controller/rpi/lux.cpp                    |    0
 .../{raspberrypi => rpi}/controller/rpi/lux.h |    0
 .../controller/rpi/noise.cpp                  |    0
 .../controller/rpi/noise.h                    |    0
 .../controller/rpi/sdn.cpp                    |    0
 .../{raspberrypi => rpi}/controller/rpi/sdn.h |    0
 .../controller/rpi/sharpen.cpp                |    0
 .../controller/rpi/sharpen.h                  |    0
 .../controller/sharpen_algorithm.h            |    0
 .../controller/sharpen_status.h               |    0
 .../controller}/statistics.h                  |    0
 .../{raspberrypi => rpi/vc4}/data/imx219.json |    0
 .../vc4}/data/imx219_noir.json                |    0
 .../{raspberrypi => rpi/vc4}/data/imx290.json |    0
 .../{raspberrypi => rpi/vc4}/data/imx296.json |    0
 .../vc4}/data/imx296_mono.json                |    0
 .../{raspberrypi => rpi/vc4}/data/imx378.json |    0
 .../{raspberrypi => rpi/vc4}/data/imx477.json |    0
 .../vc4}/data/imx477_noir.json                |    0
 .../vc4}/data/imx477_scientific.json          |    0
 .../vc4}/data/imx477_v1.json                  |    0
 .../{raspberrypi => rpi/vc4}/data/imx519.json |    0
 .../{raspberrypi => rpi/vc4}/data/imx708.json |    0
 .../vc4}/data/imx708_noir.json                |    0
 .../vc4}/data/imx708_wide.json                |    0
 .../vc4}/data/imx708_wide_noir.json           |    0
 .../{raspberrypi => rpi/vc4}/data/meson.build |    2 +-
 .../{raspberrypi => rpi/vc4}/data/ov5647.json |    0
 .../vc4}/data/ov5647_noir.json                |    0
 .../vc4}/data/ov9281_mono.json                |    0
 .../vc4}/data/se327m12.json                   |    0
 .../vc4}/data/uncalibrated.json               |    0
 src/ipa/rpi/vc4/meson.build                   |   44 +
 src/ipa/rpi/vc4/vc4.cpp                       |  552 ++++
 src/libcamera/ipa_proxy.cpp                   |   11 +-
 src/libcamera/pipeline/ipu3/ipu3.cpp          |    4 +-
 src/libcamera/pipeline/meson.build            |    9 +
 .../pipeline/raspberrypi/raspberrypi.cpp      | 2433 -----------------
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      |    4 +-
 .../common}/delayed_controls.cpp              |    0
 .../common}/delayed_controls.h                |    0
 .../{raspberrypi => rpi/common}/meson.build   |    5 +-
 .../pipeline/rpi/common/pipeline_base.cpp     | 1449 ++++++++++
 .../pipeline/rpi/common/pipeline_base.h       |  276 ++
 .../common}/rpi_stream.cpp                    |   52 +-
 .../{raspberrypi => rpi/common}/rpi_stream.h  |   62 +-
 .../vc4}/data/example.yaml                    |    4 +-
 .../{raspberrypi => rpi/vc4}/data/meson.build |    2 +-
 .../{raspberrypi => rpi/vc4}/dma_heaps.cpp    |    0
 .../{raspberrypi => rpi/vc4}/dma_heaps.h      |    0
 src/libcamera/pipeline/rpi/vc4/meson.build    |    8 +
 src/libcamera/pipeline/rpi/vc4/vc4.cpp        |  995 +++++++
 src/libcamera/pipeline/vimc/vimc.cpp          |    2 +-
 test/ipa/ipa_interface_test.cpp               |    2 +-
 134 files changed, 4058 insertions(+), 3609 deletions(-)
 delete mode 100644 src/ipa/raspberrypi/meson.build
 rename src/ipa/{raspberrypi => rpi}/README.md (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper.h (99%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_imx219.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_imx290.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_imx296.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_imx477.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_imx519.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_imx708.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_ov5647.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/cam_helper_ov9281.cpp (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/md_parser.h (100%)
 rename src/ipa/{raspberrypi => rpi/cam_helper}/md_parser_smia.cpp (100%)
 create mode 100644 src/ipa/rpi/cam_helper/meson.build
 rename src/ipa/{raspberrypi/raspberrypi.cpp => rpi/common/ipa_base.cpp} (62%)
 create mode 100644 src/ipa/rpi/common/ipa_base.h
 create mode 100644 src/ipa/rpi/common/meson.build
 rename src/ipa/{raspberrypi => rpi}/controller/af_algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/af_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/agc_algorithm.h (93%)
 rename src/ipa/{raspberrypi => rpi}/controller/agc_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/algorithm.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/alsc_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/awb_algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/awb_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/black_level_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/camera_mode.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/ccm_algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/ccm_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/contrast_algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/contrast_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/controller.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/controller.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/denoise_algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/denoise_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/device_status.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/device_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/dpc_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/geq_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/histogram.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/histogram.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/lux_status.h (100%)
 create mode 100644 src/ipa/rpi/controller/meson.build
 rename src/ipa/{raspberrypi => rpi}/controller/metadata.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/noise_status.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/pdaf_data.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/pwl.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/pwl.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/region_stats.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/af.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/af.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/agc.cpp (97%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/agc.h (98%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/alsc.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/alsc.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/awb.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/awb.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/black_level.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/black_level.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/ccm.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/ccm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/contrast.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/contrast.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/dpc.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/dpc.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/focus.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/geq.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/geq.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/lux.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/lux.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/noise.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/noise.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/sdn.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/sdn.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/sharpen.cpp (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/rpi/sharpen.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/sharpen_algorithm.h (100%)
 rename src/ipa/{raspberrypi => rpi}/controller/sharpen_status.h (100%)
 rename src/ipa/{raspberrypi => rpi/controller}/statistics.h (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx219.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx219_noir.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx290.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx296.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx296_mono.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx378.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx477.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx477_noir.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx477_scientific.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx477_v1.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx519.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx708.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx708_noir.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx708_wide.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/imx708_wide_noir.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/meson.build (89%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/ov5647.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/ov5647_noir.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/ov9281_mono.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/se327m12.json (100%)
 rename src/ipa/{raspberrypi => rpi/vc4}/data/uncalibrated.json (100%)
 create mode 100644 src/ipa/rpi/vc4/meson.build
 create mode 100644 src/ipa/rpi/vc4/vc4.cpp
 delete mode 100644 src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
 rename src/libcamera/pipeline/{raspberrypi => rpi/common}/delayed_controls.cpp (100%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/common}/delayed_controls.h (100%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/common}/meson.build (65%)
 create mode 100644 src/libcamera/pipeline/rpi/common/pipeline_base.cpp
 create mode 100644 src/libcamera/pipeline/rpi/common/pipeline_base.h
 rename src/libcamera/pipeline/{raspberrypi => rpi/common}/rpi_stream.cpp (87%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/common}/rpi_stream.h (76%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/vc4}/data/example.yaml (94%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/vc4}/data/meson.build (63%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/vc4}/dma_heaps.cpp (100%)
 rename src/libcamera/pipeline/{raspberrypi => rpi/vc4}/dma_heaps.h (100%)
 create mode 100644 src/libcamera/pipeline/rpi/vc4/meson.build
 create mode 100644 src/libcamera/pipeline/rpi/vc4/vc4.cpp