[0/4] Add Sync Layer

24259 mbox series

This series depends on:
- v4 of "Add Layers support" [0]
- v1 of "libcamera: clock_recovery: Use nanoseconds in addSample()" [1]
- v1 of "utils: Add unary negation operation to Duration" [2]

There is a branch available at [3]. There is another version of the
branch available at [4], which is the same branch rebased on top of
Raspberry Pi's next [5] branch with an additional patch [6] on top to
make Raspberry Pi use the Sync Layer instead of RPiSync.

This patch series adds a layer implementation of the Sync algorithm,
where any number cameras can synchronize the timing of their captures.
This allows any camera to use the Sync algorithm, as long as it
implements the SyncAdjustment control and SensorTimestamp.

I tested this on a pi5 (with an imx219 and imx708) and a Debix SOM (with
an imx335 and imx283) with the following capture scripts, running the
client first before running the server in separate terminals:

$ cat cscript-sync-client.yaml 
frames:
  - 1:
      FrameDurationLimits: [33333, 33333]
      SyncMode: 2

$ cat cscript-sync-server.yaml 
frames:
  - 1:
      FrameDurationLimits: [33333, 33333]
      SyncMode: 1

I ran:

- pi5 with pure rpi/next (RPiSync) [7]
- pi5 with this series on top of rpi/next (SyncLayer) [8]
- som with this series (SyncLayer) [9]

And I logged the correction time over a single run each of 8+ minutes at
30fps.
Here are the descriptive statistics for the correction values (all in
microseconds):

    pi5 with RPiSync
    Min. 1st Qu. Median Mean 3rd Qu. Max.
    225.0 243.0 249.0 248.8 255.0 274.0

    pi5 with SyncLayer
    Min. 1st Qu. Median Mean 3rd Qu. Max.
    233 244 249 249 254 262

    som with SyncLayer
    Min. 1st Qu. Median Mean 3rd Qu. Max.
    -16623 -11205 -5373 -1936 11363 16585

I have graphs of these runs here [10]

[0] https://patchwork.libcamera.org/project/libcamera/list/?series=5401
[1] https://patchwork.libcamera.org/project/libcamera/list/?series=5403
[2] https://patchwork.libcamera.org/project/libcamera/list/?series=5404
[3] https://git.uk.ideasonboard.com/epaul/libcamera/src/branch/epaul/dev/sync
[4] https://git.uk.ideasonboard.com/epaul/libcamera/src/branch/epaul/dev/layer-pi5
[5] https://github.com/raspberrypi/libcamera/tree/next
[6] https://git.uk.ideasonboard.com/epaul/libcamera/commit/d5a389405040c6cf2f37686ca3cbfbc13bb68150
[7] https://git.uk.ideasonboard.com/epaul/libcamera/src/commit/c576b4375a4b630ba0a8fad4bec4c16786d9dd8b
[8] https://git.uk.ideasonboard.com/epaul/libcamera/commit/d5a389405040c6cf2f37686ca3cbfbc13bb68150
[9] https://git.uk.ideasonboard.com/epaul/libcamera/commit/28c4f9fe2db8c772fbba5db48ced2ab9572b8363
[10] https://git.uk.ideasonboard.com/epaul/libcamera/pulls/1#issuecomment-18703

Paul Elder (4):
  controls: Add SyncAdjustment and SyncInterface controls
  layer: Add layer that implements the sync algorithm
  libipa: Add SyncHelper
  ipa: rkisp1: agc: Add support for sync

 src/ipa/libipa/meson.build          |   2 +
 src/ipa/libipa/sync_helper.cpp      |  99 ++++++
 src/ipa/libipa/sync_helper.h        |  36 +++
 src/ipa/rkisp1/algorithms/agc.cpp   |  22 +-
 src/ipa/rkisp1/algorithms/agc.h     |   3 +-
 src/ipa/rkisp1/ipa_context.h        |   1 +
 src/layer/meson.build               |   1 +
 src/layer/sync/meson.build          |  16 +
 src/layer/sync/sync.cpp             | 465 ++++++++++++++++++++++++++++
 src/layer/sync/sync.h               |  96 ++++++
 src/libcamera/control_ids_core.yaml |  23 ++
 11 files changed, 762 insertions(+), 2 deletions(-)
 create mode 100644 src/ipa/libipa/sync_helper.cpp
 create mode 100644 src/ipa/libipa/sync_helper.h
 create mode 100644 src/layer/sync/meson.build
 create mode 100644 src/layer/sync/sync.cpp
 create mode 100644 src/layer/sync/sync.h