[libcamera-devel,v8,0/9] IPA isolation: Part 1: Core components

11269 mbox series

v8 is split in three parts, core components, conversion + plumbing, and
tests + documentation.

This is part 1, and implements IPA isolation.

To restate the problem, we have two goals:
- to be able to run IPAs isolated in a separate process
- the isolation must be transparent to both the pipeline handler and
  the IPA
During design of the IPC mechanism, we realized that we could support
both custom fuctions and custom data structures, which would be a lot
nicer than the tedious manual de/serialization that we had before with
IPAOperationData.

The architecture of the whole thing is as follows:

pipeline handler -> IPAProxyRPi --thread--> IPARPi
                                \
                                 \-> IPCPipe --IPC--> IPAProxyRPiWorker -> IPARPi

Where the pipeline author defines the interface between the pipeline
handler and the IPA, as well as any data structures. Based on this,
headers, de/serializers, and the proxy (including the worker) will be
generated.

Patches 2/9 to 9/9 add the meat of this IPC and custom IPA interface
mechanisms, including the code generation templates, serializers, and
IPC message pipes.

1/9 is new in v6, and implements caching ControlInfoMaps in
ControlSerializer. This allows us to avoid reserializing ControlInfoMaps
every time we send a ControlList over IPC.

In v6 we newly support custom start() definitions.

The biggest change in v6 is that we now support defining structs in
mojom for core libcamera structs, as is shown in core.mojom in 9/9. We
can also designate individually for every struct if we want a header
definition as well, or just a serializer to be generated.

Defining consts in the mojom file is also supported as of v6, meaning
that the rkisp1 and vimc headers can be removed (in Part 2), since those
headers only define consts and enums.

Changes in v8:
- move string, ControlList, and ControlInfoMap (de)serializers from
  ipa_data_serializer.h to ipa_data_serializer.cpp
- clean up the IPADataSerializer a bit
- other minor fixes

Changes in v7:
- simplify ControlSerializer::isCached implementation
- make sendSync and sendAsync only take an IPCMessage
  - sequence number is svaed by proxy instead of IPCPipe
- replace genHeader and genSerdes with skipHeader and skipSerdes in
  core.mojom
- move DEFINE_POD_SERIALIZER from ipa_data_serializer.h to cpp
- add some overflow checks to IPADataSerializer
- add underflow check on IPCPipeUnixSocket receive
- fix CameraSensorInfo
- pull in Niklas's fix for IPCUnixSocket

Changes in v6:
- ControlInfoMap caching in ControlSerializer
- support custom start()
- support defining libcamera structs in core.mojom
- support consts in mojom
- namespacing is required in mojom files
- expand documentation on mojom, in core.mojom

Changes in v5:
- rename IPAIPC to IPCPipe
- IPCPipe passes IPCMessages, instead of byte vector + fd vector
- rename the generated files so the naming is consistent
- make IPADataSerializer use const references and const iterators
- removal of IPA wrapper test moved earlier
- squash all patches related to replacing the C IPA interface with the
  customizable C++ IPA interface
- squash all patches related to making the IPAProxy one-per-pipeline

Changes in v4:
- add IPA guide
- add test for header and serializer generation
- fix style in generated code and IPADataSerializer
- add segfault protections in read/append helpers in de/serializer
- rename generated header and serializer
- rename IPA callback interface to IPA event interface

Changes in v3:
- add support for namespaces in the mojom file
  - note that they individually must not collide with existing
    namespaces (enforced by the C++ compiler)
- move IPAIPCUnixSocket helper functions away from global and into the
  class
- add SPDX and copyright to python scripts
- add parser.py to wrap the mojo parser
- make IPADataSerializer definition a bit cleaner with reimplemented POD
  manipulation functions
- expand mojom documentation in core.mojom

Changes in v2:
- upgrade documentation
- fix documentation compiling
- plumb the new IPC system through the other pipelines
- fix the issue where editing jinja templates wouldn't trigger
  recompile
- enforce IPA interface rules in code generator
- fix previously-untriggered code generation bugs
- add de/serializers for all primitive types, string, and const
  ControlList
- customized the RPi IPA interface
- add licenses
- add tests

Niklas Söderlund (1):
  libcamera: IPCPipeUnixSocket: Check that insertion succeeds

Paul Elder (9):
  libcamera: control_serializer: Save serialized ControlInfoMap in a
    cache
  utils: ipc: add templates for code generation for IPC mechanism
  utils: ipc: add generator script
  utils: ipc: add parser script
  Documentation: skip generating documentation for generated code
  libcamera: Add IPADataSerializer
  libcamera: Add IPCPipe
  libcamera: Add IPCPipe implementation based on unix socket
  ipa: Add core.mojom

 Documentation/Doxyfile.in                     |   6 +-
 .../libcamera/internal/control_serializer.h   |   2 +
 .../libcamera/internal/ipa_data_serializer.h  | 660 ++++++++++++++++++
 include/libcamera/internal/ipc_pipe.h         |  69 ++
 .../libcamera/internal/ipc_pipe_unixsocket.h  |  49 ++
 include/libcamera/ipa/core.mojom              | 215 ++++++
 src/libcamera/control_serializer.cpp          |  26 +
 src/libcamera/ipa_data_serializer.cpp         | 244 +++++++
 src/libcamera/ipc_pipe.cpp                    | 220 ++++++
 src/libcamera/ipc_pipe_unixsocket.cpp         | 149 ++++
 src/libcamera/meson.build                     |   3 +
 utils/ipc/generate.py                         |  29 +
 .../core_ipa_interface.h.tmpl                 |  40 ++
 .../core_ipa_serializer.h.tmpl                |  47 ++
 .../definition_functions.tmpl                 |  53 ++
 .../libcamera_templates/meson.build           |  14 +
 .../module_ipa_interface.h.tmpl               |  87 +++
 .../module_ipa_proxy.cpp.tmpl                 | 236 +++++++
 .../module_ipa_proxy.h.tmpl                   | 128 ++++
 .../module_ipa_proxy_worker.cpp.tmpl          | 226 ++++++
 .../module_ipa_serializer.h.tmpl              |  48 ++
 .../libcamera_templates/proxy_functions.tmpl  | 194 +++++
 .../libcamera_templates/serializer.tmpl       | 313 +++++++++
 .../generators/mojom_libcamera_generator.py   | 508 ++++++++++++++
 utils/ipc/parser.py                           |  20 +
 25 files changed, 3585 insertions(+), 1 deletion(-)
 create mode 100644 include/libcamera/internal/ipa_data_serializer.h
 create mode 100644 include/libcamera/internal/ipc_pipe.h
 create mode 100644 include/libcamera/internal/ipc_pipe_unixsocket.h
 create mode 100644 include/libcamera/ipa/core.mojom
 create mode 100644 src/libcamera/ipa_data_serializer.cpp
 create mode 100644 src/libcamera/ipc_pipe.cpp
 create mode 100644 src/libcamera/ipc_pipe_unixsocket.cpp
 create mode 100755 utils/ipc/generate.py
 create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/definition_functions.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/meson.build
 create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl
 create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl
 create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py
 create mode 100755 utils/ipc/parser.py