| Message ID | 20200826110926.67192-1-paul.elder@ideasonboard.com |
|---|---|
| State | Superseded |
| Headers | show |
diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 26458153..be4fd6db 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -292,7 +292,7 @@ std::unique_ptr<IPAProxy> IPAManager::createIPA(PipelineHandler *pipe, } std::unique_ptr<IPAProxy> proxy = - pf->create(m, !self_->isSignatureValid(m)); + pf->create(m, true); if (!proxy->isValid()) { LOG(IPAManager, Error) << "Failed to load proxy";
This patchset implements IPA isolation, and is fully plumbed and runnable on the raspberrypi pipeline handler. This patchset is not organized as a proper patchset, but rather such that it is easier to understand what components there are, and how they will fit together. To restate the problem, we have two goals: - to be able to run IPAs isolated in a separate process - the isolation must be transparent to both the pipeline handler and the IPA During design of the IPC mechanism, we realized that we could support both custom fuctions and custom data structures, which would be a lot nicer than the tedious manual de/serialization that we had before with IPAOperationData. This patchset shows how this can be achieved. The architecture of the whole thing is as follows: pipeline handler -> IPAProxyRPi --thread--> IPARPi \ \-> IPAIPC --IPA--> IPAProxyRPiWorker -> IPARPi The pipeline handler author will write an IPA interface definition, in mojo format. This will declare what functions the IPA shall implement, what callbacks the IPA can signal, and what custom data structures can be passed (in addition to primitives and libcamera data structures). An example of this can be seen in 1/17. This IPA interface will become the IPAInterface specific to the pipeline handler. In the example in this RFC, that will be IPARPiInterface. In the diagram above, IPAProxyRPi and IPARPi both implement IPARPiInterface. A compiler will generate a header for the custom data structures, as well as a header for the IPA interface. This can also be seen in 1/17. Patches 2/17 and 3/17 show how the pipeline author and IPA author would expect to use the IPA interface and data structures defined in the mojo file. I think it is quite straightforward and transparent. Patch 4/17 shows the hand-written IPADataSerializer for built-in and libcamera types, as well as the IPADataSerializer for the custom data types, which will be generated. These will be used by the proxies, which can be seen in 5/17. Each pipeline shall have its own proxy. This is to allow custom data types and functions. The proxy will be generated from some proxy template, and crossed with the IPA interface as defined in the mojo file. The proxy decides whether to call the IPA function directly, or via IPC. Thus, all IPA modules will be wrapped in an IPAProxy. The proxy is also responsible for de/serializing the data, before sending it through the IPC mechanism. The IPC mechanism is designed to be swap-outabble, and simply exposes an async and synchronous version of send(), and a callback signal. These can be seen in 6/17, along with an implementation using UnixSocket. 7/17 enables this whole infrastructure in meson. Note that this only implements IPC for raspberrypi, so make sure to disable the other pipeline handlers. This can be done like: meson -Dpipelines=raspberrypi build Also, to test isolation, the following needs to be applied: return nullptr; }) And don't forget to set permission on the dma heap: sudo chmod 766 /dev/dma_heap/linux,cma 8/17 is a collection of tests that I implmented during debug. If there is any value in them, I'll polish them and merge them. Otherwise I'll discard them. The rest of the patches are all small tweaks in other parts of libcamera to get this whole IPC infrastructure to run. Paul Elder (17): IPA: IPC: raspberrypi: Add data definition and generated header libcamera: pipeline: raspberrypi: Use generated IPARPiInterface libcamera: IPA: raspberrypi: Use generated IPARPiInterface IPA: IPC: add IPADataSerializer IPA: IPC: add IPAProxyRPi and IPAProxyRPiWorker IPA: IPC: add IPAIPC meson: Enable IPC tests: IPC: Add various tests for the IPC framework libcamera: ProcessManager: make ProcessManager lifetime explicitly managed libcamera: IPAModule: Replace ipa_context with IPAInterface libcamera: IPAProxy: Remove stop() override libcamera: IPAProxy: Add isolate parameter to create() libcamera: PipelineHandler: Remove IPA from base class libcamera: IPAInterface: Remove all functions from IPAInterface libcamera: IPAInterface: make ipaCreate return IPAInterface libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline libcamera: IPAManager: add isolation flag to proxy creation .../libcamera/internal/ipa_data_serializer.h | 816 ++++++++++++++++++ include/libcamera/internal/ipa_ipc.h | 73 ++ .../libcamera/internal/ipa_ipc_unixsocket.h | 116 +++ include/libcamera/internal/ipa_module.h | 4 +- include/libcamera/internal/ipa_proxy.h | 24 +- .../internal/ipa_proxy_raspberrypi.h | 117 +++ include/libcamera/internal/pipeline_handler.h | 1 - include/libcamera/internal/process.h | 27 + include/libcamera/ipa/ipa_interface.h | 23 +- include/libcamera/ipa/raspberrypi.h | 61 +- include/libcamera/ipa/raspberrypi.mojom | 97 +++ .../libcamera/ipa/raspberrypi_serializer.h | 487 +++++++++++ include/libcamera/ipa/raspberrypi_wrapper.h | 136 +++ src/ipa/libipa/meson.build | 4 +- src/ipa/raspberrypi/raspberrypi.cpp | 107 ++- src/libcamera/camera_manager.cpp | 2 + src/libcamera/ipa_data_serializer.cpp | 29 + src/libcamera/ipa_ipc.cpp | 58 ++ src/libcamera/ipa_ipc_unixsocket.cpp | 177 ++++ src/libcamera/ipa_manager.cpp | 7 +- src/libcamera/ipa_module.cpp | 16 +- src/libcamera/meson.build | 5 +- .../pipeline/raspberrypi/raspberrypi.cpp | 125 +-- src/libcamera/process.cpp | 46 +- src/libcamera/proxy/ipa_proxy_raspberrypi.cpp | 381 ++++++++ src/libcamera/proxy/meson.build | 5 +- .../worker/ipa_proxy_raspberrypi_worker.cpp | 355 ++++++++ src/libcamera/proxy/worker/meson.build | 3 +- test/ipa/meson.build | 4 +- test/ipc/meson.build | 3 +- test/ipc/unixsocket_fds.cpp | 284 ++++++ .../control_serialization_ipa.cpp | 85 ++ test/serialization/meson.build | 5 +- .../rpi_action_serialization.cpp | 141 +++ .../rpi_config_serialization.cpp | 162 ++++ 35 files changed, 3778 insertions(+), 208 deletions(-) create mode 100644 include/libcamera/internal/ipa_data_serializer.h create mode 100644 include/libcamera/internal/ipa_ipc.h create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h create mode 100644 include/libcamera/internal/ipa_proxy_raspberrypi.h create mode 100644 include/libcamera/ipa/raspberrypi.mojom create mode 100644 include/libcamera/ipa/raspberrypi_serializer.h create mode 100644 include/libcamera/ipa/raspberrypi_wrapper.h create mode 100644 src/libcamera/ipa_data_serializer.cpp create mode 100644 src/libcamera/ipa_ipc.cpp create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp create mode 100644 src/libcamera/proxy/ipa_proxy_raspberrypi.cpp create mode 100644 src/libcamera/proxy/worker/ipa_proxy_raspberrypi_worker.cpp create mode 100644 test/ipc/unixsocket_fds.cpp create mode 100644 test/serialization/control_serialization_ipa.cpp create mode 100644 test/serialization/rpi_action_serialization.cpp create mode 100644 test/serialization/rpi_config_serialization.cpp