From patchwork Fri Nov 6 10:36:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10349 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 83F0FBE082 for ; Fri, 6 Nov 2020 10:37:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 09F1B62D24; Fri, 6 Nov 2020 11:37:22 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sWBkBZzV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9DE5D629A6 for ; Fri, 6 Nov 2020 11:37:20 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CE07CA19; Fri, 6 Nov 2020 11:37:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604659040; bh=J9Vvly5qLJS04tzT2mgBm9PNXAh3Swjb7yfUcLuroV4=; h=From:To:Cc:Subject:Date:From; b=sWBkBZzVY2SNkiDQOQ58SQa2GsRCj7NEtFCctvBvECrtMEx0/JnwRWwcr/BJhB9eZ 1C4gLeeFybUHs5TeNqo5/cZXCo9DHSjB7NIBqdSFEoAQc/kw8WBiAIC9g48E8pwhfd kENM4Sj6W7JzlYFAc6W3Qg5H8fTV2cj3WRxCvzrM= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 6 Nov 2020 19:36:30 +0900 Message-Id: <20201106103707.49660-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 00/37] IPA isolation implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patchset implements IPA isolation, and is fully plumbed and runnable on all pipeline handlers. This also includes code generators for the IPA proxies, and headers and serializers for custom data structures and functions. Unlike the RFC, this patch series is structured like a normal patch series, in order that should be merged. The only exception is patches 13/37-33/37, which must be squashed to avoid bisection breakage, and are only kept separate to ease review. To see samples of generated code (without running this), see the RFC "IPA isolation example, with IPC, fully plumbed". To restate the problem, we have two goals: - to be able to run IPAs isolated in a separate process - the isolation must be transparent to both the pipeline handler and the IPA During design of the IPC mechanism, we realized that we could support both custom fuctions and custom data structures, which would be a lot nicer than the tedious manual de/serialization that we had before with IPAOperationData. The architecture of the whole thing is as follows: pipeline handler -> IPAProxyRPi --thread--> IPARPi \ \-> IPAIPC --IPC--> IPAProxyRPiWorker -> IPARPi Where the pipeline author defines the interface between the pipeline handler and the IPA, as well as any data structures. Based on this, headers, de/serializers, and the proxy (including the worker) will be generated. Patch 1/37 imports mojo directly from the chromium repos, with some big things pruned. Patches 4/37 to 11/37 add the meat of this IPC and custom IPA interface mechanisms. Patches 13/37 to 33/37 are modifications in the rest of libcamera to make the whole mechanism work, and must be squashed together to avoid bisection breakage. Patches 34/37 and 35/37 add tests for the serializer and IPC components of this series. Newly in v4, patches 36/37 adds an IPA guide, and 37/37 adds a test to test generated serializers and headers. Patch 25/37 may be of interest, as it shows how one would write the interface and data definition. In v2 I have customized the raspberrypi mojom file to take advantage of the freedom in definition the IPA interface and data structures (as opposed to v1, where it was practically a direct translation of IPAOperationData). In v3 I have added namespace support to the data definition, and this is reflected in its usage. In v2 I have added some rules to the mojom file, such as start(), stop(), and init() are required, along with the main and callback interfaces. These are documented in core.mojom in patch 24/37 and enforced in the code generator in patch 04/37. core.mojom is a new addition in v2, and is where common mojom "structs" (which represent libcamera objects) and IPA interface definition documentaion are. Patch 26/37 is also noteworthy, as it shows how one would use the interface that has been defined. Since the headers and proxies are autogenerated, there isn't a patch where the generated code can be seen, however. Patches 27/37 and 28/37 also define mojom files and use them, but are for vimc and rkisp1, respectively. These will also give an idea on how the mojom definition looks and how it will be used. Changes in v4: - add IPA guide - add test for header and serializer generation - fix style in generated code and IPADataSerializer - add segfault protections in read/append helpers in de/serializer - rename generated header and serializer - rename IPA callback interface to IPA event interface Changes in v3: - add support for namespaces in the mojom file - note that they individually must not collide with existing namespaces (enforced by the C++ compiler) - move IPAIPCUnixSocket helper functions away from global and into the class - add SPDX and copyright to python scripts - add parser.py to wrap the mojo parser - make IPADataSerializer definition a bit cleaner with reimplemented POD manipulation functions - expand mojom documentation in core.mojom Changes in v2: - upgrade documentation - fix documentation compiling - plumb the new IPC system through the other pipelines - fix the issue where editing jinja templates wouldn't trigger recompile - enforce IPA interface rules in code generator - fix previously-untriggered code generation bugs - add de/serializers for all primitive types, string, and const ControlList - customized the RPi IPA interface - add licenses - add tests Paul Elder (37): utils: ipc: import mojo README, meson: Add dependency on ply and jinja2 for IPA interface generation libcamera: Update dep5 to specify license for mojo utils: ipc: add templates for code generation for IPC mechanism utils: ipc: add generator script utils: ipc: add parser script Documentation: skip generating documentation for generated code libcamera: Add IPADataSerializer libcamera: Add IPAIPC libcamera: Add IPAIPC implementation based on unix socket meson: ipa, proxy: Generate headers and proxy with mojo ipa: raspberrypi: meson: Add dependency on generated headers libcamera: IPAModule: Replace ipa_context with IPAInterface libcamera: ipa_context_wrapper: Remove ipa_context_wrapper libcamera: IPAProxy: Remove stop() override libcamera: IPAProxy: Add isolate parameter to create() libcamera: PipelineHandler: Remove IPA from base class libcamera: IPAInterface: Remove all functions from IPAInterface libcamera: IPAInterface: make ipaCreate return IPAInterface libcamera: IPAInterface: remove ipa_context and functions from documentation libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline libcamera: IPAManager: add isolation flag to proxy creation libcamera: IPAManager: Make createIPA return proxy directly ipa: Add core.mojom ipa: raspberrypi: Add mojom data definition file libcamera: pipeline, ipa: raspberrypi: Use new data definition libcamera: pipeline, ipa: vimc: Support the new IPC mechanism libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism libcamera: IPAProxy: Remove registration mechanism libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread ipa: remove libipa tests: ipa_interface_test: Update to use new createIPA tests: Remove IPA wrappers test tests: Add IPADataSerializer test tests: Add test for IPAIPCUnixSocket Documentation: Add IPA writers guide tests: Test IPA serializer generation .reuse/dep5 | 6 + Documentation/Doxyfile.in | 7 +- Documentation/guides/ipa.rst | 112 + Documentation/index.rst | 1 + Documentation/meson.build | 3 +- README.rst | 2 +- .../libcamera/internal/ipa_context_wrapper.h | 52 - .../libcamera/internal/ipa_data_serializer.h | 1004 +++++++++ include/libcamera/internal/ipa_ipc.h | 42 + .../libcamera/internal/ipa_ipc_unixsocket.h | 62 + include/libcamera/internal/ipa_manager.h | 31 +- include/libcamera/internal/ipa_module.h | 4 +- include/libcamera/internal/ipa_proxy.h | 31 - include/libcamera/internal/meson.build | 1 - include/libcamera/internal/pipeline_handler.h | 1 - include/libcamera/ipa/core.mojom | 18 + include/libcamera/ipa/ipa_interface.h | 126 +- include/libcamera/ipa/meson.build | 94 + include/libcamera/ipa/raspberrypi.h | 58 +- include/libcamera/ipa/raspberrypi.mojom | 158 ++ include/libcamera/ipa/rkisp1.h | 18 +- include/libcamera/ipa/rkisp1.mojom | 42 + include/libcamera/ipa/vimc.h | 8 + include/libcamera/ipa/vimc.mojom | 12 + src/ipa/libipa/ipa_interface_wrapper.cpp | 285 --- src/ipa/libipa/ipa_interface_wrapper.h | 61 - src/ipa/libipa/meson.build | 15 - src/ipa/meson.build | 2 - src/ipa/raspberrypi/meson.build | 4 +- src/ipa/raspberrypi/raspberrypi.cpp | 162 +- src/ipa/rkisp1/meson.build | 5 +- src/ipa/rkisp1/rkisp1.cpp | 55 +- src/ipa/vimc/meson.build | 5 +- src/ipa/vimc/vimc.cpp | 18 +- src/libcamera/ipa_context_wrapper.cpp | 297 --- src/libcamera/ipa_data_serializer.cpp | 178 ++ src/libcamera/ipa_interface.cpp | 517 +---- src/libcamera/ipa_ipc.cpp | 110 + src/libcamera/ipa_ipc_unixsocket.cpp | 213 ++ src/libcamera/ipa_manager.cpp | 47 +- src/libcamera/ipa_module.cpp | 10 +- src/libcamera/ipa_proxy.cpp | 101 - src/libcamera/meson.build | 5 +- .../pipeline/raspberrypi/raspberrypi.cpp | 193 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 44 +- src/libcamera/pipeline/vimc/vimc.cpp | 8 +- src/libcamera/pipeline_handler.cpp | 8 - src/libcamera/proxy/ipa_proxy_linux.cpp | 103 - src/libcamera/proxy/ipa_proxy_thread.cpp | 172 -- src/libcamera/proxy/meson.build | 21 +- .../proxy/worker/ipa_proxy_linux_worker.cpp | 90 - src/libcamera/proxy/worker/meson.build | 23 +- test/ipa/ipa_interface_test.cpp | 8 +- test/ipa/ipa_wrappers_test.cpp | 452 ---- test/ipa/meson.build | 7 +- test/ipc/meson.build | 3 +- test/ipc/unixsocket_ipc.cpp | 238 ++ .../generated_serializer_test.cpp | 130 ++ .../generated_serializer/meson.build | 49 + .../generated_serializer/vimc.mojom | 23 + .../ipa_data_serializer_test.cpp | 464 ++++ test/serialization/meson.build | 5 +- utils/ipc/generate.py | 29 + .../libcamera_templates/meson.build | 11 + .../module_ipa_interface.h.tmpl | 113 + .../module_ipa_proxy.cpp.tmpl | 238 ++ .../module_ipa_proxy.h.tmpl | 118 + .../module_ipa_proxy_worker.cpp.tmpl | 187 ++ .../module_ipa_serializer.h.tmpl | 44 + .../libcamera_templates/proxy_functions.tmpl | 205 ++ .../libcamera_templates/serializer.tmpl | 280 +++ utils/ipc/generators/meson.build | 3 + .../generators/mojom_libcamera_generator.py | 488 +++++ utils/ipc/meson.build | 16 + utils/ipc/mojo/public/LICENSE | 27 + utils/ipc/mojo/public/tools/.style.yapf | 6 + utils/ipc/mojo/public/tools/BUILD.gn | 18 + utils/ipc/mojo/public/tools/bindings/BUILD.gn | 108 + .../ipc/mojo/public/tools/bindings/README.md | 816 +++++++ .../chromium_bindings_configuration.gni | 51 + .../tools/bindings/compile_typescript.py | 27 + .../tools/bindings/concatenate-files.py | 54 + ...concatenate_and_replace_closure_exports.py | 73 + .../bindings/format_typemap_generator_args.py | 36 + .../tools/bindings/gen_data_files_list.py | 52 + .../tools/bindings/generate_type_mappings.py | 187 ++ .../ipc/mojo/public/tools/bindings/mojom.gni | 1941 +++++++++++++++++ .../bindings/mojom_bindings_generator.py | 390 ++++ .../mojom_bindings_generator_unittest.py | 62 + .../tools/bindings/mojom_types_downgrader.py | 119 + .../tools/bindings/validate_typemap_config.py | 57 + utils/ipc/mojo/public/tools/mojom/README.md | 14 + .../mojom/check_stable_mojom_compatibility.py | 170 ++ ...eck_stable_mojom_compatibility_unittest.py | 260 +++ .../mojo/public/tools/mojom/const_unittest.py | 90 + .../mojo/public/tools/mojom/enum_unittest.py | 92 + .../mojo/public/tools/mojom/mojom/BUILD.gn | 43 + .../mojo/public/tools/mojom/mojom/__init__.py | 0 .../mojo/public/tools/mojom/mojom/error.py | 28 + .../mojo/public/tools/mojom/mojom/fileutil.py | 45 + .../tools/mojom/mojom/fileutil_unittest.py | 40 + .../tools/mojom/mojom/generate/__init__.py | 0 .../mojom/mojom/generate/constant_resolver.py | 93 + .../tools/mojom/mojom/generate/generator.py | 325 +++ .../mojom/generate/generator_unittest.py | 74 + .../tools/mojom/mojom/generate/module.py | 1635 ++++++++++++++ .../mojom/mojom/generate/module_unittest.py | 31 + .../public/tools/mojom/mojom/generate/pack.py | 258 +++ .../mojom/mojom/generate/pack_unittest.py | 225 ++ .../mojom/mojom/generate/template_expander.py | 83 + .../tools/mojom/mojom/generate/translate.py | 854 ++++++++ .../mojom/generate/translate_unittest.py | 73 + .../tools/mojom/mojom/parse/__init__.py | 0 .../public/tools/mojom/mojom/parse/ast.py | 427 ++++ .../tools/mojom/mojom/parse/ast_unittest.py | 121 + .../mojom/mojom/parse/conditional_features.py | 82 + .../parse/conditional_features_unittest.py | 233 ++ .../public/tools/mojom/mojom/parse/lexer.py | 251 +++ .../tools/mojom/mojom/parse/lexer_unittest.py | 198 ++ .../public/tools/mojom/mojom/parse/parser.py | 488 +++++ .../mojom/mojom/parse/parser_unittest.py | 1390 ++++++++++++ .../mojo/public/tools/mojom/mojom_parser.py | 361 +++ .../tools/mojom/mojom_parser_test_case.py | 73 + .../tools/mojom/mojom_parser_unittest.py | 171 ++ .../tools/mojom/stable_attribute_unittest.py | 127 ++ .../mojom/version_compatibility_unittest.py | 397 ++++ .../public/tools/run_all_python_unittests.py | 28 + utils/ipc/parser.py | 20 + utils/ipc/tools/diagnosis/crbug_1001171.py | 51 + utils/meson.build | 1 + 130 files changed, 17964 insertions(+), 2668 deletions(-) create mode 100644 Documentation/guides/ipa.rst delete mode 100644 include/libcamera/internal/ipa_context_wrapper.h create mode 100644 include/libcamera/internal/ipa_data_serializer.h create mode 100644 include/libcamera/internal/ipa_ipc.h create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h create mode 100644 include/libcamera/ipa/core.mojom create mode 100644 include/libcamera/ipa/raspberrypi.mojom create mode 100644 include/libcamera/ipa/rkisp1.mojom create mode 100644 include/libcamera/ipa/vimc.mojom delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h delete mode 100644 src/ipa/libipa/meson.build delete mode 100644 src/libcamera/ipa_context_wrapper.cpp create mode 100644 src/libcamera/ipa_data_serializer.cpp create mode 100644 src/libcamera/ipa_ipc.cpp create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp delete mode 100644 test/ipa/ipa_wrappers_test.cpp create mode 100644 test/ipc/unixsocket_ipc.cpp create mode 100644 test/serialization/generated_serializer/generated_serializer_test.cpp create mode 100644 test/serialization/generated_serializer/meson.build create mode 100644 test/serialization/generated_serializer/vimc.mojom create mode 100644 test/serialization/ipa_data_serializer_test.cpp create mode 100755 utils/ipc/generate.py create mode 100644 utils/ipc/generators/libcamera_templates/meson.build create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl create mode 100644 utils/ipc/generators/meson.build create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py create mode 100644 utils/ipc/meson.build create mode 100644 utils/ipc/mojo/public/LICENSE create mode 100644 utils/ipc/mojo/public/tools/.style.yapf create mode 100644 utils/ipc/mojo/public/tools/BUILD.gn create mode 100644 utils/ipc/mojo/public/tools/bindings/BUILD.gn create mode 100644 utils/ipc/mojo/public/tools/bindings/README.md create mode 100644 utils/ipc/mojo/public/tools/bindings/chromium_bindings_configuration.gni create mode 100644 utils/ipc/mojo/public/tools/bindings/compile_typescript.py create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate-files.py create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py create mode 100755 utils/ipc/mojo/public/tools/bindings/format_typemap_generator_args.py create mode 100644 utils/ipc/mojo/public/tools/bindings/gen_data_files_list.py create mode 100755 utils/ipc/mojo/public/tools/bindings/generate_type_mappings.py create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom.gni create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_types_downgrader.py create mode 100755 utils/ipc/mojo/public/tools/bindings/validate_typemap_config.py create mode 100644 utils/ipc/mojo/public/tools/mojom/README.md create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/const_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/enum_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/BUILD.gn create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/__init__.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/error.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/constant_resolver.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py create mode 100755 utils/ipc/mojo/public/tools/mojom/mojom_parser.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py create mode 100755 utils/ipc/mojo/public/tools/run_all_python_unittests.py create mode 100755 utils/ipc/parser.py create mode 100644 utils/ipc/tools/diagnosis/crbug_1001171.py