From patchwork Tue Sep 15 14:20:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 9616 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77467BF01C for ; Tue, 15 Sep 2020 14:21:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2646362E35; Tue, 15 Sep 2020 16:21:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LoOzU2ot"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B328462E17 for ; Tue, 15 Sep 2020 16:21:07 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 639AD275; Tue, 15 Sep 2020 16:20:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1600179651; bh=DVjzdyI61YRY2raQ9+2l7AS5ixb9casudhQHxO6ILeo=; h=From:To:Cc:Subject:Date:From; b=LoOzU2otBm6DMK9Z9L43l3tFKZG/+FzuwrXfCUqYfUfqP5dOV/M/9aXsR/fABpoZ8 TiexZSJSL7VRytHRrUp7kSQGGkxG3KEirVKNjrQyQa8thZ4aDu4VRD/AzK9Ykj08VE dKHzugWNZ5uFafrAxqMMTTvg2PdwsuIZgqfMd08o= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Tue, 15 Sep 2020 23:20:15 +0900 Message-Id: <20200915142038.28757-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 00/23] IPA isolation implementation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patchset implements IPA isolation, and is fully plumbed and runnable on the raspberrypi pipeline handler. This also includes code generators for the IPA proxies, and headers and serializers for custom data structures and functions. Unlike the RFC, this patch series is structures like a normal patch series, in order that should be merged. The only exception is patches 9/23-22/23, which must be squashed to avoid bisection breakage, and are only kept separate to ease review. To see samples of generated code (without running this), see the RFC "IPA isolation example, with IPC, fully plumbed". To restate the problem, we have two goals: - to be able to run IPAs isolated in a separate process - the isolation must be transparent to both the pipeline handler and the IPA During design of the IPC mechanism, we realized that we could support both custom fuctions and custom data structures, which would be a lot nicer than the tedious manual de/serialization that we had before with IPAOperationData. The architecture of the whole thing is as follows: pipeline handler -> IPAProxyRPi --thread--> IPARPi \ \-> IPAIPC --IPC--> IPAProxyRPiWorker -> IPARPi Where the pipeline author defines the interface between the pipeline handler and the IPA, as well as any data structures. Based on this, headers, de/serializers, and the proxy (including the worker) will be generated. Patche 2/23 imports mojo directly from the chromium repos, with some big things pruned. Patches 3/23 to 8/23 add the meat of this IPC and custom IPA interface mechanisms. Patches 9/23 to 22/23 are modifications in the rest of libcamera to make the whole mechanism work, and must be squashed together to avoid bisection breakage. Patch 17/23 may be of interest, as it shows how one would write the interface and data definition. We may need to have some rules, such as start(), stop(), and init() are required, or something along those lines. Patch 18/23 is also noteworthy, as it shows how one would use the interface that has been defined. Since the headers and proxies are autogenerated, there isn't a patch where the generated code can be seen, however. Paul Elder (23): libcamera: ProcessManager: make ProcessManager lifetime explicitly managed utils: ipc: import mojo utils: ipc: add templates for code generation for IPC mechanism utils: ipc: add generator script meson: ipa, proxy: Generate headers and proxy with mojo libcamera: Add IPADataSerializer libcamera: Add IPAIPC libcamera: Add IPAIPC implementation based on unix socket libcamera: IPAModule: Replace ipa_context with IPAInterface libcamera: IPAProxy: Remove stop() override libcamera: IPAProxy: Add isolate parameter to create() libcamera: PipelineHandler: Remove IPA from base class libcamera: IPAInterface: Remove all functions from IPAInterface libcamera: IPAInterface: make ipaCreate return IPAInterface libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline libcamera: IPAManager: add isolation flag to proxy creation ipa: raspberrypi: Add mojom data definition file libcamera: pipeline, ipa: raspberrypi: Use new data definition libcamera: IPAProxy: Remove registration mechanism libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread libcamera: IPAManager: Make createIPA return proxy directly ipa: remove libipa libcamera: PipelineHandler: Move printing pipeline names to CameraManager .../libcamera/internal/ipa_data_serializer.h | 876 ++++++++ include/libcamera/internal/ipa_ipc.h | 41 + .../libcamera/internal/ipa_ipc_unixsocket.h | 115 + include/libcamera/internal/ipa_manager.h | 31 +- include/libcamera/internal/ipa_module.h | 4 +- include/libcamera/internal/ipa_proxy.h | 31 - include/libcamera/internal/pipeline_handler.h | 1 - include/libcamera/internal/process.h | 27 + include/libcamera/ipa/ipa_interface.h | 23 +- include/libcamera/ipa/meson.build | 89 + include/libcamera/ipa/raspberrypi.h | 55 +- include/libcamera/ipa/raspberrypi.mojom | 110 + src/ipa/libipa/ipa_interface_wrapper.cpp | 285 --- src/ipa/libipa/ipa_interface_wrapper.h | 61 - src/ipa/libipa/meson.build | 15 - src/ipa/meson.build | 2 - src/ipa/raspberrypi/meson.build | 2 - src/ipa/raspberrypi/raspberrypi.cpp | 110 +- src/libcamera/camera_manager.cpp | 4 + src/libcamera/ipa_data_serializer.cpp | 29 + src/libcamera/ipa_ipc.cpp | 36 + src/libcamera/ipa_ipc_unixsocket.cpp | 175 ++ src/libcamera/ipa_manager.cpp | 47 +- src/libcamera/ipa_module.cpp | 16 +- src/libcamera/ipa_proxy.cpp | 85 - src/libcamera/meson.build | 4 +- .../pipeline/raspberrypi/raspberrypi.cpp | 119 +- src/libcamera/pipeline_handler.cpp | 3 - src/libcamera/process.cpp | 46 +- src/libcamera/proxy/ipa_proxy_linux.cpp | 103 - src/libcamera/proxy/ipa_proxy_thread.cpp | 172 -- src/libcamera/proxy/meson.build | 24 +- .../proxy/worker/ipa_proxy_linux_worker.cpp | 90 - src/libcamera/proxy/worker/meson.build | 28 +- utils/ipc/generate.py | 17 + .../module_generated.h.tmpl | 95 + .../module_ipa_proxy.cpp.tmpl | 219 ++ .../module_ipa_proxy.h.tmpl | 108 + .../module_ipa_proxy_worker.cpp.tmpl | 167 ++ .../module_serializer.h.tmpl | 43 + .../libcamera_templates/proxy_functions.tmpl | 180 ++ .../libcamera_templates/serializer.tmpl | 262 +++ .../generators/mojom_libcamera_generator.py | 403 ++++ utils/ipc/mojo/public/LICENSE | 27 + utils/ipc/mojo/public/tools/.style.yapf | 6 + utils/ipc/mojo/public/tools/BUILD.gn | 18 + utils/ipc/mojo/public/tools/bindings/BUILD.gn | 108 + .../ipc/mojo/public/tools/bindings/README.md | 816 +++++++ .../chromium_bindings_configuration.gni | 51 + .../tools/bindings/compile_typescript.py | 27 + .../tools/bindings/concatenate-files.py | 54 + ...concatenate_and_replace_closure_exports.py | 73 + .../bindings/format_typemap_generator_args.py | 36 + .../tools/bindings/gen_data_files_list.py | 52 + .../tools/bindings/generate_type_mappings.py | 187 ++ .../ipc/mojo/public/tools/bindings/mojom.gni | 1941 +++++++++++++++++ .../bindings/mojom_bindings_generator.py | 390 ++++ .../mojom_bindings_generator_unittest.py | 62 + .../tools/bindings/mojom_types_downgrader.py | 119 + .../tools/bindings/validate_typemap_config.py | 57 + utils/ipc/mojo/public/tools/mojom/README.md | 14 + .../mojom/check_stable_mojom_compatibility.py | 170 ++ ...eck_stable_mojom_compatibility_unittest.py | 260 +++ .../mojo/public/tools/mojom/const_unittest.py | 90 + .../mojo/public/tools/mojom/enum_unittest.py | 92 + .../mojo/public/tools/mojom/mojom/BUILD.gn | 43 + .../mojo/public/tools/mojom/mojom/__init__.py | 0 .../mojo/public/tools/mojom/mojom/error.py | 28 + .../mojo/public/tools/mojom/mojom/fileutil.py | 45 + .../tools/mojom/mojom/fileutil_unittest.py | 40 + .../tools/mojom/mojom/generate/__init__.py | 0 .../mojom/mojom/generate/constant_resolver.py | 93 + .../tools/mojom/mojom/generate/generator.py | 325 +++ .../mojom/generate/generator_unittest.py | 74 + .../tools/mojom/mojom/generate/module.py | 1635 ++++++++++++++ .../mojom/mojom/generate/module_unittest.py | 31 + .../public/tools/mojom/mojom/generate/pack.py | 258 +++ .../mojom/mojom/generate/pack_unittest.py | 225 ++ .../mojom/mojom/generate/template_expander.py | 83 + .../tools/mojom/mojom/generate/translate.py | 854 ++++++++ .../mojom/generate/translate_unittest.py | 73 + .../tools/mojom/mojom/parse/__init__.py | 0 .../public/tools/mojom/mojom/parse/ast.py | 427 ++++ .../tools/mojom/mojom/parse/ast_unittest.py | 121 + .../mojom/mojom/parse/conditional_features.py | 82 + .../parse/conditional_features_unittest.py | 233 ++ .../public/tools/mojom/mojom/parse/lexer.py | 251 +++ .../tools/mojom/mojom/parse/lexer_unittest.py | 198 ++ .../public/tools/mojom/mojom/parse/parser.py | 488 +++++ .../mojom/mojom/parse/parser_unittest.py | 1390 ++++++++++++ .../mojo/public/tools/mojom/mojom_parser.py | 361 +++ .../tools/mojom/mojom_parser_test_case.py | 73 + .../tools/mojom/mojom_parser_unittest.py | 171 ++ .../tools/mojom/stable_attribute_unittest.py | 127 ++ .../mojom/version_compatibility_unittest.py | 397 ++++ .../public/tools/run_all_python_unittests.py | 28 + utils/ipc/tools/diagnosis/crbug_1001171.py | 51 + 97 files changed, 16107 insertions(+), 1101 deletions(-) create mode 100644 include/libcamera/internal/ipa_data_serializer.h create mode 100644 include/libcamera/internal/ipa_ipc.h create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h create mode 100644 include/libcamera/ipa/raspberrypi.mojom delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h delete mode 100644 src/ipa/libipa/meson.build create mode 100644 src/libcamera/ipa_data_serializer.cpp create mode 100644 src/libcamera/ipa_ipc.cpp create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp create mode 100755 utils/ipc/generate.py create mode 100644 utils/ipc/generators/libcamera_templates/module_generated.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/module_serializer.h.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py create mode 100644 utils/ipc/mojo/public/LICENSE create mode 100644 utils/ipc/mojo/public/tools/.style.yapf create mode 100644 utils/ipc/mojo/public/tools/BUILD.gn create mode 100644 utils/ipc/mojo/public/tools/bindings/BUILD.gn create mode 100644 utils/ipc/mojo/public/tools/bindings/README.md create mode 100644 utils/ipc/mojo/public/tools/bindings/chromium_bindings_configuration.gni create mode 100644 utils/ipc/mojo/public/tools/bindings/compile_typescript.py create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate-files.py create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py create mode 100755 utils/ipc/mojo/public/tools/bindings/format_typemap_generator_args.py create mode 100644 utils/ipc/mojo/public/tools/bindings/gen_data_files_list.py create mode 100755 utils/ipc/mojo/public/tools/bindings/generate_type_mappings.py create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom.gni create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_types_downgrader.py create mode 100755 utils/ipc/mojo/public/tools/bindings/validate_typemap_config.py create mode 100644 utils/ipc/mojo/public/tools/mojom/README.md create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/const_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/enum_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/BUILD.gn create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/__init__.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/error.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/constant_resolver.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py create mode 100755 utils/ipc/mojo/public/tools/mojom/mojom_parser.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py create mode 100644 utils/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py create mode 100755 utils/ipc/mojo/public/tools/run_all_python_unittests.py create mode 100644 utils/ipc/tools/diagnosis/crbug_1001171.py