Cover Letter Detail
Show a cover letter.
GET /api/1.1/covers/9616/?format=api
{ "id": 9616, "url": "https://patchwork.libcamera.org/api/1.1/covers/9616/?format=api", "web_url": "https://patchwork.libcamera.org/cover/9616/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20200915142038.28757-1-paul.elder@ideasonboard.com>", "date": "2020-09-15T14:20:15", "name": "[libcamera-devel,00/23] IPA isolation implementation", "submitter": { "id": 17, "url": "https://patchwork.libcamera.org/api/1.1/people/17/?format=api", "name": "Paul Elder", "email": "paul.elder@ideasonboard.com" }, "mbox": "https://patchwork.libcamera.org/cover/9616/mbox/", "series": [ { "id": 1291, "url": "https://patchwork.libcamera.org/api/1.1/series/1291/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=1291", "date": "2020-09-15T14:20:16", "name": "IPA isolation implementation", "version": 1, "mbox": "https://patchwork.libcamera.org/series/1291/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/covers/9616/comments/", "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 77467BF01C\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue, 15 Sep 2020 14:21:10 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2646362E35;\n\tTue, 15 Sep 2020 16:21:10 +0200 (CEST)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B328462E17\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 15 Sep 2020 16:21:07 +0200 (CEST)", "from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 639AD275;\n\tTue, 15 Sep 2020 16:20:50 +0200 (CEST)" ], "Authentication-Results": "lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"LoOzU2ot\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1600179651;\n\tbh=DVjzdyI61YRY2raQ9+2l7AS5ixb9casudhQHxO6ILeo=;\n\th=From:To:Cc:Subject:Date:From;\n\tb=LoOzU2otBm6DMK9Z9L43l3tFKZG/+FzuwrXfCUqYfUfqP5dOV/M/9aXsR/fABpoZ8\n\tTiexZSJSL7VRytHRrUp7kSQGGkxG3KEirVKNjrQyQa8thZ4aDu4VRD/AzK9Ykj08VE\n\tdKHzugWNZ5uFafrAxqMMTTvg2PdwsuIZgqfMd08o=", "From": "Paul Elder <paul.elder@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Date": "Tue, 15 Sep 2020 23:20:15 +0900", "Message-Id": "<20200915142038.28757-1-paul.elder@ideasonboard.com>", "X-Mailer": "git-send-email 2.27.0", "MIME-Version": "1.0", "Subject": "[libcamera-devel] [PATCH 00/23] IPA isolation implementation", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "This patchset implements IPA isolation, and is fully plumbed and\nrunnable on the raspberrypi pipeline handler. This also includes code\ngenerators for the IPA proxies, and headers and serializers for custom\ndata structures and functions. Unlike the RFC, this patch series is\nstructures like a normal patch series, in order that should be merged.\nThe only exception is patches 9/23-22/23, which must be squashed to\navoid bisection breakage, and are only kept separate to ease review.\n\nTo see samples of generated code (without running this), see the RFC\n\"IPA isolation example, with IPC, fully plumbed\".\n\nTo restate the problem, we have two goals:\n- to be able to run IPAs isolated in a separate process\n- the isolation must be transparent to both the pipeline handler and\n the IPA\nDuring design of the IPC mechanism, we realized that we could support\nboth custom fuctions and custom data structures, which would be a lot\nnicer than the tedious manual de/serialization that we had before with\nIPAOperationData.\n\nThe architecture of the whole thing is as follows:\n\npipeline handler -> IPAProxyRPi --thread--> IPARPi\n \\\n \\-> IPAIPC --IPC--> IPAProxyRPiWorker -> IPARPi\n\nWhere the pipeline author defines the interface between the pipeline\nhandler and the IPA, as well as any data structures. Based on this,\nheaders, de/serializers, and the proxy (including the worker) will be\ngenerated.\n\nPatche 2/23 imports mojo directly from the chromium repos, with some\nbig things pruned. Patches 3/23 to 8/23 add the meat of this IPC and\ncustom IPA interface mechanisms. Patches 9/23 to 22/23 are modifications\nin the rest of libcamera to make the whole mechanism work, and must be\nsquashed together to avoid bisection breakage.\n\nPatch 17/23 may be of interest, as it shows how one would write the\ninterface and data definition. We may need to have some rules, such as\nstart(), stop(), and init() are required, or something along those\nlines.\n\nPatch 18/23 is also noteworthy, as it shows how one would use the\ninterface that has been defined. Since the headers and proxies are\nautogenerated, there isn't a patch where the generated code can be seen,\nhowever.\n\nPaul Elder (23):\n libcamera: ProcessManager: make ProcessManager lifetime explicitly\n managed\n utils: ipc: import mojo\n utils: ipc: add templates for code generation for IPC mechanism\n utils: ipc: add generator script\n meson: ipa, proxy: Generate headers and proxy with mojo\n libcamera: Add IPADataSerializer\n libcamera: Add IPAIPC\n libcamera: Add IPAIPC implementation based on unix socket\n libcamera: IPAModule: Replace ipa_context with IPAInterface\n libcamera: IPAProxy: Remove stop() override\n libcamera: IPAProxy: Add isolate parameter to create()\n libcamera: PipelineHandler: Remove IPA from base class\n libcamera: IPAInterface: Remove all functions from IPAInterface\n libcamera: IPAInterface: make ipaCreate return IPAInterface\n libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline\n libcamera: IPAManager: add isolation flag to proxy creation\n ipa: raspberrypi: Add mojom data definition file\n libcamera: pipeline, ipa: raspberrypi: Use new data definition\n libcamera: IPAProxy: Remove registration mechanism\n libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread\n libcamera: IPAManager: Make createIPA return proxy directly\n ipa: remove libipa\n libcamera: PipelineHandler: Move printing pipeline names to\n CameraManager\n\n .../libcamera/internal/ipa_data_serializer.h | 876 ++++++++\n include/libcamera/internal/ipa_ipc.h | 41 +\n .../libcamera/internal/ipa_ipc_unixsocket.h | 115 +\n include/libcamera/internal/ipa_manager.h | 31 +-\n include/libcamera/internal/ipa_module.h | 4 +-\n include/libcamera/internal/ipa_proxy.h | 31 -\n include/libcamera/internal/pipeline_handler.h | 1 -\n include/libcamera/internal/process.h | 27 +\n include/libcamera/ipa/ipa_interface.h | 23 +-\n include/libcamera/ipa/meson.build | 89 +\n include/libcamera/ipa/raspberrypi.h | 55 +-\n include/libcamera/ipa/raspberrypi.mojom | 110 +\n src/ipa/libipa/ipa_interface_wrapper.cpp | 285 ---\n src/ipa/libipa/ipa_interface_wrapper.h | 61 -\n src/ipa/libipa/meson.build | 15 -\n src/ipa/meson.build | 2 -\n src/ipa/raspberrypi/meson.build | 2 -\n src/ipa/raspberrypi/raspberrypi.cpp | 110 +-\n src/libcamera/camera_manager.cpp | 4 +\n src/libcamera/ipa_data_serializer.cpp | 29 +\n src/libcamera/ipa_ipc.cpp | 36 +\n src/libcamera/ipa_ipc_unixsocket.cpp | 175 ++\n src/libcamera/ipa_manager.cpp | 47 +-\n src/libcamera/ipa_module.cpp | 16 +-\n src/libcamera/ipa_proxy.cpp | 85 -\n src/libcamera/meson.build | 4 +-\n .../pipeline/raspberrypi/raspberrypi.cpp | 119 +-\n src/libcamera/pipeline_handler.cpp | 3 -\n src/libcamera/process.cpp | 46 +-\n src/libcamera/proxy/ipa_proxy_linux.cpp | 103 -\n src/libcamera/proxy/ipa_proxy_thread.cpp | 172 --\n src/libcamera/proxy/meson.build | 24 +-\n .../proxy/worker/ipa_proxy_linux_worker.cpp | 90 -\n src/libcamera/proxy/worker/meson.build | 28 +-\n utils/ipc/generate.py | 17 +\n .../module_generated.h.tmpl | 95 +\n .../module_ipa_proxy.cpp.tmpl | 219 ++\n .../module_ipa_proxy.h.tmpl | 108 +\n .../module_ipa_proxy_worker.cpp.tmpl | 167 ++\n .../module_serializer.h.tmpl | 43 +\n .../libcamera_templates/proxy_functions.tmpl | 180 ++\n .../libcamera_templates/serializer.tmpl | 262 +++\n .../generators/mojom_libcamera_generator.py | 403 ++++\n utils/ipc/mojo/public/LICENSE | 27 +\n utils/ipc/mojo/public/tools/.style.yapf | 6 +\n utils/ipc/mojo/public/tools/BUILD.gn | 18 +\n utils/ipc/mojo/public/tools/bindings/BUILD.gn | 108 +\n .../ipc/mojo/public/tools/bindings/README.md | 816 +++++++\n .../chromium_bindings_configuration.gni | 51 +\n .../tools/bindings/compile_typescript.py | 27 +\n .../tools/bindings/concatenate-files.py | 54 +\n ...concatenate_and_replace_closure_exports.py | 73 +\n .../bindings/format_typemap_generator_args.py | 36 +\n .../tools/bindings/gen_data_files_list.py | 52 +\n .../tools/bindings/generate_type_mappings.py | 187 ++\n .../ipc/mojo/public/tools/bindings/mojom.gni | 1941 +++++++++++++++++\n .../bindings/mojom_bindings_generator.py | 390 ++++\n .../mojom_bindings_generator_unittest.py | 62 +\n .../tools/bindings/mojom_types_downgrader.py | 119 +\n .../tools/bindings/validate_typemap_config.py | 57 +\n utils/ipc/mojo/public/tools/mojom/README.md | 14 +\n .../mojom/check_stable_mojom_compatibility.py | 170 ++\n ...eck_stable_mojom_compatibility_unittest.py | 260 +++\n .../mojo/public/tools/mojom/const_unittest.py | 90 +\n .../mojo/public/tools/mojom/enum_unittest.py | 92 +\n .../mojo/public/tools/mojom/mojom/BUILD.gn | 43 +\n .../mojo/public/tools/mojom/mojom/__init__.py | 0\n .../mojo/public/tools/mojom/mojom/error.py | 28 +\n .../mojo/public/tools/mojom/mojom/fileutil.py | 45 +\n .../tools/mojom/mojom/fileutil_unittest.py | 40 +\n .../tools/mojom/mojom/generate/__init__.py | 0\n .../mojom/mojom/generate/constant_resolver.py | 93 +\n .../tools/mojom/mojom/generate/generator.py | 325 +++\n .../mojom/generate/generator_unittest.py | 74 +\n .../tools/mojom/mojom/generate/module.py | 1635 ++++++++++++++\n .../mojom/mojom/generate/module_unittest.py | 31 +\n .../public/tools/mojom/mojom/generate/pack.py | 258 +++\n .../mojom/mojom/generate/pack_unittest.py | 225 ++\n .../mojom/mojom/generate/template_expander.py | 83 +\n .../tools/mojom/mojom/generate/translate.py | 854 ++++++++\n .../mojom/generate/translate_unittest.py | 73 +\n .../tools/mojom/mojom/parse/__init__.py | 0\n .../public/tools/mojom/mojom/parse/ast.py | 427 ++++\n .../tools/mojom/mojom/parse/ast_unittest.py | 121 +\n .../mojom/mojom/parse/conditional_features.py | 82 +\n .../parse/conditional_features_unittest.py | 233 ++\n .../public/tools/mojom/mojom/parse/lexer.py | 251 +++\n .../tools/mojom/mojom/parse/lexer_unittest.py | 198 ++\n .../public/tools/mojom/mojom/parse/parser.py | 488 +++++\n .../mojom/mojom/parse/parser_unittest.py | 1390 ++++++++++++\n .../mojo/public/tools/mojom/mojom_parser.py | 361 +++\n .../tools/mojom/mojom_parser_test_case.py | 73 +\n .../tools/mojom/mojom_parser_unittest.py | 171 ++\n .../tools/mojom/stable_attribute_unittest.py | 127 ++\n .../mojom/version_compatibility_unittest.py | 397 ++++\n .../public/tools/run_all_python_unittests.py | 28 +\n utils/ipc/tools/diagnosis/crbug_1001171.py | 51 +\n 97 files changed, 16107 insertions(+), 1101 deletions(-)\n create mode 100644 include/libcamera/internal/ipa_data_serializer.h\n create mode 100644 include/libcamera/internal/ipa_ipc.h\n create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h\n create mode 100644 include/libcamera/ipa/raspberrypi.mojom\n delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp\n delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h\n delete mode 100644 src/ipa/libipa/meson.build\n create mode 100644 src/libcamera/ipa_data_serializer.cpp\n create mode 100644 src/libcamera/ipa_ipc.cpp\n create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp\n delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp\n delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp\n delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp\n create mode 100755 utils/ipc/generate.py\n create mode 100644 utils/ipc/generators/libcamera_templates/module_generated.h.tmpl\n create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n create mode 100644 utils/ipc/generators/libcamera_templates/module_serializer.h.tmpl\n create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl\n create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n create mode 100644 utils/ipc/mojo/public/LICENSE\n create mode 100644 utils/ipc/mojo/public/tools/.style.yapf\n create mode 100644 utils/ipc/mojo/public/tools/BUILD.gn\n create mode 100644 utils/ipc/mojo/public/tools/bindings/BUILD.gn\n create mode 100644 utils/ipc/mojo/public/tools/bindings/README.md\n create mode 100644 utils/ipc/mojo/public/tools/bindings/chromium_bindings_configuration.gni\n create mode 100644 utils/ipc/mojo/public/tools/bindings/compile_typescript.py\n create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate-files.py\n create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py\n create mode 100755 utils/ipc/mojo/public/tools/bindings/format_typemap_generator_args.py\n create mode 100644 utils/ipc/mojo/public/tools/bindings/gen_data_files_list.py\n create mode 100755 utils/ipc/mojo/public/tools/bindings/generate_type_mappings.py\n create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom.gni\n create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py\n create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py\n create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_types_downgrader.py\n create mode 100755 utils/ipc/mojo/public/tools/bindings/validate_typemap_config.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/README.md\n create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py\n create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/const_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/enum_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/BUILD.gn\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/__init__.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/error.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/constant_resolver.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py\n create mode 100755 utils/ipc/mojo/public/tools/mojom/mojom_parser.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py\n create mode 100644 utils/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py\n create mode 100755 utils/ipc/mojo/public/tools/run_all_python_unittests.py\n create mode 100644 utils/ipc/tools/diagnosis/crbug_1001171.py" }