[{"id":12641,"web_url":"https://patchwork.libcamera.org/comment/12641/","msgid":"<20200922134249.GF45948@pyrite.rasen.tech>","date":"2020-09-22T13:42:49","subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"On Tue, Sep 22, 2020 at 10:34:59PM +0900, Paul Elder wrote:\n> This patch series depends on the patch series \"Unify utils locations\".\n> \n> This patchset implements IPA isolation, and is fully plumbed and\n> runnable on all pipeline handlers. This also includes code\n> generators for the IPA proxies, and headers and serializers for custom\n> data structures and functions. Unlike the RFC, this patch series is\n> structured like a normal patch series, in order that should be merged.\n> The only exception is patches 16/38-36/38, which must be squashed to\n> avoid bisection breakage, and are only kept separate to ease review.\n\nHello all,\n\nIm so sorry, I forgot to put the v2 tag on this whole series.\n\n\nPaul\n\n> To see samples of generated code (without running this), see the RFC\n> \"IPA isolation example, with IPC, fully plumbed\".\n> \n> To restate the problem, we have two goals:\n> - to be able to run IPAs isolated in a separate process\n> - the isolation must be transparent to both the pipeline handler and\n>   the IPA\n> During design of the IPC mechanism, we realized that we could support\n> both custom fuctions and custom data structures, which would be a lot\n> nicer than the tedious manual de/serialization that we had before with\n> IPAOperationData.\n> \n> The architecture of the whole thing is as follows:\n> \n> pipeline handler -> IPAProxyRPi --thread--> IPARPi\n>                                 \\\n>                                  \\-> IPAIPC --IPC--> IPAProxyRPiWorker -> IPARPi\n> \n> Where the pipeline author defines the interface between the pipeline\n> handler and the IPA, as well as any data structures. Based on this,\n> headers, de/serializers, and the proxy (including the worker) will be\n> generated.\n> \n> Patches 1/38~04/38 will be fast tracked, and I plan to merge them\n> separately from the rest of this series after they are approved in this\n> round.\n> \n> Patch 6/38 imports mojo directly from the chromium repos, with some\n> big things pruned. Patches 8/38 to 15/38 add the meat of this IPC and\n> custom IPA interface mechanisms. Patches 16/38 to 36/38 are modifications\n> in the rest of libcamera to make the whole mechanism work, and must be\n> squashed together to avoid bisection breakage. Patches 37/38 and 38/38\n> add tests for the serializer and IPC components of this series.\n> \n> Patch 27/38 may be of interest, as it shows how one would write the\n> interface and data definition. In v2 I have customized the raspberrypi\n> mojom file to take advantage of the freedom in definition the IPA\n> interface and data structures (as opposed to v1, where it was\n> practically a direct translation of IPAOperationData).\n> \n> In v2 I have added some rules to the mojom file, such as start(),\n> stop(), and init() are required, along with the main and callback\n> interfaces. These are documented in core.mojom in patch 26/38 and\n> enforced in the code generator in patch 08/38. core.mojom is a new\n> addition in v2, and is where common mojom \"structs\" (which represent\n> libcamera objects) and IPA interface definition documentaion are.\n> \n> Patch 28/38 is also noteworthy, as it shows how one would use the\n> interface that has been defined. Since the headers and proxies are\n> autogenerated, there isn't a patch where the generated code can be seen,\n> however.\n> \n> Patches 29/38 and 30/38 also define mojom files and use them, but are\n> for vimc and rkisp1, respectively. These will also give an idea on how\n> the mojom definition looks and how it will be used.\n> \n> Changes in v2:\n> - upgrade documentation\n> - fix documentation compiling\n> - plumb the new IPC system through the other pipelines\n> - fix the issue where editing jinja templates wouldn't trigger\n>   recompile\n> - enforce IPA interface rules in code generator\n> - fix previously-untriggered code generation bugs\n> - add de/serializers for all primitive types, string, and const\n>   ControlList\n> - customized the RPi IPA interface\n> - add licenses\n> - add tests\n> \n> Paul Elder (38):\n>   Documentation: coding-style: Document global variable guidelines\n>   libcamera: ProcessManager: make ProcessManager lifetime explicitly\n>     managed\n>   libcamera: PipelineHandler: Move printing pipeline names to\n>     CameraManager\n>   libcamera: process: forward-declare EventNotifier\n>   libcamera: meson: Add internal headers to sources\n>   utils: ipc: import mojo\n>   libcamera: Update dep5 to specify license for mojo\n>   utils: ipc: add templates for code generation for IPC mechanism\n>   utils: ipc: add generator script\n>   Documentation: skip generating documentation for generated code\n>   meson: ipa, proxy: Generate headers and proxy with mojo\n>   ipa: raspberrypi: meson: Add dependency on generated headers\n>   libcamera: Add IPADataSerializer\n>   libcamera: Add IPAIPC\n>   libcamera: Add IPAIPC implementation based on unix socket\n>   libcamera: IPAModule: Replace ipa_context with IPAInterface\n>   libcamera: ipa_context_wrapper: Remove ipa_context_wrapper\n>   libcamera: IPAProxy: Remove stop() override\n>   libcamera: IPAProxy: Add isolate parameter to create()\n>   libcamera: PipelineHandler: Remove IPA from base class\n>   libcamera: IPAInterface: Remove all functions from IPAInterface\n>   libcamera: IPAInterface: make ipaCreate return IPAInterface\n>   libcamera: IPAInterface: remove ipa_context and functions from\n>     documentation\n>   libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline\n>   libcamera: IPAManager: add isolation flag to proxy creation\n>   ipa: Add core.mojom\n>   ipa: raspberrypi: Add mojom data definition file\n>   libcamera: pipeline, ipa: raspberrypi: Use new data definition\n>   libcamera: pipeline, ipa: vimc: Support the new IPC mechanism\n>   libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism\n>   libcamera: IPAProxy: Remove registration mechanism\n>   libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread\n>   libcamera: IPAManager: Make createIPA return proxy directly\n>   ipa: remove libipa\n>   tests: ipa_interface_test: Update to use new createIPA\n>   tests: Remove IPA wrappers test\n>   tests: Add IPADataSerializer test\n>   tests: Add test for IPAIPCUnixSocket\n> \n>  .reuse/dep5                                   |    6 +\n>  Documentation/Doxyfile.in                     |    7 +-\n>  Documentation/coding-style.rst                |   15 +\n>  Documentation/meson.build                     |    2 -\n>  .../libcamera/internal/ipa_context_wrapper.h  |   52 -\n>  .../libcamera/internal/ipa_data_serializer.h  | 1220 +++++++++++\n>  include/libcamera/internal/ipa_ipc.h          |   41 +\n>  .../libcamera/internal/ipa_ipc_unixsocket.h   |  115 +\n>  include/libcamera/internal/ipa_manager.h      |   31 +-\n>  include/libcamera/internal/ipa_module.h       |    4 +-\n>  include/libcamera/internal/ipa_proxy.h        |   31 -\n>  include/libcamera/internal/meson.build        |    1 -\n>  include/libcamera/internal/pipeline_handler.h |    1 -\n>  include/libcamera/internal/process.h          |   29 +\n>  include/libcamera/ipa/core.mojom              |   74 +\n>  include/libcamera/ipa/ipa_interface.h         |  126 +-\n>  include/libcamera/ipa/meson.build             |   89 +\n>  include/libcamera/ipa/raspberrypi.h           |   55 +-\n>  include/libcamera/ipa/raspberrypi.mojom       |  157 ++\n>  include/libcamera/ipa/rkisp1.h                |   14 +-\n>  include/libcamera/ipa/rkisp1.mojom            |   42 +\n>  include/libcamera/ipa/{ipa_vimc.h => vimc.h}  |    4 +\n>  include/libcamera/ipa/vimc.mojom              |   12 +\n>  src/ipa/libipa/ipa_interface_wrapper.cpp      |  285 ---\n>  src/ipa/libipa/ipa_interface_wrapper.h        |   61 -\n>  src/ipa/libipa/meson.build                    |   15 -\n>  src/ipa/meson.build                           |    2 -\n>  src/ipa/raspberrypi/meson.build               |    4 +-\n>  src/ipa/raspberrypi/raspberrypi.cpp           |  154 +-\n>  src/ipa/rkisp1/meson.build                    |    5 +-\n>  src/ipa/rkisp1/rkisp1.cpp                     |   55 +-\n>  src/ipa/vimc/meson.build                      |    5 +-\n>  src/ipa/vimc/vimc.cpp                         |   20 +-\n>  src/libcamera/camera_manager.cpp              |    5 +\n>  src/libcamera/ipa_context_wrapper.cpp         |  297 ---\n>  src/libcamera/ipa_data_serializer.cpp         |  154 ++\n>  src/libcamera/ipa_interface.cpp               |  517 +----\n>  src/libcamera/ipa_ipc.cpp                     |  110 +\n>  src/libcamera/ipa_ipc_unixsocket.cpp          |  175 ++\n>  src/libcamera/ipa_manager.cpp                 |   47 +-\n>  src/libcamera/ipa_module.cpp                  |   10 +-\n>  src/libcamera/ipa_proxy.cpp                   |  101 -\n>  src/libcamera/meson.build                     |    6 +-\n>  .../pipeline/raspberrypi/raspberrypi.cpp      |  170 +-\n>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |   44 +-\n>  src/libcamera/pipeline/vimc/vimc.cpp          |    8 +-\n>  src/libcamera/pipeline_handler.cpp            |   11 -\n>  src/libcamera/process.cpp                     |   46 +-\n>  src/libcamera/proxy/ipa_proxy_linux.cpp       |  103 -\n>  src/libcamera/proxy/ipa_proxy_thread.cpp      |  172 --\n>  src/libcamera/proxy/meson.build               |   19 +-\n>  .../proxy/worker/ipa_proxy_linux_worker.cpp   |   90 -\n>  src/libcamera/proxy/worker/meson.build        |   22 +-\n>  test/ipa/ipa_interface_test.cpp               |    7 +-\n>  test/ipa/ipa_wrappers_test.cpp                |  452 ----\n>  test/ipa/meson.build                          |    5 +-\n>  test/ipc/meson.build                          |    3 +-\n>  test/ipc/unixsocket_ipc.cpp                   |  238 ++\n>  test/log/log_process.cpp                      |    2 +\n>  test/process/process_test.cpp                 |    2 +\n>  .../ipa_data_serializer_test.cpp              |  463 ++++\n>  test/serialization/meson.build                |    3 +-\n>  utils/ipc/generate.py                         |   21 +\n>  .../libcamera_templates/meson.build           |   11 +\n>  .../module_generated.h.tmpl                   |   96 +\n>  .../module_ipa_proxy.cpp.tmpl                 |  222 ++\n>  .../module_ipa_proxy.h.tmpl                   |  107 +\n>  .../module_ipa_proxy_worker.cpp.tmpl          |  178 ++\n>  .../module_serializer.h.tmpl                  |   44 +\n>  .../libcamera_templates/proxy_functions.tmpl  |  185 ++\n>  .../libcamera_templates/serializer.tmpl       |  268 +++\n>  utils/ipc/generators/meson.build              |    3 +\n>  .../generators/mojom_libcamera_generator.py   |  452 ++++\n>  utils/ipc/meson.build                         |   14 +\n>  utils/ipc/mojo/public/LICENSE                 |   27 +\n>  utils/ipc/mojo/public/tools/.style.yapf       |    6 +\n>  utils/ipc/mojo/public/tools/BUILD.gn          |   18 +\n>  utils/ipc/mojo/public/tools/bindings/BUILD.gn |  108 +\n>  .../ipc/mojo/public/tools/bindings/README.md  |  816 +++++++\n>  .../chromium_bindings_configuration.gni       |   51 +\n>  .../tools/bindings/compile_typescript.py      |   27 +\n>  .../tools/bindings/concatenate-files.py       |   54 +\n>  ...concatenate_and_replace_closure_exports.py |   73 +\n>  .../bindings/format_typemap_generator_args.py |   36 +\n>  .../tools/bindings/gen_data_files_list.py     |   52 +\n>  .../tools/bindings/generate_type_mappings.py  |  187 ++\n>  .../ipc/mojo/public/tools/bindings/mojom.gni  | 1941 +++++++++++++++++\n>  .../bindings/mojom_bindings_generator.py      |  390 ++++\n>  .../mojom_bindings_generator_unittest.py      |   62 +\n>  .../tools/bindings/mojom_types_downgrader.py  |  119 +\n>  .../tools/bindings/validate_typemap_config.py |   57 +\n>  utils/ipc/mojo/public/tools/mojom/README.md   |   14 +\n>  .../mojom/check_stable_mojom_compatibility.py |  170 ++\n>  ...eck_stable_mojom_compatibility_unittest.py |  260 +++\n>  .../mojo/public/tools/mojom/const_unittest.py |   90 +\n>  .../mojo/public/tools/mojom/enum_unittest.py  |   92 +\n>  .../mojo/public/tools/mojom/mojom/BUILD.gn    |   43 +\n>  .../mojo/public/tools/mojom/mojom/__init__.py |    0\n>  .../mojo/public/tools/mojom/mojom/error.py    |   28 +\n>  .../mojo/public/tools/mojom/mojom/fileutil.py |   45 +\n>  .../tools/mojom/mojom/fileutil_unittest.py    |   40 +\n>  .../tools/mojom/mojom/generate/__init__.py    |    0\n>  .../mojom/mojom/generate/constant_resolver.py |   93 +\n>  .../tools/mojom/mojom/generate/generator.py   |  325 +++\n>  .../mojom/generate/generator_unittest.py      |   74 +\n>  .../tools/mojom/mojom/generate/module.py      | 1635 ++++++++++++++\n>  .../mojom/mojom/generate/module_unittest.py   |   31 +\n>  .../public/tools/mojom/mojom/generate/pack.py |  258 +++\n>  .../mojom/mojom/generate/pack_unittest.py     |  225 ++\n>  .../mojom/mojom/generate/template_expander.py |   83 +\n>  .../tools/mojom/mojom/generate/translate.py   |  854 ++++++++\n>  .../mojom/generate/translate_unittest.py      |   73 +\n>  .../tools/mojom/mojom/parse/__init__.py       |    0\n>  .../public/tools/mojom/mojom/parse/ast.py     |  427 ++++\n>  .../tools/mojom/mojom/parse/ast_unittest.py   |  121 +\n>  .../mojom/mojom/parse/conditional_features.py |   82 +\n>  .../parse/conditional_features_unittest.py    |  233 ++\n>  .../public/tools/mojom/mojom/parse/lexer.py   |  251 +++\n>  .../tools/mojom/mojom/parse/lexer_unittest.py |  198 ++\n>  .../public/tools/mojom/mojom/parse/parser.py  |  488 +++++\n>  .../mojom/mojom/parse/parser_unittest.py      | 1390 ++++++++++++\n>  .../mojo/public/tools/mojom/mojom_parser.py   |  361 +++\n>  .../tools/mojom/mojom_parser_test_case.py     |   73 +\n>  .../tools/mojom/mojom_parser_unittest.py      |  171 ++\n>  .../tools/mojom/stable_attribute_unittest.py  |  127 ++\n>  .../mojom/version_compatibility_unittest.py   |  397 ++++\n>  .../public/tools/run_all_python_unittests.py  |   28 +\n>  utils/ipc/tools/diagnosis/crbug_1001171.py    |   51 +\n>  utils/meson.build                             |    1 +\n>  129 files changed, 17795 insertions(+), 2678 deletions(-)\n>  delete mode 100644 include/libcamera/internal/ipa_context_wrapper.h\n>  create mode 100644 include/libcamera/internal/ipa_data_serializer.h\n>  create mode 100644 include/libcamera/internal/ipa_ipc.h\n>  create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h\n>  create mode 100644 include/libcamera/ipa/core.mojom\n>  create mode 100644 include/libcamera/ipa/raspberrypi.mojom\n>  create mode 100644 include/libcamera/ipa/rkisp1.mojom\n>  rename include/libcamera/ipa/{ipa_vimc.h => vimc.h} (87%)\n>  create mode 100644 include/libcamera/ipa/vimc.mojom\n>  delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp\n>  delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h\n>  delete mode 100644 src/ipa/libipa/meson.build\n>  delete mode 100644 src/libcamera/ipa_context_wrapper.cpp\n>  create mode 100644 src/libcamera/ipa_data_serializer.cpp\n>  create mode 100644 src/libcamera/ipa_ipc.cpp\n>  create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp\n>  delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp\n>  delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp\n>  delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp\n>  delete mode 100644 test/ipa/ipa_wrappers_test.cpp\n>  create mode 100644 test/ipc/unixsocket_ipc.cpp\n>  create mode 100644 test/serialization/ipa_data_serializer_test.cpp\n>  create mode 100755 utils/ipc/generate.py\n>  create mode 100644 utils/ipc/generators/libcamera_templates/meson.build\n>  create mode 100644 utils/ipc/generators/libcamera_templates/module_generated.h.tmpl\n>  create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n>  create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n>  create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n>  create mode 100644 utils/ipc/generators/libcamera_templates/module_serializer.h.tmpl\n>  create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n>  create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl\n>  create mode 100644 utils/ipc/generators/meson.build\n>  create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n>  create mode 100644 utils/ipc/meson.build\n>  create mode 100644 utils/ipc/mojo/public/LICENSE\n>  create mode 100644 utils/ipc/mojo/public/tools/.style.yapf\n>  create mode 100644 utils/ipc/mojo/public/tools/BUILD.gn\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/BUILD.gn\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/README.md\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/chromium_bindings_configuration.gni\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/compile_typescript.py\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate-files.py\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/format_typemap_generator_args.py\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/gen_data_files_list.py\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/generate_type_mappings.py\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom.gni\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py\n>  create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/mojom_types_downgrader.py\n>  create mode 100755 utils/ipc/mojo/public/tools/bindings/validate_typemap_config.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/README.md\n>  create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py\n>  create mode 100755 utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/const_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/enum_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/BUILD.gn\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/__init__.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/error.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/constant_resolver.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py\n>  create mode 100755 utils/ipc/mojo/public/tools/mojom/mojom_parser.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py\n>  create mode 100644 utils/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py\n>  create mode 100755 utils/ipc/mojo/public/tools/run_all_python_unittests.py\n>  create mode 100644 utils/ipc/tools/diagnosis/crbug_1001171.py\n> \n> -- \n> 2.27.0\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9F9DEC3B5B\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue, 22 Sep 2020 13:43:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1FF1C62FE6;\n\tTue, 22 Sep 2020 15:43:06 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5296962FD6\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 22 Sep 2020 15:43:05 +0200 (CEST)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6D948555;\n\tTue, 22 Sep 2020 15:42:56 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"AUWSvlMP\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1600782178;\n\tbh=2h5rnewjMpmcMtDYBwE0kW6idnooVADnUDR2av0rYP0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=AUWSvlMPWxpRPlTT5GGRk3xHwGQ2WUDgmd2q6JaIvGR/uhFesGQYzaKBiOr76QgGk\n\t0Y4Lvs0zzPBxCFAQHuOk5AmSpsk7kr/HU6UKAP6Fctr1sxPE32a8A32vVd3wPWgNCa\n\tTAle3lNYCi7ytkKj7t034oiKfxDPr4i+CgSRY4QY=","Date":"Tue, 22 Sep 2020 22:42:49 +0900","From":"paul.elder@ideasonboard.com","To":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200922134249.GF45948@pyrite.rasen.tech>","References":"<20200922133537.258098-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200922133537.258098-1-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":12902,"web_url":"https://patchwork.libcamera.org/comment/12902/","msgid":"<CANJVT8evZPHy3XAvq50hJ2dA52=vTaA+7PAnyeov9M9cqH69Cg@mail.gmail.com>","date":"2020-09-30T09:43:23","subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","submitter":{"id":72,"url":"https://patchwork.libcamera.org/api/people/72/","name":"Han-lin Chen","email":"hanlinchen@google.com"},"content":"Hi Paul and Laurent,\n\nSome reflection from ChromeOS integration's perspective.\nCurrently, ChromeOS separates HAL and 3A into distinct upstart services.\nThe upstart will launch each one in minijail with different settings.\nThe reason is that we'd like the 3A component to have more restrictions\nthan the HAL process, since it usually contains a closed sourced library.\nThe upstart will launch them separately and they will try to connect to\neach other by a specific named socket.\nOn the socket connection, they communicate by a predefined mojo interface.\nFor vendors, we require the close sourced library to expose symbols\nidentical to the interface.\n\nIn my opinion, ChromeOS doesn't need to stick to mojo communication, and we\ncan apply this solution.\nBut we hope to maintain the two-service structure, and thus something I can\nthink of might influence.\nI don't have a clear opinion for them, but I'd like to add them here for\ndiscussion.\n\n1. Since the 3A component is launched by an independent service and in our\ncase it should be ipa_proxy_worker. ChromeOS needs a way to control the\nlibcamera not to fork the ipa_proxy_worker, but to connect to a predefined\nnamed socket. Maybe by specific build args for ChromeOS or an extended API\nto its hal adaptor.\n2. Because 3A is close-sourced, we expected it to crash for unknown\nreasons. In this case, the upstart service will respawn the 3A process, and\nthe HAL process should be able to detect it and try to reconnect, assuming\n3A is the server side of the socket.\n3. Sometimes the HAL process might crash too :). The best case is that the\nrespawned HAL can reconnect to 3A again. Or for simplicity, 3A can exit()\non disconnection and assume upstart will restart everything.\n4. This one may not be an actual problem but just a concern. For vendors,\nwe require the close sourced library to expose the same symbols defined by\nus.\nThe definition:\nhttps://chromium.googlesource.com/chromiumos/platform2/+/refs/heads/master/camera/include/cros-camera/camera_algorithm.h\nFor existing ChromeOS products, the binaries are deployed and following the\nChromeOS API. On the other hand, libcamera has more freedom on defining the\n3A API. In the ideal situation, we could discuss with every vendor to\nsupport libcamera's API too, but I have to admit that it depends on the\nvendor, and is not always possible. If this happens, and the libcamera IPA\nAPI can be easily translated to ChromeOS IPA API, we can add an\nintermediate layer to adapt the existing binaries, and avoid the problem.\nFor other platforms we don't have products, we don't need to worry about\nit, but It may affect rockchip, ipu3/ipu6, Mediatek and maybe Qualcomm\ndevices.\n\nThanks for the implementation Paul, the idea to reuse mojom parser is so\nclever.\n\nOn Tue, Sep 22, 2020 at 9:43 PM <paul.elder@ideasonboard.com> wrote:\n\n> On Tue, Sep 22, 2020 at 10:34:59PM +0900, Paul Elder wrote:\n> > This patch series depends on the patch series \"Unify utils locations\".\n> >\n> > This patchset implements IPA isolation, and is fully plumbed and\n> > runnable on all pipeline handlers. This also includes code\n> > generators for the IPA proxies, and headers and serializers for custom\n> > data structures and functions. Unlike the RFC, this patch series is\n> > structured like a normal patch series, in order that should be merged.\n> > The only exception is patches 16/38-36/38, which must be squashed to\n> > avoid bisection breakage, and are only kept separate to ease review.\n>\n> Hello all,\n>\n> Im so sorry, I forgot to put the v2 tag on this whole series.\n>\n>\n> Paul\n>\n> > To see samples of generated code (without running this), see the RFC\n> > \"IPA isolation example, with IPC, fully plumbed\".\n> >\n> > To restate the problem, we have two goals:\n> > - to be able to run IPAs isolated in a separate process\n> > - the isolation must be transparent to both the pipeline handler and\n> >   the IPA\n> > During design of the IPC mechanism, we realized that we could support\n> > both custom fuctions and custom data structures, which would be a lot\n> > nicer than the tedious manual de/serialization that we had before with\n> > IPAOperationData.\n> >\n> > The architecture of the whole thing is as follows:\n> >\n> > pipeline handler -> IPAProxyRPi --thread--> IPARPi\n> >                                 \\\n> >                                  \\-> IPAIPC --IPC--> IPAProxyRPiWorker\n> -> IPARPi\n> >\n> > Where the pipeline author defines the interface between the pipeline\n> > handler and the IPA, as well as any data structures. Based on this,\n> > headers, de/serializers, and the proxy (including the worker) will be\n> > generated.\n> >\n> > Patches 1/38~04/38 will be fast tracked, and I plan to merge them\n> > separately from the rest of this series after they are approved in this\n> > round.\n> >\n> > Patch 6/38 imports mojo directly from the chromium repos, with some\n> > big things pruned. Patches 8/38 to 15/38 add the meat of this IPC and\n> > custom IPA interface mechanisms. Patches 16/38 to 36/38 are modifications\n> > in the rest of libcamera to make the whole mechanism work, and must be\n> > squashed together to avoid bisection breakage. Patches 37/38 and 38/38\n> > add tests for the serializer and IPC components of this series.\n> >\n> > Patch 27/38 may be of interest, as it shows how one would write the\n> > interface and data definition. In v2 I have customized the raspberrypi\n> > mojom file to take advantage of the freedom in definition the IPA\n> > interface and data structures (as opposed to v1, where it was\n> > practically a direct translation of IPAOperationData).\n> >\n> > In v2 I have added some rules to the mojom file, such as start(),\n> > stop(), and init() are required, along with the main and callback\n> > interfaces. These are documented in core.mojom in patch 26/38 and\n> > enforced in the code generator in patch 08/38. core.mojom is a new\n> > addition in v2, and is where common mojom \"structs\" (which represent\n> > libcamera objects) and IPA interface definition documentaion are.\n> >\n> > Patch 28/38 is also noteworthy, as it shows how one would use the\n> > interface that has been defined. Since the headers and proxies are\n> > autogenerated, there isn't a patch where the generated code can be seen,\n> > however.\n> >\n> > Patches 29/38 and 30/38 also define mojom files and use them, but are\n> > for vimc and rkisp1, respectively. These will also give an idea on how\n> > the mojom definition looks and how it will be used.\n> >\n> > Changes in v2:\n> > - upgrade documentation\n> > - fix documentation compiling\n> > - plumb the new IPC system through the other pipelines\n> > - fix the issue where editing jinja templates wouldn't trigger\n> >   recompile\n> > - enforce IPA interface rules in code generator\n> > - fix previously-untriggered code generation bugs\n> > - add de/serializers for all primitive types, string, and const\n> >   ControlList\n> > - customized the RPi IPA interface\n> > - add licenses\n> > - add tests\n> >\n> > Paul Elder (38):\n> >   Documentation: coding-style: Document global variable guidelines\n> >   libcamera: ProcessManager: make ProcessManager lifetime explicitly\n> >     managed\n> >   libcamera: PipelineHandler: Move printing pipeline names to\n> >     CameraManager\n> >   libcamera: process: forward-declare EventNotifier\n> >   libcamera: meson: Add internal headers to sources\n> >   utils: ipc: import mojo\n> >   libcamera: Update dep5 to specify license for mojo\n> >   utils: ipc: add templates for code generation for IPC mechanism\n> >   utils: ipc: add generator script\n> >   Documentation: skip generating documentation for generated code\n> >   meson: ipa, proxy: Generate headers and proxy with mojo\n> >   ipa: raspberrypi: meson: Add dependency on generated headers\n> >   libcamera: Add IPADataSerializer\n> >   libcamera: Add IPAIPC\n> >   libcamera: Add IPAIPC implementation based on unix socket\n> >   libcamera: IPAModule: Replace ipa_context with IPAInterface\n> >   libcamera: ipa_context_wrapper: Remove ipa_context_wrapper\n> >   libcamera: IPAProxy: Remove stop() override\n> >   libcamera: IPAProxy: Add isolate parameter to create()\n> >   libcamera: PipelineHandler: Remove IPA from base class\n> >   libcamera: IPAInterface: Remove all functions from IPAInterface\n> >   libcamera: IPAInterface: make ipaCreate return IPAInterface\n> >   libcamera: IPAInterface: remove ipa_context and functions from\n> >     documentation\n> >   libcamera: IPAManager: Fetch IPAProxy corresponding to pipeline\n> >   libcamera: IPAManager: add isolation flag to proxy creation\n> >   ipa: Add core.mojom\n> >   ipa: raspberrypi: Add mojom data definition file\n> >   libcamera: pipeline, ipa: raspberrypi: Use new data definition\n> >   libcamera: pipeline, ipa: vimc: Support the new IPC mechanism\n> >   libcamera: pipeline, ipa: rkisp1: Support the new IPC mechanism\n> >   libcamera: IPAProxy: Remove registration mechanism\n> >   libcamera: proxy: Remove IPAProxyLinux and IPAProxyThread\n> >   libcamera: IPAManager: Make createIPA return proxy directly\n> >   ipa: remove libipa\n> >   tests: ipa_interface_test: Update to use new createIPA\n> >   tests: Remove IPA wrappers test\n> >   tests: Add IPADataSerializer test\n> >   tests: Add test for IPAIPCUnixSocket\n> >\n> >  .reuse/dep5                                   |    6 +\n> >  Documentation/Doxyfile.in                     |    7 +-\n> >  Documentation/coding-style.rst                |   15 +\n> >  Documentation/meson.build                     |    2 -\n> >  .../libcamera/internal/ipa_context_wrapper.h  |   52 -\n> >  .../libcamera/internal/ipa_data_serializer.h  | 1220 +++++++++++\n> >  include/libcamera/internal/ipa_ipc.h          |   41 +\n> >  .../libcamera/internal/ipa_ipc_unixsocket.h   |  115 +\n> >  include/libcamera/internal/ipa_manager.h      |   31 +-\n> >  include/libcamera/internal/ipa_module.h       |    4 +-\n> >  include/libcamera/internal/ipa_proxy.h        |   31 -\n> >  include/libcamera/internal/meson.build        |    1 -\n> >  include/libcamera/internal/pipeline_handler.h |    1 -\n> >  include/libcamera/internal/process.h          |   29 +\n> >  include/libcamera/ipa/core.mojom              |   74 +\n> >  include/libcamera/ipa/ipa_interface.h         |  126 +-\n> >  include/libcamera/ipa/meson.build             |   89 +\n> >  include/libcamera/ipa/raspberrypi.h           |   55 +-\n> >  include/libcamera/ipa/raspberrypi.mojom       |  157 ++\n> >  include/libcamera/ipa/rkisp1.h                |   14 +-\n> >  include/libcamera/ipa/rkisp1.mojom            |   42 +\n> >  include/libcamera/ipa/{ipa_vimc.h => vimc.h}  |    4 +\n> >  include/libcamera/ipa/vimc.mojom              |   12 +\n> >  src/ipa/libipa/ipa_interface_wrapper.cpp      |  285 ---\n> >  src/ipa/libipa/ipa_interface_wrapper.h        |   61 -\n> >  src/ipa/libipa/meson.build                    |   15 -\n> >  src/ipa/meson.build                           |    2 -\n> >  src/ipa/raspberrypi/meson.build               |    4 +-\n> >  src/ipa/raspberrypi/raspberrypi.cpp           |  154 +-\n> >  src/ipa/rkisp1/meson.build                    |    5 +-\n> >  src/ipa/rkisp1/rkisp1.cpp                     |   55 +-\n> >  src/ipa/vimc/meson.build                      |    5 +-\n> >  src/ipa/vimc/vimc.cpp                         |   20 +-\n> >  src/libcamera/camera_manager.cpp              |    5 +\n> >  src/libcamera/ipa_context_wrapper.cpp         |  297 ---\n> >  src/libcamera/ipa_data_serializer.cpp         |  154 ++\n> >  src/libcamera/ipa_interface.cpp               |  517 +----\n> >  src/libcamera/ipa_ipc.cpp                     |  110 +\n> >  src/libcamera/ipa_ipc_unixsocket.cpp          |  175 ++\n> >  src/libcamera/ipa_manager.cpp                 |   47 +-\n> >  src/libcamera/ipa_module.cpp                  |   10 +-\n> >  src/libcamera/ipa_proxy.cpp                   |  101 -\n> >  src/libcamera/meson.build                     |    6 +-\n> >  .../pipeline/raspberrypi/raspberrypi.cpp      |  170 +-\n> >  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |   44 +-\n> >  src/libcamera/pipeline/vimc/vimc.cpp          |    8 +-\n> >  src/libcamera/pipeline_handler.cpp            |   11 -\n> >  src/libcamera/process.cpp                     |   46 +-\n> >  src/libcamera/proxy/ipa_proxy_linux.cpp       |  103 -\n> >  src/libcamera/proxy/ipa_proxy_thread.cpp      |  172 --\n> >  src/libcamera/proxy/meson.build               |   19 +-\n> >  .../proxy/worker/ipa_proxy_linux_worker.cpp   |   90 -\n> >  src/libcamera/proxy/worker/meson.build        |   22 +-\n> >  test/ipa/ipa_interface_test.cpp               |    7 +-\n> >  test/ipa/ipa_wrappers_test.cpp                |  452 ----\n> >  test/ipa/meson.build                          |    5 +-\n> >  test/ipc/meson.build                          |    3 +-\n> >  test/ipc/unixsocket_ipc.cpp                   |  238 ++\n> >  test/log/log_process.cpp                      |    2 +\n> >  test/process/process_test.cpp                 |    2 +\n> >  .../ipa_data_serializer_test.cpp              |  463 ++++\n> >  test/serialization/meson.build                |    3 +-\n> >  utils/ipc/generate.py                         |   21 +\n> >  .../libcamera_templates/meson.build           |   11 +\n> >  .../module_generated.h.tmpl                   |   96 +\n> >  .../module_ipa_proxy.cpp.tmpl                 |  222 ++\n> >  .../module_ipa_proxy.h.tmpl                   |  107 +\n> >  .../module_ipa_proxy_worker.cpp.tmpl          |  178 ++\n> >  .../module_serializer.h.tmpl                  |   44 +\n> >  .../libcamera_templates/proxy_functions.tmpl  |  185 ++\n> >  .../libcamera_templates/serializer.tmpl       |  268 +++\n> >  utils/ipc/generators/meson.build              |    3 +\n> >  .../generators/mojom_libcamera_generator.py   |  452 ++++\n> >  utils/ipc/meson.build                         |   14 +\n> >  utils/ipc/mojo/public/LICENSE                 |   27 +\n> >  utils/ipc/mojo/public/tools/.style.yapf       |    6 +\n> >  utils/ipc/mojo/public/tools/BUILD.gn          |   18 +\n> >  utils/ipc/mojo/public/tools/bindings/BUILD.gn |  108 +\n> >  .../ipc/mojo/public/tools/bindings/README.md  |  816 +++++++\n> >  .../chromium_bindings_configuration.gni       |   51 +\n> >  .../tools/bindings/compile_typescript.py      |   27 +\n> >  .../tools/bindings/concatenate-files.py       |   54 +\n> >  ...concatenate_and_replace_closure_exports.py |   73 +\n> >  .../bindings/format_typemap_generator_args.py |   36 +\n> >  .../tools/bindings/gen_data_files_list.py     |   52 +\n> >  .../tools/bindings/generate_type_mappings.py  |  187 ++\n> >  .../ipc/mojo/public/tools/bindings/mojom.gni  | 1941 +++++++++++++++++\n> >  .../bindings/mojom_bindings_generator.py      |  390 ++++\n> >  .../mojom_bindings_generator_unittest.py      |   62 +\n> >  .../tools/bindings/mojom_types_downgrader.py  |  119 +\n> >  .../tools/bindings/validate_typemap_config.py |   57 +\n> >  utils/ipc/mojo/public/tools/mojom/README.md   |   14 +\n> >  .../mojom/check_stable_mojom_compatibility.py |  170 ++\n> >  ...eck_stable_mojom_compatibility_unittest.py |  260 +++\n> >  .../mojo/public/tools/mojom/const_unittest.py |   90 +\n> >  .../mojo/public/tools/mojom/enum_unittest.py  |   92 +\n> >  .../mojo/public/tools/mojom/mojom/BUILD.gn    |   43 +\n> >  .../mojo/public/tools/mojom/mojom/__init__.py |    0\n> >  .../mojo/public/tools/mojom/mojom/error.py    |   28 +\n> >  .../mojo/public/tools/mojom/mojom/fileutil.py |   45 +\n> >  .../tools/mojom/mojom/fileutil_unittest.py    |   40 +\n> >  .../tools/mojom/mojom/generate/__init__.py    |    0\n> >  .../mojom/mojom/generate/constant_resolver.py |   93 +\n> >  .../tools/mojom/mojom/generate/generator.py   |  325 +++\n> >  .../mojom/generate/generator_unittest.py      |   74 +\n> >  .../tools/mojom/mojom/generate/module.py      | 1635 ++++++++++++++\n> >  .../mojom/mojom/generate/module_unittest.py   |   31 +\n> >  .../public/tools/mojom/mojom/generate/pack.py |  258 +++\n> >  .../mojom/mojom/generate/pack_unittest.py     |  225 ++\n> >  .../mojom/mojom/generate/template_expander.py |   83 +\n> >  .../tools/mojom/mojom/generate/translate.py   |  854 ++++++++\n> >  .../mojom/generate/translate_unittest.py      |   73 +\n> >  .../tools/mojom/mojom/parse/__init__.py       |    0\n> >  .../public/tools/mojom/mojom/parse/ast.py     |  427 ++++\n> >  .../tools/mojom/mojom/parse/ast_unittest.py   |  121 +\n> >  .../mojom/mojom/parse/conditional_features.py |   82 +\n> >  .../parse/conditional_features_unittest.py    |  233 ++\n> >  .../public/tools/mojom/mojom/parse/lexer.py   |  251 +++\n> >  .../tools/mojom/mojom/parse/lexer_unittest.py |  198 ++\n> >  .../public/tools/mojom/mojom/parse/parser.py  |  488 +++++\n> >  .../mojom/mojom/parse/parser_unittest.py      | 1390 ++++++++++++\n> >  .../mojo/public/tools/mojom/mojom_parser.py   |  361 +++\n> >  .../tools/mojom/mojom_parser_test_case.py     |   73 +\n> >  .../tools/mojom/mojom_parser_unittest.py      |  171 ++\n> >  .../tools/mojom/stable_attribute_unittest.py  |  127 ++\n> >  .../mojom/version_compatibility_unittest.py   |  397 ++++\n> >  .../public/tools/run_all_python_unittests.py  |   28 +\n> >  utils/ipc/tools/diagnosis/crbug_1001171.py    |   51 +\n> >  utils/meson.build                             |    1 +\n> >  129 files changed, 17795 insertions(+), 2678 deletions(-)\n> >  delete mode 100644 include/libcamera/internal/ipa_context_wrapper.h\n> >  create mode 100644 include/libcamera/internal/ipa_data_serializer.h\n> >  create mode 100644 include/libcamera/internal/ipa_ipc.h\n> >  create mode 100644 include/libcamera/internal/ipa_ipc_unixsocket.h\n> >  create mode 100644 include/libcamera/ipa/core.mojom\n> >  create mode 100644 include/libcamera/ipa/raspberrypi.mojom\n> >  create mode 100644 include/libcamera/ipa/rkisp1.mojom\n> >  rename include/libcamera/ipa/{ipa_vimc.h => vimc.h} (87%)\n> >  create mode 100644 include/libcamera/ipa/vimc.mojom\n> >  delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.cpp\n> >  delete mode 100644 src/ipa/libipa/ipa_interface_wrapper.h\n> >  delete mode 100644 src/ipa/libipa/meson.build\n> >  delete mode 100644 src/libcamera/ipa_context_wrapper.cpp\n> >  create mode 100644 src/libcamera/ipa_data_serializer.cpp\n> >  create mode 100644 src/libcamera/ipa_ipc.cpp\n> >  create mode 100644 src/libcamera/ipa_ipc_unixsocket.cpp\n> >  delete mode 100644 src/libcamera/proxy/ipa_proxy_linux.cpp\n> >  delete mode 100644 src/libcamera/proxy/ipa_proxy_thread.cpp\n> >  delete mode 100644 src/libcamera/proxy/worker/ipa_proxy_linux_worker.cpp\n> >  delete mode 100644 test/ipa/ipa_wrappers_test.cpp\n> >  create mode 100644 test/ipc/unixsocket_ipc.cpp\n> >  create mode 100644 test/serialization/ipa_data_serializer_test.cpp\n> >  create mode 100755 utils/ipc/generate.py\n> >  create mode 100644 utils/ipc/generators/libcamera_templates/meson.build\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/module_generated.h.tmpl\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/module_serializer.h.tmpl\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> >  create mode 100644\n> utils/ipc/generators/libcamera_templates/serializer.tmpl\n> >  create mode 100644 utils/ipc/generators/meson.build\n> >  create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n> >  create mode 100644 utils/ipc/meson.build\n> >  create mode 100644 utils/ipc/mojo/public/LICENSE\n> >  create mode 100644 utils/ipc/mojo/public/tools/.style.yapf\n> >  create mode 100644 utils/ipc/mojo/public/tools/BUILD.gn\n> >  create mode 100644 utils/ipc/mojo/public/tools/bindings/BUILD.gn\n> >  create mode 100644 utils/ipc/mojo/public/tools/bindings/README.md\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/bindings/chromium_bindings_configuration.gni\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/bindings/compile_typescript.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/concatenate-files.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/format_typemap_generator_args.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/bindings/gen_data_files_list.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/generate_type_mappings.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/bindings/mojom.gni\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/mojom_types_downgrader.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/bindings/validate_typemap_config.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/README.md\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/const_unittest.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/enum_unittest.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/BUILD.gn\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/__init__.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/error.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/fileutil.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/constant_resolver.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/generator.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/module.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/pack.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/translate.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py\n> >  create mode 100644 utils/ipc/mojo/public/tools/mojom/mojom/parse/ast.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/parser.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py\n> >  create mode 100755 utils/ipc/mojo/public/tools/mojom/mojom_parser.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py\n> >  create mode 100644\n> utils/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py\n> >  create mode 100755\n> utils/ipc/mojo/public/tools/run_all_python_unittests.py\n> >  create mode 100644 utils/ipc/tools/diagnosis/crbug_1001171.py\n> >\n> > --\n> > 2.27.0\n> >\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A3DFAC3B5C\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 30 Sep 2020 09:43:37 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 264EC6220F;\n\tWed, 30 Sep 2020 11:43:37 +0200 (CEST)","from mail-wr1-x444.google.com (mail-wr1-x444.google.com\n\t[IPv6:2a00:1450:4864:20::444])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E00C6035F\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 30 Sep 2020 11:43:35 +0200 (CEST)","by mail-wr1-x444.google.com with SMTP id g4so1026113wrs.5\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 30 Sep 2020 02:43:35 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"MZJNaZTY\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=20161025; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=FGYfSjdzcnxo6d8v2/J4dfFDbrsxQtAvG62D+nSlW9M=;\n\tb=MZJNaZTYXJNBykmHvrgu2Jj+Hs9iC8/ehhQ/2rF8ydqlH0/WUCS7Z35qUkGWvigl6Y\n\tx9g9/AKZLKBM+KRmm+8XAI4FFbwfAmP1ChqDJ4vg7kwwPzl8LL6Mt4HIWrUJVoPVIRa0\n\tXwc4tCtysxgPx+g8PBchapha60kevOla50PHx+MjCKQve9ALQMj/ndI5KrlhuS/53XNo\n\t+2A9Epnful2ux1Acb7DPpjlCfyXVKfhvmioGq/4iN6HgxzVnRbHvHHmIGmMKqzQs3nIM\n\tnmeO1yYW5Q2WzqdDpfy54I9sVV/1HFnyKb4GWNjt9DTj9WpzvLnAj49vfJbh0BoUHax3\n\tJUDQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=FGYfSjdzcnxo6d8v2/J4dfFDbrsxQtAvG62D+nSlW9M=;\n\tb=WKMF37mRKz5awfWPugJ/vqBW/xbnDUpYFG5IwvbefuycfYZQj+rKS7eTgvNKjhFyhN\n\trkCLImVXnxZMslAdQjpevB4SbUCEXl9xhGhegdNTYxWdQR8JZ21IE4yP4pi4AbZzKmyt\n\t+csOmAQvjdMu4sLbbbPVbXbpB8uWh8/a7fcXk5hTYymdkySxGAUXX1I6ZtSe+d9nqUKD\n\tb/Q8L2koBuLHapjhAIN2a6/yKGdn1td4TnkTKkk53CViKuc32+/Vt3ZfBbMoiV9T5NUz\n\tbtQL8syeq2lbCYBnZE14hAAzgNy8ajs/ETkDZ332YsOiPzSOsIHI2m4CPzTaaArL0QYQ\n\txSMQ==","X-Gm-Message-State":"AOAM530QN59T7IFkaTA3Tx6gHJ2r8CfyXUJ2Tu7P/lwEHjTCah0v3OFp\n\ta8tABovFmLnk3OuwnmnlUtuuScB8rbzXX9p8MpQLrlfgSWY=","X-Google-Smtp-Source":"ABdhPJwdOy8SnmCxnsGSAt6PPISl4YBSvoxpAzP0PVVNtOrfqv7mdVw92flGXj683uRRGvtG8+VU4QKQyhEkWPzZ9WM=","X-Received":"by 2002:a5d:6283:: with SMTP id k3mr2195632wru.191.1601459014293;\n\tWed, 30 Sep 2020 02:43:34 -0700 (PDT)","MIME-Version":"1.0","References":"<20200922133537.258098-1-paul.elder@ideasonboard.com>\n\t<20200922134249.GF45948@pyrite.rasen.tech>","In-Reply-To":"<20200922134249.GF45948@pyrite.rasen.tech>","From":"Han-lin Chen <hanlinchen@google.com>","Date":"Wed, 30 Sep 2020 17:43:23 +0800","Message-ID":"<CANJVT8evZPHy3XAvq50hJ2dA52=vTaA+7PAnyeov9M9cqH69Cg@mail.gmail.com>","To":"paul.elder@ideasonboard.com","Subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"multipart/mixed;\n\tboundary=\"===============7122216845119067492==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":12965,"web_url":"https://patchwork.libcamera.org/comment/12965/","msgid":"<20201004220253.GM8774@pendragon.ideasonboard.com>","date":"2020-10-04T22:02:53","subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Han-lin,\n\nOn Wed, Sep 30, 2020 at 05:43:23PM +0800, Han-lin Chen wrote:\n> Hi Paul and Laurent,\n> \n> Some reflection from ChromeOS integration's perspective.\n> Currently, ChromeOS separates HAL and 3A into distinct upstart services.\n> The upstart will launch each one in minijail with different settings.\n> The reason is that we'd like the 3A component to have more restrictions than\n> the HAL process, since it usually contains a closed sourced library.\n> The upstart will launch them separately and they will try to connect to each\n> other by a specific named socket.\n> On the socket connection, they communicate by a predefined mojo interface.\n> For vendors, we require the close sourced library to expose symbols identical\n> to the interface.\n> \n> In my opinion, ChromeOS doesn't need to stick to mojo communication, and we can\n> apply this solution.\n> But we hope to maintain the two-service structure, and thus something I can\n> think of might influence.\n> I don't have a clear opinion for them, but I'd like to add them here for\n> discussion.\n>\n> 1. Since the 3A component is launched by an independent service and in our case\n> it should be ipa_proxy_worker. ChromeOS needs a way to control the libcamera\n> not to fork the ipa_proxy_worker, but to connect to a predefined named socket.\n> Maybe by specific build args for ChromeOS or an extended API to its hal\n> adaptor.\n> 2. Because 3A is close-sourced, we expected it to crash for unknown reasons. In\n> this case, the upstart service will respawn the 3A process, and the HAL process\n> should be able to detect it and try to reconnect, assuming 3A is the server\n> side of the socket.\n\nThe 3A process will then lose its state. Does the HAL detect this and\nreinitialize it ?\n\n> 3. Sometimes the HAL process might crash too :). The best case is that the\n> respawned HAL can reconnect to 3A again. Or for simplicity, 3A can exit() on\n> disconnection and assume upstart will restart everything.\n> 4. This one may not be an actual problem but just a concern. For vendors, we\n> require the close sourced library to expose the same symbols defined by us.\n> The definition: https://chromium.googlesource.com/chromiumos/platform2/+/refs/\n> heads/master/camera/include/cros-camera/camera_algorithm.h\n> For existing ChromeOS products, the binaries are deployed and following the\n> ChromeOS API. On the other hand, libcamera has more freedom on defining the 3A\n> API. In the ideal situation, we could discuss with every vendor to support\n> libcamera's API too, but I have to admit that it depends on the vendor, and is\n> not always possible. If this happens, and the libcamera IPA API can be easily\n> translated to ChromeOS IPA API, we can add an intermediate layer to adapt the\n> existing binaries, and avoid the problem.\n\nWe're planning to have a look at this for IPU3, and write an adaptation\nbetween the Chrome OS interface and libcamera. Given that the 3A binary\nfrom Intel is closed-source it will be challenging to debug issues, but\nwe'll do our best :-)\n\n> For other platforms we don't have products, we don't need to worry about it,\n> but It may affect rockchip, ipu3/ipu6, Mediatek and maybe Qualcomm devices.\n\nI agree with you overal. To shed a bit of light on the rationale for the\ndesign, we are envisioning different IPC backends for different\nplatforms. We don't want at this point to implement a camera algorithm\ndaemon specific to libcamera (mostly because it will take time to do\nso), so we have implemented a solution that isolates the IPA module in a\nprocess dynamically forked at runtime. That process is not currently\nsandboxed, but we're considering using minijail for this purpose. Once\nthe initial implementation will be merged, with a functional end-to-end\nIPC mechanism for IPA modules, we will consider extending this with\nother options, such as an IPC mechanism that will communicate with a\ndaemon instead of forking a process.\n\nWould this make sense for you ?\n\n> Thanks for the implementation Paul, the idea to reuse mojom parser is so\n> clever.","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 715EFC3B5D\n\tfor <parsemail@patchwork.libcamera.org>;\n\tSun,  4 Oct 2020 22:03:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 03E9863B82;\n\tMon,  5 Oct 2020 00:03:35 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CA31D6221D\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon,  5 Oct 2020 00:03:32 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3FCF33B;\n\tMon,  5 Oct 2020 00:03:32 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"seZrCDFL\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1601849012;\n\tbh=QK22TWqle7AQ7HHDPGNajUqZQGP9qFhh8VrXfEr0B2Y=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=seZrCDFL0furIs6ceUJXwaH96UMrD2smar2B6kuTNBY6H5j9x9RCuvyyV5dW2QJ7I\n\t/eHgTVyCbgiCvD0y3dJl7L8/bzl2/5j3lWB/f63W0JTJO46gRNlGDziGjV9NnputTE\n\ttOiVsZmz7NOJTZV4fqx5rqw+PtrITDiMoE0WWe0g=","Date":"Mon, 5 Oct 2020 01:02:53 +0300","From":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","To":"Han-lin Chen <hanlinchen@google.com>","Message-ID":"<20201004220253.GM8774@pendragon.ideasonboard.com>","References":"<20200922133537.258098-1-paul.elder@ideasonboard.com>\n\t<20200922134249.GF45948@pyrite.rasen.tech>\n\t<CANJVT8evZPHy3XAvq50hJ2dA52=vTaA+7PAnyeov9M9cqH69Cg@mail.gmail.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<CANJVT8evZPHy3XAvq50hJ2dA52=vTaA+7PAnyeov9M9cqH69Cg@mail.gmail.com>","Subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":12966,"web_url":"https://patchwork.libcamera.org/comment/12966/","msgid":"<CANJVT8ekvvotZoPbHWj7ESVJ1bCBhLpSPWOEQomnBTNCMDYgBw@mail.gmail.com>","date":"2020-10-05T08:32:07","subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","submitter":{"id":72,"url":"https://patchwork.libcamera.org/api/people/72/","name":"Han-lin Chen","email":"hanlinchen@google.com"},"content":"On Mon, Oct 5, 2020 at 6:03 AM Laurent Pinchart\n<laurent.pinchart@ideasonboard.com> wrote:\n>\n> Hi Han-lin,\n>\n> On Wed, Sep 30, 2020 at 05:43:23PM +0800, Han-lin Chen wrote:\n> > Hi Paul and Laurent,\n> >\n> > Some reflection from ChromeOS integration's perspective.\n> > Currently, ChromeOS separates HAL and 3A into distinct upstart services.\n> > The upstart will launch each one in minijail with different settings.\n> > The reason is that we'd like the 3A component to have more restrictions than\n> > the HAL process, since it usually contains a closed sourced library.\n> > The upstart will launch them separately and they will try to connect to each\n> > other by a specific named socket.\n> > On the socket connection, they communicate by a predefined mojo interface.\n> > For vendors, we require the close sourced library to expose symbols identical\n> > to the interface.\n> >\n> > In my opinion, ChromeOS doesn't need to stick to mojo communication, and we can\n> > apply this solution.\n> > But we hope to maintain the two-service structure, and thus something I can\n> > think of might influence.\n> > I don't have a clear opinion for them, but I'd like to add them here for\n> > discussion.\n> >\n> > 1. Since the 3A component is launched by an independent service and in our case\n> > it should be ipa_proxy_worker. ChromeOS needs a way to control the libcamera\n> > not to fork the ipa_proxy_worker, but to connect to a predefined named socket.\n> > Maybe by specific build args for ChromeOS or an extended API to its hal\n> > adaptor.\n> > 2. Because 3A is close-sourced, we expected it to crash for unknown reasons. In\n> > this case, the upstart service will respawn the 3A process, and the HAL process\n> > should be able to detect it and try to reconnect, assuming 3A is the server\n> > side of the socket.\n>\n> The 3A process will then lose its state. Does the HAL detect this and\n> reinitialize it ?\n>\nThe current process is like below:\n1. When the HAL needs to create a connection, it requests the HAL\nAdaptor for mojo setup, and provides a callback.\n2. On mojo connection error (usually means the 3A process crashes),\nthe HAL Adaptor notices HAL the problem.\n3. In theory, HAL could recover the communication by creating the bridge again.\n4. If it's hard to do so, it can notify CAMERA3_MSG_ERROR_DEVICE back\nto HAL Adaptor. The HAL Adaptor will do exit() and hope upstart to\nrespawn both processes. CCA/Chrome should detect it and reconnect to\nthe HAL process when it onlines again.\nIn current implementation, ipu3, rk and mediatek are doing the 4 :-).\n\nOn the other hand, Android framework should treat\nCAMERA3_MSG_ERROR_DEVICE as device closed and try to re-open it again.\nhttps://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera3.h#2035\nMaybe it's an easier way for simplicity.\n\n> > 3. Sometimes the HAL process might crash too :). The best case is that the\n> > respawned HAL can reconnect to 3A again. Or for simplicity, 3A can exit() on\n> > disconnection and assume upstart will restart everything.\n> > 4. This one may not be an actual problem but just a concern. For vendors, we\n> > require the close sourced library to expose the same symbols defined by us.\n> > The definition: https://chromium.googlesource.com/chromiumos/platform2/+/refs/\n> > heads/master/camera/include/cros-camera/camera_algorithm.h\n> > For existing ChromeOS products, the binaries are deployed and following the\n> > ChromeOS API. On the other hand, libcamera has more freedom on defining the 3A\n> > API. In the ideal situation, we could discuss with every vendor to support\n> > libcamera's API too, but I have to admit that it depends on the vendor, and is\n> > not always possible. If this happens, and the libcamera IPA API can be easily\n> > translated to ChromeOS IPA API, we can add an intermediate layer to adapt the\n> > existing binaries, and avoid the problem.\n>\n> We're planning to have a look at this for IPU3, and write an adaptation\n> between the Chrome OS interface and libcamera. Given that the 3A binary\n> from Intel is closed-source it will be challenging to debug issues, but\n> we'll do our best :-)\n>\n> > For other platforms we don't have products, we don't need to worry about it,\n> > but It may affect rockchip, ipu3/ipu6, Mediatek and maybe Qualcomm devices.\n>\n> I agree with you overal. To shed a bit of light on the rationale for the\n> design, we are envisioning different IPC backends for different\n> platforms. We don't want at this point to implement a camera algorithm\n> daemon specific to libcamera (mostly because it will take time to do\n> so), so we have implemented a solution that isolates the IPA module in a\n> process dynamically forked at runtime. That process is not currently\n> sandboxed, but we're considering using minijail for this purpose. Once\n> the initial implementation will be merged, with a functional end-to-end\n> IPC mechanism for IPA modules, we will consider extending this with\n> other options, such as an IPC mechanism that will communicate with a\n> daemon instead of forking a process.\n>\n> Would this make sense for you ?\nTotally! Thanks for the explanation.\n>\n> > Thanks for the implementation Paul, the idea to reuse mojom parser is so\n> > clever.\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n\n\n\n--\nCheers.\nHanlin Chen","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 60C73C3B5C\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon,  5 Oct 2020 08:32:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DCF1763B34;\n\tMon,  5 Oct 2020 10:32:19 +0200 (CEST)","from mail-wr1-x429.google.com (mail-wr1-x429.google.com\n\t[IPv6:2a00:1450:4864:20::429])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DAE0463B19\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon,  5 Oct 2020 10:32:18 +0200 (CEST)","by mail-wr1-x429.google.com with SMTP id o5so8494236wrn.13\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 05 Oct 2020 01:32:18 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=google.com header.i=@google.com\n\theader.b=\"ITuDvzmQ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=20161025; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=K80DHwRXtguVSAxqvCaMJZrdh6JzvepeycUreP2Esg0=;\n\tb=ITuDvzmQE+s0RlwuYQ8CKaCswNCcyT35/ri/lSrM4BkYp+rQVBV3zqDAMIcz4nykNt\n\teEVfT+8IhV0wFfeVwHJQQ0yf+2dTHoOXgfl7w7EXKKATL3Q77H3cMePLvPRAUuPdiPSk\n\tzNkGPs2UYJHKhAcQcGYco1ViPtn6hsLgLw5S1TV+rVcLuwsIbuMVeCxpUGvKQDp9xdUH\n\tyK4ffGLhSZHQh7cskhmlGMma0n/kfjfNvAWQ1xSglJjL+wGIt2ibu5Yk/cEA7WyPt6Ug\n\tXycSkoID5S5OZROCk6vlunbpwZ2DB+QVy0QJQE0WEAhYqKeozQNzhetpZx5Qa56NHm6w\n\tiqeA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=K80DHwRXtguVSAxqvCaMJZrdh6JzvepeycUreP2Esg0=;\n\tb=cJ+s0yq+GrWgbUWPpsxkCIAqZh20hFcD/9VghMxsv2LUFSNQzUT2GDKMoeHGcxXutb\n\tsCjrF2FK6znPa0deMYAzbv1IMHW4fY/is9wrSjge20JL7D+Mca5HpOR1Prlm9nrIKim5\n\tQRPBO4xSl54p3/MUuPSNWiVGS3ELhhW/urgWTGXVFVXCGWYNmeMUBNpRP9Ps4dK4mRHN\n\tmtvdtLLzpS6pgKRFRB86iBI5x+XYVnKtex40oGrGYW31VEiWQUdlXIoHBX8ZtqBMFJBT\n\tvtwNmossLmixiD4YzST6lnwLR77SdoeyefeEdot35BvqeTt7m4YY9vLFBsR5EAds36VJ\n\tF9Rg==","X-Gm-Message-State":"AOAM533FtD6WDC/ErbUmUcwMPmm2kfTCGVIt9B++jjn+VRZlZKOAiJwZ\n\tqFvZcYP1DfaJ12DqshQWYrk3mSuf24Ga2DokUS7S7g==","X-Google-Smtp-Source":"ABdhPJx6P3oDQCQpUBWUzBPJYLktcMjo7MetKJCzEsxONlrG6stlP25nmQNSb01nzU4Jh+inrF04vMleS//Q6xprrwk=","X-Received":"by 2002:a5d:6886:: with SMTP id\n\th6mr16898758wru.374.1601886738115; \n\tMon, 05 Oct 2020 01:32:18 -0700 (PDT)","MIME-Version":"1.0","References":"<20200922133537.258098-1-paul.elder@ideasonboard.com>\n\t<20200922134249.GF45948@pyrite.rasen.tech>\n\t<CANJVT8evZPHy3XAvq50hJ2dA52=vTaA+7PAnyeov9M9cqH69Cg@mail.gmail.com>\n\t<20201004220253.GM8774@pendragon.ideasonboard.com>","In-Reply-To":"<20201004220253.GM8774@pendragon.ideasonboard.com>","From":"Han-lin Chen <hanlinchen@google.com>","Date":"Mon, 5 Oct 2020 16:32:07 +0800","Message-ID":"<CANJVT8ekvvotZoPbHWj7ESVJ1bCBhLpSPWOEQomnBTNCMDYgBw@mail.gmail.com>","To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 00/38] IPA isolation implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]